2024-04-24T22:27:17.110 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:27:17.111 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136 --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/orig.config.yaml 2024-04-24T22:27:17.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:27:17.259 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672136' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnWeREf0wZA9HoL8TVtsgt25Y0k/NF9+50M4c/BKXpkUAQFt27MIJ111hT9PXCx1cxjrtEAB0JlSQg4eIGyVsQ= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB1OAOWMcVcm35UHsST8EUiB/4wum1Or/Nk1DshIz+NJrdH9b2dk8WO7ITaQ7z0jeMK7k2jrgY09ntzMcEWHVhU= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-24T22:27:17.259 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:27:17.260 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:27:17.260 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:27:17.262 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:27:17.285 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:27:17.286 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:27:17.286 INFO:teuthology.packaging:ref: None 2024-04-24T22:27:17.286 INFO:teuthology.packaging:tag: None 2024-04-24T22:27:17.286 INFO:teuthology.packaging:branch: squid 2024-04-24T22:27:17.286 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:27:17.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:27:17.578 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:27:17.580 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:27:17.585 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:27:17.586 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:27:17.650 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:27:17.671 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:27:17.678 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:27:17.703 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:22:04.938762', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnWeREf0wZA9HoL8TVtsgt25Y0k/NF9+50M4c/BKXpkUAQFt27MIJ111hT9PXCx1cxjrtEAB0JlSQg4eIGyVsQ='} 2024-04-24T22:27:17.724 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:22:04.940802', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB1OAOWMcVcm35UHsST8EUiB/4wum1Or/Nk1DshIz+NJrdH9b2dk8WO7ITaQ7z0jeMK7k2jrgY09ntzMcEWHVhU='} 2024-04-24T22:27:17.724 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:27:17.732 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-24T22:27:17.732 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-24T22:27:17.732 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:27:17.808 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb3adb9e3a0>, signals=[15]) 2024-04-24T22:27:17.809 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:27:17.816 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:27:17.817 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:27:17.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:27:17.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-04-24T22:27:17.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:27:17.994 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:27:18.002 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-04-24T22:27:18.021 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-04-24T22:27:18.022 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-04-24T22:27:18.081 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-04-24T22:27:18.081 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-04-24T22:27:18.081 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-04-24T22:27:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:27:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-04-24T22:27:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:27:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:27:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:27:18.083 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:27:18.084 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-04-24T22:27:18.108 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-04-24T22:27:18.125 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-04-24T22:27:18.125 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:27:18.183 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:27:18.184 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:27:18.184 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:27:18.184 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-04-24T22:27:18.208 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:27:18.217 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:27:18.225 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:27:18.225 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:27:18.228 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:27:18.244 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:27:18.252 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:27:18.252 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:27:18.286 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:27:18.378 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:27:18.435 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:27:18.449 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:27:18.450 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T22:27:18.450 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:27:18.451 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:27:18.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:27:18.467 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-04-24T22:27:18.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:27:18.468 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-04-24T22:27:18.524 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:27:18.524 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-437.el9.x86_64 2024-04-24T22:27:18.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-04-24T22:27:18.527 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:27:18.527 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-437.el9.x86_64 2024-04-24T22:27:18.528 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-04-24T22:27:21.393 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-24T22:27:21.712 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-24T22:27:38.668 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-24T22:27:40.146 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-04-24T22:27:42.793 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-24T22:27:44.553 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-24T22:27:44.845 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:01 2024-04-24T22:27:46.563 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-24T22:27:48.206 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T22:27:48.849 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 21 MB 00:04 2024-04-24T22:27:53.464 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T22:27:53.919 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-24T22:27:55.437 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:27:55.437 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:27:55.526 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:27:55.527 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:27:55.528 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:27:55.638 DEBUG:teuthology.orchestra.run.smithi063:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:27:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:27:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:27:55.848 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:27:55.848 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:27:55.849 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:27:55.960 DEBUG:teuthology.orchestra.run.smithi192:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:27:56.158 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:27:53 PM UTC. 2024-04-24T22:27:56.282 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:27:56.293 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.293 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-24T22:27:56.293 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.293 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalling: 2024-04-24T22:27:56.293 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 255 k 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 0 2024-04-24T22:27:56.294 INFO:teuthology.orchestra.run.smithi063.stderr:Operation aborted. 2024-04-24T22:27:56.356 INFO:teuthology.orchestra.run.smithi063.stdout:Is this ok [y/N]: 2024-04-24T22:27:56.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum reinstall -y kernel || true 2024-04-24T22:27:56.455 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:27:53 PM UTC. 2024-04-24T22:27:56.574 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:27:56.617 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:Reinstalling: 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 255 k 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 0 2024-04-24T22:27:56.618 INFO:teuthology.orchestra.run.smithi192.stderr:Operation aborted. 2024-04-24T22:27:56.678 INFO:teuthology.orchestra.run.smithi192.stdout:Is this ok [y/N]: 2024-04-24T22:27:56.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum reinstall -y kernel || true 2024-04-24T22:27:56.869 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:27:53 PM UTC. 2024-04-24T22:27:56.990 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalling: 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 255 k 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 0 2024-04-24T22:27:56.991 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:27:57.174 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 10:27:53 PM UTC. 2024-04-24T22:27:57.266 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-24T22:27:57.267 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:27:57.267 INFO:teuthology.orchestra.run.smithi063.stdout:Total 926 kB/s | 255 kB 00:00 2024-04-24T22:27:57.284 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:27:57.292 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:Reinstalling: 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:57.293 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 255 k 2024-04-24T22:27:57.294 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 0 2024-04-24T22:27:57.294 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:27:57.482 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:27:57.482 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:27:57.487 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:27:57.488 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:27:57.625 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:27:57.700 INFO:teuthology.orchestra.run.smithi063.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:27:57.701 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:58.055 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-437.el9.x86_64.rpm 396 kB/s | 255 kB 00:00 2024-04-24T22:27:58.055 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:27:58.055 INFO:teuthology.orchestra.run.smithi192.stdout:Total 335 kB/s | 255 kB 00:00 2024-04-24T22:27:58.070 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:27:58.256 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:27:58.257 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:27:58.262 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:27:58.262 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:27:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:27:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:27:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:59.032 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:59.033 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:59.033 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalled: 2024-04-24T22:27:59.033 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:27:59.033 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:27:59.033 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:27:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:27:59.215 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:27:59.264 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:27:59.265 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:27:59.265 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:27:59.265 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-24T22:27:59.265 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:27:59.292 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-04-24T22:27:59.313 DEBUG:teuthology.parallel:result is None 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Reinstalled: 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:27:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:27:59.961 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:28:00.008 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:28:00.008 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:28:00.008 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:28:00.008 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-24T22:28:00.009 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:28:00.047 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-04-24T22:28:00.068 DEBUG:teuthology.parallel:result is None 2024-04-24T22:28:00.069 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:28:00.076 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:28:00.076 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:28:00.079 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:28:00.124 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:28:00.130 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:28:00.145 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:28:00.145 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:28:00.148 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:28:00.187 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:28:00.193 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:28:00.194 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:28:00.219 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:28:00.255 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:28:00.266 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:28:00.292 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:28:00.301 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:28:00.303 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:28:00.310 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:28:00.310 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:28:00.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:28:00.338 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:28:00.347 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:28:00.347 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:28:00.381 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:28:00.395 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:28:00.478 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:28:00.525 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:28:00.607 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:28:00.648 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:28:00.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:28:00.712 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:28:00.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:28:00.806 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:28:00.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:28:00.926 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:28:00.927 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:28:00.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-04-24T22:28:00.995 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-04-24T22:28:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:28:01.067 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:28:01.457 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:28:01.466 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:28:01.466 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:28:01.481 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:28:01.490 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:28:01.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-04-24T22:28:01.532 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-04-24T22:28:01.534 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-04-24T22:28:01.590 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-04-24T22:28:01.592 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:28:01.592 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-04-24T22:28:01.614 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-04-24T22:28:01.614 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-04-24T22:28:01.636 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-04-24T22:28:01.636 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:28:01.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:28:01.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:28:01.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:28:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:28:01.704 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:28:01.704 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:28:01.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:28:01.796 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:28:01.807 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T22:28:01.812 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T22:28:01.935 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:28:01.951 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:28:01.953 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:31:35.450 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-04-24T22:31:35.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-04-24T22:31:35.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:31:35.557 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-04-24T22:31:35.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-04-24T22:31:35.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-24T22:31:35.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:31:35.716 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-04-24T22:31:35.755 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-24T22:31:35.755 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:31:35.765 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:31:35.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:31:35.766 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:31:35.768 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:31:35.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:31:35.795 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:31:35.808 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:31:35.822 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:31:35.831 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-04-24T22:31:35.835 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:31:35.842 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-04-24T22:31:35.855 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:31:35.857 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-04-24T22:31:35.867 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-04-24T22:31:35.868 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:31:35.880 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:31:35.893 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:31:35.925 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.929 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.947 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:31:35.950 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:31:35.950 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-04-24T22:31:35.951 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.951 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.951 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.951 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:31:35.951 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:31:35.960 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:31:35.960 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:31:35.960 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:31:35.960 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:31:35.970 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:31:35.971 INFO:teuthology.task.install:extra packages: [] 2024-04-24T22:31:35.971 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:31:35.972 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:31:35.972 INFO:teuthology.packaging:ref: None 2024-04-24T22:31:35.972 INFO:teuthology.packaging:tag: None 2024-04-24T22:31:35.972 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:31:35.972 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:31:35.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T22:31:35.976 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:31:35.976 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:31:35.976 INFO:teuthology.packaging:ref: None 2024-04-24T22:31:35.976 INFO:teuthology.packaging:tag: None 2024-04-24T22:31:35.976 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:31:35.977 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:31:35.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T22:31:36.194 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-24T22:31:36.194 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-24T22:31:36.213 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-24T22:31:36.213 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-24T22:31:36.316 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:31:36.316 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:31:36.316 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:31:36.342 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T22:31:36.342 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:31:36.342 INFO:teuthology.packaging:ref: None 2024-04-24T22:31:36.342 INFO:teuthology.packaging:tag: None 2024-04-24T22:31:36.342 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:31:36.343 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:31:36.343 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:31:36.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:31:36.428 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:31:36.429 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:31:36.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:31:36.456 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T22:31:36.456 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:31:36.456 INFO:teuthology.packaging:ref: None 2024-04-24T22:31:36.456 INFO:teuthology.packaging:tag: None 2024-04-24T22:31:36.456 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:31:36.456 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:31:36.456 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:31:36.484 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:31:36.522 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:31:36.548 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-04-24T22:31:36.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-04-24T22:31:36.598 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:31:36.662 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-04-24T22:31:36.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-04-24T22:31:36.773 INFO:teuthology.orchestra.run.smithi192.stdout:48 files removed 2024-04-24T22:31:36.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-04-24T22:31:36.897 INFO:teuthology.orchestra.run.smithi063.stdout:48 files removed 2024-04-24T22:31:36.924 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw 2024-04-24T22:31:37.602 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 142 kB/s | 77 kB 00:00 2024-04-24T22:31:37.656 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 167 kB/s | 77 kB 00:00 2024-04-24T22:31:38.059 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-04-24T22:31:38.091 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-04-24T22:31:38.416 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 6.2 kB/s | 1.8 kB 00:00 2024-04-24T22:31:38.470 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 4.7 kB/s | 1.8 kB 00:00 2024-04-24T22:31:40.053 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-04-24T22:31:41.558 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-24T22:31:44.325 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-24T22:31:50.960 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-24T22:31:52.661 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-24T22:31:54.728 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:31:57.870 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-24T22:31:59.936 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T22:32:02.177 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:02.179 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:02.179 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:32:02.179 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:02.179 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:32:02.180 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-24T22:32:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout:Install 22 Packages 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 2 Packages 2024-04-24T22:32:02.182 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:02.183 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 58 M 2024-04-24T22:32:02.183 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:32:03.162 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-24T22:32:04.146 INFO:teuthology.orchestra.run.smithi063.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 3.5 MB/s | 5.0 MB 00:01 2024-04-24T22:32:04.221 INFO:teuthology.orchestra.run.smithi063.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 335 kB/s | 25 kB 00:00 2024-04-24T22:32:04.404 INFO:teuthology.orchestra.run.smithi063.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 3.4 MB/s | 637 kB 00:00 2024-04-24T22:32:04.565 INFO:teuthology.orchestra.run.smithi063.stdout:(4/24): ceph-radosgw-17.2.7-902.gc61472c8.el9.x 7.6 MB/s | 14 MB 00:01 2024-04-24T22:32:04.590 INFO:teuthology.orchestra.run.smithi063.stdout:(5/24): libradosstriper1-17.2.7-902.gc61472c8.e 2.4 MB/s | 457 kB 00:00 2024-04-24T22:32:04.749 INFO:teuthology.orchestra.run.smithi063.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 285 kB/s | 45 kB 00:00 2024-04-24T22:32:04.841 INFO:teuthology.orchestra.run.smithi063.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 1.3 MB/s | 117 kB 00:00 2024-04-24T22:32:04.899 INFO:teuthology.orchestra.run.smithi063.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 2.6 MB/s | 156 kB 00:00 2024-04-24T22:32:04.962 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-24T22:32:04.966 INFO:teuthology.orchestra.run.smithi063.stdout:(9/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 12 MB/s | 4.9 MB 00:00 2024-04-24T22:32:04.999 INFO:teuthology.orchestra.run.smithi063.stdout:(10/24): python3-rados-17.2.7-902.gc61472c8.el9 3.4 MB/s | 320 kB 00:00 2024-04-24T22:32:05.042 INFO:teuthology.orchestra.run.smithi063.stdout:(11/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 3.8 MB/s | 293 kB 00:00 2024-04-24T22:32:05.068 INFO:teuthology.orchestra.run.smithi063.stdout:(12/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.3 MB/s | 99 kB 00:00 2024-04-24T22:32:05.326 INFO:teuthology.orchestra.run.smithi063.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 175 kB/s | 45 kB 00:00 2024-04-24T22:32:05.359 INFO:teuthology.orchestra.run.smithi063.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 505 kB/s | 160 kB 00:00 2024-04-24T22:32:05.634 INFO:teuthology.orchestra.run.smithi063.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-04-24T22:32:05.659 INFO:teuthology.orchestra.run.smithi063.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 975 kB/s | 292 kB 00:00 2024-04-24T22:32:05.901 INFO:teuthology.orchestra.run.smithi063.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 158 kB/s | 42 kB 00:00 2024-04-24T22:32:05.935 INFO:teuthology.orchestra.run.smithi063.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-04-24T22:32:06.152 INFO:teuthology.orchestra.run.smithi063.stdout:(19/24): ceph-common-17.2.7-902.gc61472c8.el9.x 6.6 MB/s | 22 MB 00:03 2024-04-24T22:32:06.177 INFO:teuthology.orchestra.run.smithi063.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 176 kB/s | 49 kB 00:00 2024-04-24T22:32:06.202 INFO:teuthology.orchestra.run.smithi063.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-04-24T22:32:06.494 INFO:teuthology.orchestra.run.smithi063.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-24T22:32:06.545 INFO:teuthology.orchestra.run.smithi063.stdout:(23/24): librados2-17.2.7-902.gc61472c8.el9.x86 8.4 MB/s | 3.1 MB 00:00 2024-04-24T22:32:06.586 INFO:teuthology.orchestra.run.smithi063.stdout:(24/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 7.7 MB/s | 2.9 MB 00:00 2024-04-24T22:32:06.588 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:06.589 INFO:teuthology.orchestra.run.smithi063.stdout:Total 13 MB/s | 58 MB 00:04 2024-04-24T22:32:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:32:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:32:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:32:06.975 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:32:07.143 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:32:07.144 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:32:08.398 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:32:08.772 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-24T22:32:08.856 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:09.038 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T22:32:09.058 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:09.129 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:09.184 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-24T22:32:09.212 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-24T22:32:09.670 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-24T22:32:09.694 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T22:32:09.756 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T22:32:09.786 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-24T22:32:09.828 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-24T22:32:09.865 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T22:32:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T22:32:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:10.012 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:10.051 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-24T22:32:10.089 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-24T22:32:10.121 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-24T22:32:10.429 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-24T22:32:10.475 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T22:32:10.548 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T22:32:10.625 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-24T22:32:10.653 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-24T22:32:12.066 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-24T22:32:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:13.743 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:13.786 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T22:32:13.878 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T22:32:13.878 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:32:13.878 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:32:13.878 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:13.912 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:14.191 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-24T22:32:14.193 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-24T22:32:14.194 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:14.195 INFO:teuthology.orchestra.run.smithi192.stdout:Install 22 Packages 2024-04-24T22:32:14.196 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-04-24T22:32:14.196 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:14.196 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 58 M 2024-04-24T22:32:14.196 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:32:16.281 INFO:teuthology.orchestra.run.smithi192.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 3.6 MB/s | 5.0 MB 00:01 2024-04-24T22:32:16.348 INFO:teuthology.orchestra.run.smithi192.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 378 kB/s | 25 kB 00:00 2024-04-24T22:32:16.507 INFO:teuthology.orchestra.run.smithi192.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 3.9 MB/s | 637 kB 00:00 2024-04-24T22:32:16.624 INFO:teuthology.orchestra.run.smithi192.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 3.8 MB/s | 457 kB 00:00 2024-04-24T22:32:16.908 INFO:teuthology.orchestra.run.smithi192.stdout:(5/24): ceph-radosgw-17.2.7-902.gc61472c8.el9.x 7.0 MB/s | 14 MB 00:02 2024-04-24T22:32:17.032 INFO:teuthology.orchestra.run.smithi192.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 362 kB/s | 45 kB 00:00 2024-04-24T22:32:17.149 INFO:teuthology.orchestra.run.smithi192.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 1.0 MB/s | 117 kB 00:00 2024-04-24T22:32:17.250 INFO:teuthology.orchestra.run.smithi192.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 1.5 MB/s | 156 kB 00:00 2024-04-24T22:32:17.342 INFO:teuthology.orchestra.run.smithi192.stdout:(9/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 6.8 MB/s | 4.9 MB 00:00 2024-04-24T22:32:17.459 INFO:teuthology.orchestra.run.smithi192.stdout:(10/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 2.5 MB/s | 293 kB 00:00 2024-04-24T22:32:17.518 INFO:teuthology.orchestra.run.smithi192.stdout:(11/24): python3-rados-17.2.7-902.gc61472c8.el9 1.2 MB/s | 320 kB 00:00 2024-04-24T22:32:17.543 INFO:teuthology.orchestra.run.smithi192.stdout:(12/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.2 MB/s | 99 kB 00:00 2024-04-24T22:32:17.718 INFO:teuthology.orchestra.run.smithi192.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 259 kB/s | 45 kB 00:00 2024-04-24T22:32:17.743 INFO:teuthology.orchestra.run.smithi192.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 711 kB/s | 160 kB 00:00 2024-04-24T22:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-24T22:32:17.969 INFO:teuthology.orchestra.run.smithi192.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 727 kB/s | 42 kB 00:00 2024-04-24T22:32:18.060 INFO:teuthology.orchestra.run.smithi192.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-04-24T22:32:18.344 INFO:teuthology.orchestra.run.smithi192.stdout:(18/24): ceph-common-17.2.7-902.gc61472c8.el9.x 6.5 MB/s | 22 MB 00:03 2024-04-24T22:32:18.370 INFO:teuthology.orchestra.run.smithi192.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 157 kB/s | 49 kB 00:00 2024-04-24T22:32:18.545 INFO:teuthology.orchestra.run.smithi192.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 337 kB/s | 67 kB 00:00 2024-04-24T22:32:18.629 INFO:teuthology.orchestra.run.smithi192.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 468 kB/s | 309 kB 00:00 2024-04-24T22:32:18.712 INFO:teuthology.orchestra.run.smithi192.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-24T22:32:18.962 INFO:teuthology.orchestra.run.smithi192.stdout:(23/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 8.8 MB/s | 2.9 MB 00:00 2024-04-24T22:32:19.067 INFO:teuthology.orchestra.run.smithi192.stdout:(24/24): librados2-17.2.7-902.gc61472c8.el9.x86 5.9 MB/s | 3.1 MB 00:00 2024-04-24T22:32:19.069 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:19.069 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 58 MB 00:04 2024-04-24T22:32:19.132 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:32:19.166 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:32:19.166 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:32:19.601 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:32:19.602 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:32:20.846 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:32:21.233 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-24T22:32:21.290 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:21.366 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:21.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T22:32:21.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:21.581 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:21.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-24T22:32:21.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-24T22:32:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-24T22:32:22.137 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T22:32:22.199 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T22:32:22.228 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-24T22:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-24T22:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T22:32:22.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T22:32:22.406 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-24T22:32:22.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-24T22:32:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-24T22:32:22.876 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-24T22:32:22.918 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T22:32:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T22:32:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-24T22:32:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-04-24T22:32:23.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:25.169 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T22:32:25.248 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T22:32:25.249 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:25.249 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:32:25.249 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:32:25.249 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:32:25.249 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:25.251 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T22:32:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T22:32:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:25.495 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:25.525 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:26.030 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T22:32:26.087 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T22:32:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T22:32:26.179 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:32:26.179 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:32:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:26.713 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:26.713 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:26.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-24T22:32:26.715 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-24T22:32:27.449 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:27.449 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:27.449 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:32:27.450 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:27.451 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:32:27.714 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-test 2024-04-24T22:32:28.249 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:29 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:32:28.372 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:28.372 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:28.372 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:32:28.372 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-04-24T22:32:28.373 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:28.374 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 48 M 2024-04-24T22:32:28.374 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 192 M 2024-04-24T22:32:28.374 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:32:28.787 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-24T22:32:28.829 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 526 kB/s | 163 kB 00:00 2024-04-24T22:32:28.854 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 949 kB/s | 64 kB 00:00 2024-04-24T22:32:30.999 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 19 MB/s | 48 MB 00:02 2024-04-24T22:32:31.001 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:31.002 INFO:teuthology.orchestra.run.smithi063.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-24T22:32:31.029 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:32:31.049 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:32:31.049 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:32:31.432 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:32:31.432 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:32:31.973 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:32:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T22:32:32.099 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T22:32:32.125 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-04-24T22:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-04-24T22:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-04-24T22:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:36.740 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T22:32:36.815 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T22:32:36.864 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:32:36.933 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:36.935 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T22:32:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T22:32:37.009 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:38.292 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-24T22:32:38.293 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-24T22:32:38.294 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T22:32:38.295 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-24T22:32:38.295 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-24T22:32:38.954 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T22:32:38.955 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:38.955 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-04-24T22:32:38.955 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.955 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.955 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:32:38.956 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:38.958 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:32:38.971 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T22:32:38.971 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-24T22:32:38.971 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-24T22:32:38.971 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T22:32:39.266 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:32:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:32:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:39.377 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:32:39.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph 2024-04-24T22:32:39.856 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:27 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:32:39.977 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:32:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:39.979 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-04-24T22:32:39.979 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:39.979 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 48 M 2024-04-24T22:32:39.979 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 192 M 2024-04-24T22:32:39.979 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:32:40.174 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:41 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:32:40.257 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-04-24T22:32:40.301 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-24T22:32:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:32:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:32:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:Install 33 Packages 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 27 M 2024-04-24T22:32:40.307 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 96 M 2024-04-24T22:32:40.308 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:32:40.324 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-04-24T22:32:40.887 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 202 kB/s | 163 kB 00:00 2024-04-24T22:32:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 18 kB/s | 6.5 kB 00:00 2024-04-24T22:32:41.907 INFO:teuthology.orchestra.run.smithi063.stdout:(2/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 1.2 MB/s | 1.4 MB 00:01 2024-04-24T22:32:42.615 INFO:teuthology.orchestra.run.smithi063.stdout:(3/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_6 2.6 MB/s | 3.9 MB 00:01 2024-04-24T22:32:42.749 INFO:teuthology.orchestra.run.smithi063.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 1.8 MB/s | 240 kB 00:00 2024-04-24T22:32:42.866 INFO:teuthology.orchestra.run.smithi063.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-24T22:32:42.974 INFO:teuthology.orchestra.run.smithi063.stdout:(6/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 962 kB/s | 2.1 MB 00:02 2024-04-24T22:32:43.082 INFO:teuthology.orchestra.run.smithi063.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 5.7 MB/s | 1.2 MB 00:00 2024-04-24T22:32:43.108 INFO:teuthology.orchestra.run.smithi063.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 799 kB/s | 106 kB 00:00 2024-04-24T22:32:43.225 INFO:teuthology.orchestra.run.smithi063.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 954 kB/s | 135 kB 00:00 2024-04-24T22:32:43.250 INFO:teuthology.orchestra.run.smithi063.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-04-24T22:32:43.350 INFO:teuthology.orchestra.run.smithi063.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-04-24T22:32:43.500 INFO:teuthology.orchestra.run.smithi063.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 232 kB/s | 35 kB 00:00 2024-04-24T22:32:43.542 INFO:teuthology.orchestra.run.smithi063.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 647 kB/s | 172 kB 00:00 2024-04-24T22:32:43.584 INFO:teuthology.orchestra.run.smithi063.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-04-24T22:32:43.610 INFO:teuthology.orchestra.run.smithi063.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 620 kB/s | 42 kB 00:00 2024-04-24T22:32:43.768 INFO:teuthology.orchestra.run.smithi063.stdout:(16/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_ 8.2 MB/s | 15 MB 00:01 2024-04-24T22:32:44.035 INFO:teuthology.orchestra.run.smithi063.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 96 kB/s | 43 kB 00:00 2024-04-24T22:32:44.110 INFO:teuthology.orchestra.run.smithi063.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 343 kB/s | 172 kB 00:00 2024-04-24T22:32:44.178 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 12 MB/s | 48 MB 00:04 2024-04-24T22:32:44.179 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:44.180 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 48 MB 00:04 2024-04-24T22:32:44.202 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:32:44.219 INFO:teuthology.orchestra.run.smithi063.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 58 kB/s | 11 kB 00:00 2024-04-24T22:32:44.219 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:32:44.219 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:32:44.353 INFO:teuthology.orchestra.run.smithi063.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 614 kB/s | 358 kB 00:00 2024-04-24T22:32:44.378 INFO:teuthology.orchestra.run.smithi063.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 66 kB/s | 18 kB 00:00 2024-04-24T22:32:44.486 INFO:teuthology.orchestra.run.smithi063.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 87 kB/s | 23 kB 00:00 2024-04-24T22:32:44.579 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:32:44.579 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:32:44.611 INFO:teuthology.orchestra.run.smithi063.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 75 kB/s | 19 kB 00:00 2024-04-24T22:32:44.645 INFO:teuthology.orchestra.run.smithi063.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-04-24T22:32:44.837 INFO:teuthology.orchestra.run.smithi063.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 132 kB/s | 46 kB 00:00 2024-04-24T22:32:44.954 INFO:teuthology.orchestra.run.smithi063.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-04-24T22:32:45.096 INFO:teuthology.orchestra.run.smithi063.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 63 kB/s | 16 kB 00:00 2024-04-24T22:32:45.105 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:32:45.155 INFO:teuthology.orchestra.run.smithi063.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 534 kB/s | 272 kB 00:00 2024-04-24T22:32:45.187 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T22:32:45.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T22:32:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T22:32:45.297 INFO:teuthology.orchestra.run.smithi063.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 263 kB/s | 90 kB 00:00 2024-04-24T22:32:45.364 INFO:teuthology.orchestra.run.smithi063.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 134 kB/s | 36 kB 00:00 2024-04-24T22:32:45.631 INFO:teuthology.orchestra.run.smithi063.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 75 kB/s | 20 kB 00:00 2024-04-24T22:32:45.656 INFO:teuthology.orchestra.run.smithi063.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 459 kB/s | 230 kB 00:00 2024-04-24T22:32:45.881 INFO:teuthology.orchestra.run.smithi063.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 732 kB/s | 427 kB 00:00 2024-04-24T22:32:45.884 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:45.884 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.9 MB/s | 27 MB 00:05 2024-04-24T22:32:46.007 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:32:46.049 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:32:46.049 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:32:46.334 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:32:46.334 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:32:47.497 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:32:47.559 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T22:32:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T22:32:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T22:32:47.721 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T22:32:47.806 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T22:32:47.852 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T22:32:47.887 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T22:32:47.916 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T22:32:47.951 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T22:32:47.984 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T22:32:48.025 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T22:32:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T22:32:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T22:32:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T22:32:48.209 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T22:32:48.369 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T22:32:48.460 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T22:32:48.511 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T22:32:48.548 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T22:32:48.607 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T22:32:48.671 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T22:32:48.877 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T22:32:48.927 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T22:32:49.009 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T22:32:49.107 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T22:32:49.159 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T22:32:49.281 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T22:32:49.446 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-24T22:32:49.550 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T22:32:49.550 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T22:32:49.550 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:49.551 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:32:49.552 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:32:49.552 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:32:49.552 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:49.629 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T22:32:49.734 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T22:32:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T22:32:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T22:32:50.935 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:50.935 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:32:50.935 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:32:50.935 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:32:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T22:32:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T22:32:51.355 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:51.356 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:32:51.356 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:32:51.356 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:32:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:51.564 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T22:32:51.589 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T22:32:51.589 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:51.589 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:32:51.589 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:32:51.590 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:32:51.590 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:51.654 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T22:32:51.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T22:32:51.844 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-24T22:32:51.844 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-24T22:32:51.844 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:52.147 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:32:52.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-04-24T22:32:52.926 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:40 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:32:53.053 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:32:53.056 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:32:53.056 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:53.056 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:32:53.056 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:32:53.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:32:53.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:32:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:32:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:32:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:32:53.059 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:32:53.060 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:32:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:32:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:32:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:32:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:32:53.061 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout:Install 33 Packages 2024-04-24T22:32:53.062 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:32:53.063 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 27 M 2024-04-24T22:32:53.063 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 96 M 2024-04-24T22:32:53.063 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-24T22:32:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T22:32:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T22:32:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T22:32:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T22:32:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T22:32:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T22:32:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T22:32:53.762 INFO:teuthology.orchestra.run.smithi192.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 15 kB/s | 6.5 kB 00:00 2024-04-24T22:32:54.138 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:32:54.139 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:32:54.140 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:54.141 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:32:54.296 INFO:teuthology.orchestra.run.smithi192.stdout:(2/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 1.5 MB/s | 1.4 MB 00:00 2024-04-24T22:32:54.438 INFO:teuthology.orchestra.run.smithi192.stdout:(3/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 1.9 MB/s | 2.1 MB 00:01 2024-04-24T22:32:54.458 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-base 2024-04-24T22:32:54.563 INFO:teuthology.orchestra.run.smithi192.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 1.9 MB/s | 240 kB 00:00 2024-04-24T22:32:54.688 INFO:teuthology.orchestra.run.smithi192.stdout:(5/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_6 4.2 MB/s | 3.9 MB 00:00 2024-04-24T22:32:54.738 INFO:teuthology.orchestra.run.smithi192.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-24T22:32:54.838 INFO:teuthology.orchestra.run.smithi192.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-24T22:32:54.913 INFO:teuthology.orchestra.run.smithi192.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 5.4 MB/s | 1.2 MB 00:00 2024-04-24T22:32:54.989 INFO:teuthology.orchestra.run.smithi192.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 902 kB/s | 135 kB 00:00 2024-04-24T22:32:54.992 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:55 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:32:55.048 INFO:teuthology.orchestra.run.smithi192.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 940 kB/s | 125 kB 00:00 2024-04-24T22:32:55.073 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:32:55.115 INFO:teuthology.orchestra.run.smithi192.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-04-24T22:32:55.115 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:55.116 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:32:55.117 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:32:55.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install cephadm 2024-04-24T22:32:55.248 INFO:teuthology.orchestra.run.smithi192.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 261 kB/s | 35 kB 00:00 2024-04-24T22:32:55.282 INFO:teuthology.orchestra.run.smithi192.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 739 kB/s | 172 kB 00:00 2024-04-24T22:32:55.440 INFO:teuthology.orchestra.run.smithi192.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 266 kB/s | 51 kB 00:00 2024-04-24T22:32:55.474 INFO:teuthology.orchestra.run.smithi192.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-04-24T22:32:55.599 INFO:teuthology.orchestra.run.smithi192.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 274 kB/s | 43 kB 00:00 2024-04-24T22:32:55.683 INFO:teuthology.orchestra.run.smithi192.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 823 kB/s | 172 kB 00:00 2024-04-24T22:32:55.699 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:56 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:32:55.823 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:55.824 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 82 k 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 372 k 2024-04-24T22:32:55.825 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:32:55.850 INFO:teuthology.orchestra.run.smithi192.stdout:(18/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_ 9.9 MB/s | 15 MB 00:01 2024-04-24T22:32:55.900 INFO:teuthology.orchestra.run.smithi192.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 49 kB/s | 11 kB 00:00 2024-04-24T22:32:55.925 INFO:teuthology.orchestra.run.smithi192.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-04-24T22:32:55.959 INFO:teuthology.orchestra.run.smithi192.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 997 kB/s | 358 kB 00:00 2024-04-24T22:32:55.984 INFO:teuthology.orchestra.run.smithi192.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 332 kB/s | 19 kB 00:00 2024-04-24T22:32:56.018 INFO:teuthology.orchestra.run.smithi192.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-04-24T22:32:56.043 INFO:teuthology.orchestra.run.smithi192.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 792 kB/s | 46 kB 00:00 2024-04-24T22:32:56.068 INFO:teuthology.orchestra.run.smithi192.stdout:(25/33): python3-jaraco-collections-3.0.0-8.el9 138 kB/s | 23 kB 00:00 2024-04-24T22:32:56.102 INFO:teuthology.orchestra.run.smithi192.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 942 kB/s | 79 kB 00:00 2024-04-24T22:32:56.136 INFO:teuthology.orchestra.run.smithi192.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-24T22:32:56.161 INFO:teuthology.orchestra.run.smithi192.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-04-24T22:32:56.187 INFO:teuthology.orchestra.run.smithi192.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-04-24T22:32:56.211 INFO:teuthology.orchestra.run.smithi192.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-04-24T22:32:56.279 INFO:teuthology.orchestra.run.smithi192.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-04-24T22:32:56.304 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 171 kB/s | 82 kB 00:00 2024-04-24T22:32:56.305 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:56.305 INFO:teuthology.orchestra.run.smithi063.stdout:Total 171 kB/s | 82 kB 00:00 2024-04-24T22:32:56.305 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:32:56.311 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:32:56.311 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:32:56.312 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:32:56.313 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:32:56.329 INFO:teuthology.orchestra.run.smithi192.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-24T22:32:56.362 INFO:teuthology.orchestra.run.smithi192.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2024-04-24T22:32:56.365 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:56.365 INFO:teuthology.orchestra.run.smithi192.stdout:Total 8.2 MB/s | 27 MB 00:03 2024-04-24T22:32:56.382 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:32:56.483 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:32:56.519 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:32:56.519 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:32:56.784 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:32:56.784 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:32:56.787 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:32:56.901 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:32:57.176 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:57.416 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:32:57.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T22:32:57.920 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:32:57.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T22:32:58.013 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T22:32:58.046 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T22:32:58.130 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:59 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:32:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T22:32:58.224 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T22:32:58.254 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:32:58.255 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:58.255 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 137 k 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 379 k 2024-04-24T22:32:58.256 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:32:58.274 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T22:32:58.310 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T22:32:58.339 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T22:32:58.365 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T22:32:58.390 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T22:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T22:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T22:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T22:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T22:32:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T22:32:58.749 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T22:32:58.830 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T22:32:58.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T22:32:58.910 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 210 kB/s | 137 kB 00:00 2024-04-24T22:32:58.910 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:32:58.911 INFO:teuthology.orchestra.run.smithi063.stdout:Total 210 kB/s | 137 kB 00:00 2024-04-24T22:32:58.911 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:32:58.919 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:32:58.919 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:32:58.922 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T22:32:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T22:32:58.978 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:32:58.979 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:32:59.034 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T22:32:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:32:59.193 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:32:59.219 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:32:59.219 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:59.219 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:32:59.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:32:59.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T22:32:59.276 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T22:32:59.345 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T22:32:59.433 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T22:32:59.466 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T22:32:59.582 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T22:32:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-24T22:32:59.769 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:32:59.794 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:00.115 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:00.345 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr 2024-04-24T22:33:00.876 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:01 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:00.957 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:00.999 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:01.000 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:01.001 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:01.065 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T22:33:01.103 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:33:01.129 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:01.495 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:33:01.519 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:01.597 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:02 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:01.721 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T22:33:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:01.723 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:01.723 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:33:01.723 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout:Install 7 Packages 2024-04-24T22:33:01.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:01.725 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 3.7 M 2024-04-24T22:33:01.725 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 80 M 2024-04-24T22:33:01.725 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:33:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T22:33:02.354 INFO:teuthology.orchestra.run.smithi063.stdout:(1/7): ceph-prometheus-alerts-17.2.7-902.gc6147 27 kB/s | 15 kB 00:00 2024-04-24T22:33:02.413 INFO:teuthology.orchestra.run.smithi063.stdout:(2/7): ceph-grafana-dashboards-17.2.7-902.gc614 40 kB/s | 24 kB 00:00 2024-04-24T22:33:02.538 INFO:teuthology.orchestra.run.smithi063.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 49 kB/s | 9.0 kB 00:00 2024-04-24T22:33:02.572 INFO:teuthology.orchestra.run.smithi063.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 308 kB/s | 41 kB 00:00 2024-04-24T22:33:02.622 INFO:teuthology.orchestra.run.smithi063.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 367 kB/s | 31 kB 00:00 2024-04-24T22:33:02.680 INFO:teuthology.orchestra.run.smithi063.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T22:33:03.258 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T22:33:03.259 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T22:33:03.260 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T22:33:03.260 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T22:33:03.261 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T22:33:03.481 INFO:teuthology.orchestra.run.smithi063.stdout:(7/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 2.0 MB/s | 3.4 MB 00:01 2024-04-24T22:33:03.485 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:03.485 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.1 MB/s | 3.7 MB 00:01 2024-04-24T22:33:03.485 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:03.496 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:03.496 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:03.567 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:03.569 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:03.892 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:03.957 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T22:33:03.992 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T22:33:04.039 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T22:33:04.069 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:33:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:33:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:33:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:04.084 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:04.098 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-24T22:33:04.449 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-04-24T22:33:04.963 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:52 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:05.042 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:05.082 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:05.083 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:33:05.083 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:05.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-04-24T22:33:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-24T22:33:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T22:33:05.644 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:53 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:05.765 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-24T22:33:05.766 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 82 k 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 372 k 2024-04-24T22:33:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T22:33:06.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T22:33:06.295 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 155 kB/s | 82 kB 00:00 2024-04-24T22:33:06.296 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:06.296 INFO:teuthology.orchestra.run.smithi192.stdout:Total 155 kB/s | 82 kB 00:00 2024-04-24T22:33:06.296 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:06.301 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:06.302 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:06.303 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:06.304 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:06.377 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:06.517 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:06.709 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T22:33:06.781 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:33:06.864 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:33:07.184 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:33:07.250 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:08 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:07.376 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:07.377 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-04-24T22:33:07.377 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:33:07.378 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout:Install 12 Packages 2024-04-24T22:33:07.379 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:07.380 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 38 M 2024-04-24T22:33:07.380 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 209 M 2024-04-24T22:33:07.380 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:07.430 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T22:33:07.431 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:07.431 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:07.431 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:07.431 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:07.431 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:07.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T22:33:07.780 INFO:teuthology.orchestra.run.smithi063.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T22:33:07.813 INFO:teuthology.orchestra.run.smithi063.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-24T22:33:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 127 kB/s | 33 kB 00:00 2024-04-24T22:33:08.139 INFO:teuthology.orchestra.run.smithi063.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-04-24T22:33:08.152 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:56 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:08.272 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:08.272 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 137 k 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 379 k 2024-04-24T22:33:08.273 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:08.331 INFO:teuthology.orchestra.run.smithi063.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 184 kB/s | 35 kB 00:00 2024-04-24T22:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 380 kB/s | 137 kB 00:00 2024-04-24T22:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout:Total 378 kB/s | 137 kB 00:00 2024-04-24T22:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:08.700 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:08.700 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:08.716 INFO:teuthology.orchestra.run.smithi063.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-90 6.6 MB/s | 7.4 MB 00:01 2024-04-24T22:33:08.773 INFO:teuthology.orchestra.run.smithi063.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.1 MB/s | 3.0 MB 00:00 2024-04-24T22:33:08.846 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:08.927 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:33:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:33:08.952 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:33:08.952 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:33:08.952 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:09.215 INFO:teuthology.orchestra.run.smithi063.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.4 MB/s | 4.7 MB 00:00 2024-04-24T22:33:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-04-24T22:33:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:(10/12): python3-devel-3.9.18-3.el9.x86_64.rpm 385 kB/s | 244 kB 00:00 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:09.855 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:10.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-04-24T22:33:10.586 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:58 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:10.665 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:10.709 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:10.709 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:33:10.709 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:10.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T22:33:11.282 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:59 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-24T22:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout:Install 7 Packages 2024-04-24T22:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.7 M 2024-04-24T22:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 80 M 2024-04-24T22:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:11.730 INFO:teuthology.orchestra.run.smithi192.stdout:(1/7): ceph-prometheus-alerts-17.2.7-902.gc6147 60 kB/s | 15 kB 00:00 2024-04-24T22:33:11.755 INFO:teuthology.orchestra.run.smithi192.stdout:(2/7): ceph-grafana-dashboards-17.2.7-902.gc614 91 kB/s | 24 kB 00:00 2024-04-24T22:33:11.924 INFO:teuthology.orchestra.run.smithi192.stdout:(3/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 7.7 MB/s | 3.4 MB 00:00 2024-04-24T22:33:12.058 INFO:teuthology.orchestra.run.smithi192.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-04-24T22:33:12.224 INFO:teuthology.orchestra.run.smithi192.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 102 kB/s | 31 kB 00:00 2024-04-24T22:33:12.249 INFO:teuthology.orchestra.run.smithi192.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 83 kB/s | 41 kB 00:00 2024-04-24T22:33:12.457 INFO:teuthology.orchestra.run.smithi192.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 470 kB/s | 188 kB 00:00 2024-04-24T22:33:12.459 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:12.460 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.5 MB/s | 3.7 MB 00:01 2024-04-24T22:33:12.460 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:12.469 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:12.469 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:12.533 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:12.535 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:12.883 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T22:33:12.917 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T22:33:12.963 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T22:33:13.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T22:33:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-24T22:33:13.423 INFO:teuthology.orchestra.run.smithi063.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.1 MB/s | 5.1 MB 00:04 2024-04-24T22:33:14.051 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-24T22:33:14.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T22:33:15.148 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T22:33:15.387 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:15.388 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:15.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T22:33:15.715 INFO:teuthology.orchestra.run.smithi063.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.5 MB/s | 16 MB 00:06 2024-04-24T22:33:15.716 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:15.717 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.6 MB/s | 38 MB 00:08 2024-04-24T22:33:16.081 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:16.109 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:16.109 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:16.119 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:04 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:16.242 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:33:16.243 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-04-24T22:33:16.244 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:16.245 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 38 M 2024-04-24T22:33:16.245 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 209 M 2024-04-24T22:33:16.245 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:16.544 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:16.544 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:16.723 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-04-24T22:33:16.782 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-24T22:33:16.982 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 126 kB/s | 33 kB 00:00 2024-04-24T22:33:17.082 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-04-24T22:33:17.183 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-04-24T22:33:17.335 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:17.350 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-90 9.7 MB/s | 7.4 MB 00:00 2024-04-24T22:33:17.412 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T22:33:17.456 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T22:33:17.560 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T22:33:17.598 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T22:33:17.783 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T22:33:17.830 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T22:33:17.910 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T22:33:17.958 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 401 kB/s | 244 kB 00:00 2024-04-24T22:33:18.017 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.7 MB/s | 4.7 MB 00:00 2024-04-24T22:33:18.084 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.3 MB/s | 3.0 MB 00:01 2024-04-24T22:33:18.367 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 715 kB/s | 250 kB 00:00 2024-04-24T22:33:18.590 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T22:33:18.722 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T22:33:20.020 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T22:33:21.112 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T22:33:21.145 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T22:33:21.234 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.6 MB/s | 5.1 MB 00:03 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T22:33:23.704 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T22:33:23.705 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T22:33:23.705 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:33:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:33:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:33:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:33:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:33:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:24.065 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:24.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-rook 2024-04-24T22:33:24.954 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:25 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:25.079 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:33:25.080 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:25.081 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:25.082 INFO:teuthology.orchestra.run.smithi063.stdout:Install 13 Packages 2024-04-24T22:33:25.082 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:25.082 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.2 M 2024-04-24T22:33:25.083 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 26 M 2024-04-24T22:33:25.083 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:25.620 INFO:teuthology.orchestra.run.smithi063.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 97 kB/s | 26 kB 00:00 2024-04-24T22:33:25.654 INFO:teuthology.orchestra.run.smithi063.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 64 kB/s | 19 kB 00:00 2024-04-24T22:33:25.679 INFO:teuthology.orchestra.run.smithi063.stdout:(3/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 147 kB/s | 47 kB 00:00 2024-04-24T22:33:25.995 INFO:teuthology.orchestra.run.smithi063.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 465 kB/s | 159 kB 00:00 2024-04-24T22:33:26.079 INFO:teuthology.orchestra.run.smithi063.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 522 kB/s | 222 kB 00:00 2024-04-24T22:33:26.104 INFO:teuthology.orchestra.run.smithi063.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 493 kB/s | 54 kB 00:00 2024-04-24T22:33:26.263 INFO:teuthology.orchestra.run.smithi063.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 477 kB/s | 279 kB 00:00 2024-04-24T22:33:26.455 INFO:teuthology.orchestra.run.smithi063.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 43 kB/s | 15 kB 00:00 2024-04-24T22:33:26.522 INFO:teuthology.orchestra.run.smithi063.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 73 kB/s | 32 kB 00:00 2024-04-24T22:33:26.741 INFO:teuthology.orchestra.run.smithi063.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 270 kB/s | 59 kB 00:00 2024-04-24T22:33:26.864 INFO:teuthology.orchestra.run.smithi063.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 342 kB/s | 206 kB 00:00 2024-04-24T22:33:26.897 INFO:teuthology.orchestra.run.smithi063.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 574 kB/s | 90 kB 00:00 2024-04-24T22:33:27.023 INFO:teuthology.orchestra.run.smithi063.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-04-24T22:33:27.026 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:27.026 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.1 MB/s | 2.2 MB 00:01 2024-04-24T22:33:27.097 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:27.114 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:27.115 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:27.208 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:27.209 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:27.795 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:27.958 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T22:33:28.027 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T22:33:28.082 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T22:33:28.126 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T22:33:28.168 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T22:33:28.275 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T22:33:28.386 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T22:33:28.445 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T22:33:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T22:33:29.283 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T22:33:29.316 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T22:33:29.362 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T22:33:29.385 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T22:33:30.166 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T22:33:30.167 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:33:30.474 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:33:30.475 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:33:30.475 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:33:30.475 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:33:30.475 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:33:30.475 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:33:30.476 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:33:30.476 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:33:30.476 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:33:30.476 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:30.476 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:30.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T22:33:31.233 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:32 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:31.361 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:33:31.363 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout:Install 6 Packages 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 7.0 M 2024-04-24T22:33:31.364 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 33 M 2024-04-24T22:33:31.365 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:31.999 INFO:teuthology.orchestra.run.smithi063.stdout:(1/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 386 kB/s | 123 kB 00:00 2024-04-24T22:33:32.266 INFO:teuthology.orchestra.run.smithi063.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 422 kB/s | 248 kB 00:00 2024-04-24T22:33:32.533 INFO:teuthology.orchestra.run.smithi063.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-04-24T22:33:32.566 INFO:teuthology.orchestra.run.smithi063.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 192 kB/s | 58 kB 00:00 2024-04-24T22:33:32.625 INFO:teuthology.orchestra.run.smithi063.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 555 kB/s | 51 kB 00:00 2024-04-24T22:33:33.342 INFO:teuthology.orchestra.run.smithi063.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-04-24T22:33:33.342 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:33.343 INFO:teuthology.orchestra.run.smithi063.stdout:Total 3.5 MB/s | 7.0 MB 00:01 2024-04-24T22:33:33.416 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:33.529 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:33.530 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T22:33:34.061 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T22:33:34.614 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T22:33:34.701 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T22:33:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T22:33:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T22:33:35.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T22:33:35.779 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T22:33:35.779 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:35.780 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:36.051 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-fuse 2024-04-24T22:33:36.583 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:37 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-24T22:33:36.707 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout:Install 2 Packages 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:36.708 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 894 k 2024-04-24T22:33:36.709 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 2.6 M 2024-04-24T22:33:36.709 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:36.907 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 884 kB/s | 16 MB 00:18 2024-04-24T22:33:36.908 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:36.909 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.8 MB/s | 38 MB 00:20 2024-04-24T22:33:37.103 INFO:teuthology.orchestra.run.smithi063.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 654 kB/s | 80 kB 00:00 2024-04-24T22:33:37.261 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:37.287 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:37.287 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:37.370 INFO:teuthology.orchestra.run.smithi063.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 2.0 MB/s | 814 kB 00:00 2024-04-24T22:33:37.371 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:37.371 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.3 MB/s | 894 kB 00:00 2024-04-24T22:33:37.384 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:37.407 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:37.407 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:37.488 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:37.488 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:37.682 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:37.690 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:37.691 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T22:33:37.896 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T22:33:38.444 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:38.503 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T22:33:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T22:33:38.578 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T22:33:38.578 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-24T22:33:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T22:33:38.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:38.830 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:38.860 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T22:33:38.888 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T22:33:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T22:33:39.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados-devel 2024-04-24T22:33:39.625 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T22:33:39.751 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:39.752 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 126 k 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 448 k 2024-04-24T22:33:39.753 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:39.759 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T22:33:40.151 INFO:teuthology.orchestra.run.smithi063.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 317 kB/s | 126 kB 00:00 2024-04-24T22:33:40.151 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:40.151 INFO:teuthology.orchestra.run.smithi063.stdout:Total 316 kB/s | 126 kB 00:00 2024-04-24T22:33:40.152 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:40.158 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:40.158 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:40.224 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:40.224 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:40.878 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T22:33:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:41.100 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:41.100 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:41.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs2 2024-04-24T22:33:41.872 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:42 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:41.953 INFO:teuthology.orchestra.run.smithi063.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:41.998 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:41.999 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:41.999 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:42.068 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs-devel 2024-04-24T22:33:42.179 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T22:33:42.213 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T22:33:42.602 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:43 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:42.725 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:42.725 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:42.725 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:33:42.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:42.727 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 25 k 2024-04-24T22:33:42.727 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 110 k 2024-04-24T22:33:42.727 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:42.905 INFO:teuthology.orchestra.run.smithi063.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 140 kB/s | 25 kB 00:00 2024-04-24T22:33:42.905 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:42.905 INFO:teuthology.orchestra.run.smithi063.stdout:Total 139 kB/s | 25 kB 00:00 2024-04-24T22:33:42.905 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:42.909 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:42.909 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:42.910 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:42.911 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:43.000 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:43.422 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:43.636 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:43.636 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:43.794 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados2 2024-04-24T22:33:44.323 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:45 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:44.404 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:44.447 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:44.448 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:44.448 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:44.509 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librbd1 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T22:33:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T22:33:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T22:33:45.039 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:46 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:45.119 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:45.162 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:45.162 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:45.163 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T22:33:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:45.211 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:45.211 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:45.212 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:45.226 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rados 2024-04-24T22:33:45.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-04-24T22:33:45.756 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:46 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:45.837 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:45.881 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:45.882 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:45.882 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:45.943 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rgw 2024-04-24T22:33:46.102 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:34 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:46.223 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:33:46.224 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-04-24T22:33:46.225 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:46.226 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.2 M 2024-04-24T22:33:46.226 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-04-24T22:33:46.226 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:47 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:46.553 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:46.596 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:46.597 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:46.597 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:46.659 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-cephfs 2024-04-24T22:33:46.964 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 93 kB/s | 26 kB 00:00 2024-04-24T22:33:46.990 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 157 kB/s | 47 kB 00:00 2024-04-24T22:33:47.015 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 59 kB/s | 19 kB 00:00 2024-04-24T22:33:47.190 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:47.271 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:47.314 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:47.315 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:47.315 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:47.356 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 465 kB/s | 159 kB 00:00 2024-04-24T22:33:47.376 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rbd 2024-04-24T22:33:47.381 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 532 kB/s | 222 kB 00:00 2024-04-24T22:33:47.482 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 428 kB/s | 54 kB 00:00 2024-04-24T22:33:47.599 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 477 kB/s | 279 kB 00:00 2024-04-24T22:33:47.641 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-04-24T22:33:47.724 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-04-24T22:33:47.883 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 375 kB/s | 59 kB 00:00 2024-04-24T22:33:47.913 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2024-04-24T22:33:47.995 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:33:48.008 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 502 kB/s | 206 kB 00:00 2024-04-24T22:33:48.034 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 597 kB/s | 90 kB 00:00 2024-04-24T22:33:48.036 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:48.036 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-04-24T22:33:48.038 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:48.039 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-24T22:33:48.039 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:48.097 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:48.104 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-fuse 2024-04-24T22:33:48.112 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:48.112 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:48.195 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:48.196 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:48.644 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:49 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:48.769 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:48.770 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:48.770 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-24T22:33:48.770 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 82 k 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 222 k 2024-04-24T22:33:48.771 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:48.916 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:49.084 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T22:33:49.100 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 250 kB/s | 82 kB 00:00 2024-04-24T22:33:49.100 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:49.100 INFO:teuthology.orchestra.run.smithi063.stdout:Total 249 kB/s | 82 kB 00:00 2024-04-24T22:33:49.100 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:49.108 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:49.108 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:49.146 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T22:33:49.173 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:49.173 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:49.201 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T22:33:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T22:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T22:33:49.318 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T22:33:49.431 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:49.484 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T22:33:49.540 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T22:33:49.791 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:50.036 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:50.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-mirror 2024-04-24T22:33:50.255 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T22:33:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T22:33:50.426 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T22:33:50.426 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T22:33:50.426 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T22:33:50.787 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:51 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:50.912 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:50.913 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-24T22:33:50.913 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:50.913 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:50.914 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.9 M 2024-04-24T22:33:50.915 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 12 M 2024-04-24T22:33:50.915 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T22:33:51.092 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T22:33:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T22:33:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T22:33:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T22:33:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T22:33:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T22:33:51.392 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T22:33:51.392 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:51.392 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:51.392 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:51.393 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:51.453 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 5.4 MB/s | 2.9 MB 00:00 2024-04-24T22:33:51.453 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:51.454 INFO:teuthology.orchestra.run.smithi063.stdout:Total 5.4 MB/s | 2.9 MB 00:00 2024-04-24T22:33:51.454 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:51.464 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:51.464 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:51.547 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:51.548 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:51.601 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T22:33:52.019 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:52.088 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:52.115 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:52.116 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:33:52.116 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:33:52.116 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:33:52.116 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:33:52.116 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:52.118 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:52.239 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-24T22:33:52.240 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-04-24T22:33:52.241 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:52.242 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 7.0 M 2024-04-24T22:33:52.242 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-04-24T22:33:52.242 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:52.889 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 539 kB/s | 123 kB 00:00 2024-04-24T22:33:52.956 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 840 kB/s | 248 kB 00:00 2024-04-24T22:33:53.115 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 364 kB/s | 58 kB 00:00 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:53.202 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:53.215 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 511 kB/s | 51 kB 00:00 2024-04-24T22:33:53.240 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.5 MB/s | 548 kB 00:00 2024-04-24T22:33:53.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-nbd 2024-04-24T22:33:53.743 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-04-24T22:33:53.744 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:53.745 INFO:teuthology.orchestra.run.smithi192.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-04-24T22:33:53.813 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:53.822 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:53.822 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:53.931 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:53.932 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:53.990 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:54 ago on Wed 24 Apr 2024 10:31:59 PM UTC. 2024-04-24T22:33:54.103 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-24T22:33:54.114 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 165 k 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 477 k 2024-04-24T22:33:54.115 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-24T22:33:54.227 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:54.342 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 699 kB/s | 165 kB 00:00 2024-04-24T22:33:54.342 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:54.342 INFO:teuthology.orchestra.run.smithi063.stdout:Total 694 kB/s | 165 kB 00:00 2024-04-24T22:33:54.342 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-24T22:33:54.354 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-24T22:33:54.354 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-24T22:33:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T22:33:54.422 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-24T22:33:54.423 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-24T22:33:54.472 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T22:33:54.588 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-24T22:33:54.682 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T22:33:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:55.087 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T22:33:55.143 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T22:33:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T22:33:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:33:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:55.342 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-24T22:33:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:33:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:33:55.342 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-24T22:33:55.539 DEBUG:teuthology.parallel:result is None 2024-04-24T22:33:55.917 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T22:33:55.917 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-24T22:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T22:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T22:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T22:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T22:33:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T22:33:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:33:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:56.157 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:56.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-04-24T22:33:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:57.038 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-24T22:33:57.040 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 894 k 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.6 M 2024-04-24T22:33:57.041 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:57.376 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 386 kB/s | 80 kB 00:00 2024-04-24T22:33:57.460 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 2.7 MB/s | 814 kB 00:00 2024-04-24T22:33:57.461 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:57.461 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.1 MB/s | 894 kB 00:00 2024-04-24T22:33:57.472 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:57.494 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:57.494 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:33:57.570 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:33:57.571 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:33:57.761 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:33:57.906 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T22:33:57.974 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T22:33:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T22:33:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:58.848 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:33:59.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-04-24T22:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:33:59.764 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:33:59.765 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:59.765 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-04-24T22:33:59.765 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 126 k 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 448 k 2024-04-24T22:33:59.766 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:33:59.983 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 585 kB/s | 126 kB 00:00 2024-04-24T22:33:59.983 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:33:59.984 INFO:teuthology.orchestra.run.smithi192.stdout:Total 579 kB/s | 126 kB 00:00 2024-04-24T22:33:59.984 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:33:59.990 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:33:59.990 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:34:00.053 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:34:00.053 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:34:00.205 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:34:00.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:00.697 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:00.931 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:01.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-04-24T22:34:01.713 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:49 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:01.793 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:01.834 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:01.835 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:01.836 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:01.915 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-04-24T22:34:02.436 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:02.556 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:02.557 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-04-24T22:34:02.557 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 25 k 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 110 k 2024-04-24T22:34:02.558 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:34:02.796 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 105 kB/s | 25 kB 00:00 2024-04-24T22:34:02.797 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:34:02.797 INFO:teuthology.orchestra.run.smithi192.stdout:Total 104 kB/s | 25 kB 00:00 2024-04-24T22:34:02.797 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:34:02.801 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:34:02.801 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:34:02.802 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:34:02.802 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:34:02.883 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:34:03.012 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:03.334 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:03.538 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:03.718 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-04-24T22:34:04.226 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:04.305 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:04.345 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:04.346 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:04.346 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:04.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-04-24T22:34:04.917 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:04.996 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:05.037 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:05.038 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:05.038 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:05.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-04-24T22:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:53 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:05.693 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:05.735 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:05.735 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:05.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-04-24T22:34:06.302 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:54 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:06.381 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:06.422 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:06.423 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:06.423 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:06.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-04-24T22:34:06.989 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:54 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:07.068 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:07.108 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:07.109 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:07.109 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:07.166 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-04-24T22:34:07.670 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:55 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:07.749 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T22:34:07.790 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:07.791 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-24T22:34:07.791 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:07.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-04-24T22:34:08.357 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:56 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:08.477 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:08.478 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:34:08.479 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:08.479 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 82 k 2024-04-24T22:34:08.479 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 222 k 2024-04-24T22:34:08.479 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:34:08.683 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 402 kB/s | 82 kB 00:00 2024-04-24T22:34:08.684 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:34:08.684 INFO:teuthology.orchestra.run.smithi192.stdout:Total 399 kB/s | 82 kB 00:00 2024-04-24T22:34:08.684 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:34:08.690 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:34:08.691 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:34:08.752 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:34:08.753 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:34:08.899 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:34:09.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:09.582 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:09.583 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:09.583 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:34:09.583 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:34:09.583 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:09.583 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:09.858 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-04-24T22:34:10.376 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:58 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:10.499 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:10.500 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:10.500 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-04-24T22:34:10.501 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:10.501 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:34:10.502 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:10.503 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.9 M 2024-04-24T22:34:10.503 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 12 M 2024-04-24T22:34:10.503 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:34:11.021 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 5.6 MB/s | 2.9 MB 00:00 2024-04-24T22:34:11.021 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:34:11.022 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.6 MB/s | 2.9 MB 00:00 2024-04-24T22:34:11.022 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:34:11.030 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:34:11.030 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:34:11.111 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:34:11.112 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:34:11.539 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:34:11.604 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:11.628 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:11.629 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:34:11.629 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:34:11.629 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:34:11.629 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:34:11.629 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:12.884 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-04-24T22:34:13.407 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:01 ago on Wed 24 Apr 2024 10:32:12 PM UTC. 2024-04-24T22:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-24T22:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 165 k 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 477 k 2024-04-24T22:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-24T22:34:13.892 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 463 kB/s | 165 kB 00:00 2024-04-24T22:34:13.893 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:34:13.893 INFO:teuthology.orchestra.run.smithi192.stdout:Total 461 kB/s | 165 kB 00:00 2024-04-24T22:34:13.893 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-24T22:34:13.901 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-24T22:34:13.901 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-24T22:34:13.967 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-24T22:34:13.967 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-24T22:34:14.119 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-24T22:34:14.222 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:14.619 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:14.839 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T22:34:14.839 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:14.840 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-24T22:34:14.840 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T22:34:14.840 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:34:14.840 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-24T22:34:15.087 DEBUG:teuthology.parallel:result is None 2024-04-24T22:34:15.088 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:34:15.088 INFO:teuthology.packaging:ref: None 2024-04-24T22:34:15.088 INFO:teuthology.packaging:tag: None 2024-04-24T22:34:15.088 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:34:15.088 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:34:15.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T22:34:15.393 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:34:15.419 INFO:teuthology.orchestra.run.smithi063.stdout:17.2.7-902.gc61472c8.el9 2024-04-24T22:34:15.420 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-24T22:34:15.420 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-24T22:34:15.421 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:34:15.422 INFO:teuthology.packaging:ref: None 2024-04-24T22:34:15.422 INFO:teuthology.packaging:tag: None 2024-04-24T22:34:15.422 INFO:teuthology.packaging:branch: quincy 2024-04-24T22:34:15.422 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:34:15.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T22:34:15.665 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:34:15.688 INFO:teuthology.orchestra.run.smithi192.stdout:17.2.7-902.gc61472c8.el9 2024-04-24T22:34:15.688 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-24T22:34:15.688 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-24T22:34:15.691 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T22:34:15.691 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:34:15.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:34:15.723 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:34:15.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:34:15.753 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T22:34:15.753 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:34:15.754 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:34:15.793 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:34:15.862 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:34:15.862 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:34:15.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:34:15.949 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T22:34:15.949 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:34:15.949 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:34:15.976 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:34:16.043 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:34:16.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:34:16.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:34:16.133 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T22:34:16.134 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:34:16.134 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:34:16.160 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:34:16.233 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:34:16.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:34:16.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:34:16.325 INFO:teuthology.run_tasks:Running task print... 2024-04-24T22:34:16.452 INFO:teuthology.task.print:**** done install task... 2024-04-24T22:34:16.452 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T22:34:16.659 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:34:16.659 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-24T22:34:16.660 INFO:tasks.cephadm:Cluster fsid is c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:34:16.660 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T22:34:16.660 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-24T22:34:16.661 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi063': '172.21.15.63', 'mon.smithi192': '172.21.15.192'} 2024-04-24T22:34:16.661 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T22:34:16.661 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-04-24T22:34:16.691 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-04-24T22:34:16.717 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-24T22:34:16.717 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.011 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 24 22:34 /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.012 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.214 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 24 22:34 /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.214 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.234 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T22:34:17.505 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-24T22:34:17.505 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-24T22:34:17.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-24T22:34:19.027 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T22:34:19.081 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-04-24T22:34:59.799 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-04-24T22:35:12.885 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:35:12.885 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T22:35:12.885 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-24T22:35:12.885 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-04-24T22:35:12.886 INFO:teuthology.orchestra.run.smithi063.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-24T22:35:12.886 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-04-24T22:35:12.886 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:35:12.907 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-04-24T22:35:12.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-04-24T22:35:12.974 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-04-24T22:35:13.008 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-04-24T22:35:13.041 INFO:tasks.cephadm:Writing seed config... 2024-04-24T22:35:13.053 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-24T22:35:13.053 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-24T22:35:13.053 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T22:35:13.054 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T22:35:13.055 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T22:35:13.056 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T22:35:13.057 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:35:13.057 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T22:35:13.073 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c8a2676e-028a-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T22:35:13.073 DEBUG:teuthology.orchestra.run.smithi063:mon.smithi063> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service 2024-04-24T22:35:13.116 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T22:35:13.116 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid c8a2676e-028a-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.63 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:35:13.327 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:35:13.327 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'c8a2676e-028a-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.63', '--skip-admin-label'] 2024-04-24T22:35:13.353 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T22:35:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T22:35:13.354 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying podman|docker is present... 2024-04-24T22:35:13.378 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T22:35:13.378 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying lvm2 is present... 2024-04-24T22:35:13.378 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying time synchronization is in place... 2024-04-24T22:35:13.386 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T22:35:13.386 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T22:35:13.392 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T22:35:13.392 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:35:13.399 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-04-24T22:35:13.406 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-04-24T22:35:13.406 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-04-24T22:35:13.406 INFO:teuthology.orchestra.run.smithi063.stdout:Repeating the final host check... 2024-04-24T22:35:13.440 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T22:35:13.440 INFO:teuthology.orchestra.run.smithi063.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T22:35:13.440 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl is present 2024-04-24T22:35:13.440 INFO:teuthology.orchestra.run.smithi063.stdout:lvcreate is present 2024-04-24T22:35:13.446 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T22:35:13.446 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T22:35:13.453 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T22:35:13.453 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:35:13.460 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:Host looks OK 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:Cluster fsid: c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:Acquiring lock 140432566698912 on /run/cephadm/c8a2676e-028a-11ef-bc93-c7b262605968.lock 2024-04-24T22:35:13.467 INFO:teuthology.orchestra.run.smithi063.stdout:Lock 140432566698912 acquired on /run/cephadm/c8a2676e-028a-11ef-bc93-c7b262605968.lock 2024-04-24T22:35:13.468 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 3300 ... 2024-04-24T22:35:13.468 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 6789 ... 2024-04-24T22:35:13.468 INFO:teuthology.orchestra.run.smithi063.stdout:Base mon IP(s) is [172.21.15.63:3300, 172.21.15.63:6789], mon addrv is [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-04-24T22:35:13.471 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.63 metric 100 2024-04-24T22:35:13.471 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.63 metric 100 2024-04-24T22:35:13.474 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T22:35:13.474 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-24T22:35:13.474 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 fe80::2474:b7ab:3c0a:9ab5/64 scope link noprefixroute 2024-04-24T22:35:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T22:35:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-04-24T22:35:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-04-24T22:35:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T22:35:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T22:35:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T22:35:13.992 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-24T22:35:13.992 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T22:35:13.993 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T22:35:13.993 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:0e2af62137028b073c6ccdb8d4d7e0ccccc475ce5f6e225cb983e90813813045 2024-04-24T22:35:13.993 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T22:35:13.993 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying config sha256:0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-24T22:35:13.993 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T22:35:14.864 INFO:teuthology.orchestra.run.smithi063.stdout:ceph: stdout ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-24T22:35:14.864 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph version: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-24T22:35:14.864 INFO:teuthology.orchestra.run.smithi063.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T22:35:15.772 INFO:teuthology.orchestra.run.smithi063.stdout:stat: stdout 167 167 2024-04-24T22:35:15.772 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial keys... 2024-04-24T22:35:16.679 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQAkiSlmm6ojBxAApPAJi6E6TGUxOXJ0R1HftA== 2024-04-24T22:35:17.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQAliSlmQ2aXABAAMwuv3i9ZICPxKdLqw2nKww== 2024-04-24T22:35:18.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQAliSlmkww1OxAAio9sGgdGmvZkLbpSfSx+yQ== 2024-04-24T22:35:18.537 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial monmap... 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool for smithi063 [v2:172.21.15.63:3300,v1:172.21.15.63:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = octopus 2024-04-24T22:35:19.443 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: set fsid to c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:19.444 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T22:35:19.444 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:35:19.444 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mon... 2024-04-24T22:35:20.273 INFO:teuthology.orchestra.run.smithi063.stdout:create mon.smithi063 on 2024-04-24T22:35:20.575 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T22:35:20.742 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T22:35:20.912 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target. 2024-04-24T22:35:20.912 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target. 2024-04-24T22:35:21.129 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063 2024-04-24T22:35:21.130 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service: Unit ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service not loaded. 2024-04-24T22:35:21.303 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target.wants/ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service → /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968@.service. 2024-04-24T22:35:21.648 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 podman[26517]: 2024-04-24 22:35:21.448859635 +0000 UTC m=+0.018465081 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-24T22:35:21.830 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:35:21.830 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-04-24T22:35:21.837 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:35:21.837 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:35:21.837 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-04-24T22:35:21.837 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T22:35:21.838 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon to start... 2024-04-24T22:35:21.838 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon... 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 podman[26517]: 2024-04-24 22:35:21.723081523 +0000 UTC m=+0.292686939 container init 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c) 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 podman[26517]: 2024-04-24 22:35:21.726202427 +0000 UTC m=+0.295807845 container start 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 ceph-mon[26535]: mkfs c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 ceph-mon[26535]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 bash[26517]: 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 2024-04-24T22:35:21.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:21 smithi063 systemd[1]: Started Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:35:22.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T22:35:22.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout id: c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout services: 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi063 (age 0.57408s) 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout data: 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:22.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.257+0000 7f163dfa1700 1 Processor -- start 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.257+0000 7f163dfa1700 1 -- start start 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f163dfa1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f163dfa1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1638071640 con 0x7f1638071230 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f16377fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f16377fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47450/0 (socket says 172.21.15.63:47450) 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f16377fe700 1 -- 172.21.15.63:0/986741722 learned_addr learned my addr 172.21.15.63:0/986741722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.258+0000 7f16377fe700 1 -- 172.21.15.63:0/986741722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638105a30 con 0x7f1638071230 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f16377fe700 1 --2- 172.21.15.63:0/986741722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f162c0098e0 tx=0x7f162c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7046463bea42bcc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f16367fc700 1 -- 172.21.15.63:0/986741722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f162c00b860 con 0x7f1638071230 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f16367fc700 1 -- 172.21.15.63:0/986741722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c00b9c0 con 0x7f1638071230 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f16367fc700 1 -- 172.21.15.63:0/986741722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f162c00bc90 con 0x7f1638071230 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 -- 172.21.15.63:0/986741722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 msgr2=0x7f16381054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:22.849 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 --2- 172.21.15.63:0/986741722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f162c0098e0 tx=0x7f162c009250 comp rx=0 tx=0).stop 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 -- 172.21.15.63:0/986741722 shutdown_connections 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 --2- 172.21.15.63:0/986741722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f16381054f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 -- 172.21.15.63:0/986741722 >> 172.21.15.63:0/986741722 conn(0x7f16380fcec0 msgr2=0x7f16380ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 -- 172.21.15.63:0/986741722 shutdown_connections 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.259+0000 7f163dfa1700 1 -- 172.21.15.63:0/986741722 wait complete. 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.260+0000 7f163dfa1700 1 Processor -- start 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.260+0000 7f163dfa1700 1 -- start start 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.260+0000 7f163dfa1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.260+0000 7f163dfa1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1638111390 con 0x7f1638071230 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.260+0000 7f16377fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f16377fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47456/0 (socket says 172.21.15.63:47456) 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f16377fe700 1 -- 172.21.15.63:0/2617865655 learned_addr learned my addr 172.21.15.63:0/2617865655 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f16377fe700 1 -- 172.21.15.63:0/2617865655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0094f0 con 0x7f1638071230 2024-04-24T22:35:22.850 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f16377fe700 1 --2- 172.21.15.63:0/2617865655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f162c009290 tx=0x7f162c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f162c01c5c0 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c01c740 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1638111590 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f162c01ca50 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1638111fc0 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f162c01cbb0 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.261+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f162c01ac30 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.262+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163805f5c0 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.264+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f162c045790 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.336+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1638112b50 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f161ffff700 1 -- 172.21.15.63:0/2617865655 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+324 (secure 0 0 0) 0x7f162c034070 con 0x7f1638071230 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 msgr2=0x7f1638110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f163dfa1700 1 --2- 172.21.15.63:0/2617865655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f162c009290 tx=0x7f162c016000 comp rx=0 tx=0).stop 2024-04-24T22:35:22.851 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 shutdown_connections 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f163dfa1700 1 --2- 172.21.15.63:0/2617865655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1638071230 0x7f1638110e50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.337+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 >> 172.21.15.63:0/2617865655 conn(0x7f16380fcec0 msgr2=0x7f16380fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.338+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 shutdown_connections 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:22.338+0000 7f163dfa1700 1 -- 172.21.15.63:0/2617865655 wait complete. 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:mon is available 2024-04-24T22:35:22.852 INFO:teuthology.orchestra.run.smithi063.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: monmap e1: 1 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: fsmap 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: mgrmap e1: no daemons active 2024-04-24T22:35:23.033 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:22 smithi063 ceph-mon[26535]: from='client.? 172.21.15.63:0/2617865655' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T22:35:23.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.230+0000 7febaf59e700 1 Processor -- start 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.230+0000 7febaf59e700 1 -- start start 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.230+0000 7febaf59e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.230+0000 7febaf59e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb0102430 con 0x7febb0101ae0 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47462/0 (socket says 172.21.15.63:47462) 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febae59c700 1 -- 172.21.15.63:0/1275326879 learned_addr learned my addr 172.21.15.63:0/1275326879 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:23.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febae59c700 1 -- 172.21.15.63:0/1275326879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0102570 con 0x7febb0101ae0 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febae59c700 1 --2- 172.21.15.63:0/1275326879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38303a616bd1819e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febad59a700 1 -- 172.21.15.63:0/1275326879 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feba80092e0 con 0x7febb0101ae0 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febad59a700 1 -- 172.21.15.63:0/1275326879 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800ba20 con 0x7febb0101ae0 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.231+0000 7febad59a700 1 -- 172.21.15.63:0/1275326879 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feba800bd30 con 0x7febb0101ae0 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- 172.21.15.63:0/1275326879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 msgr2=0x7febb0101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 --2- 172.21.15.63:0/1275326879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba8009840 comp rx=0 tx=0).stop 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- 172.21.15.63:0/1275326879 shutdown_connections 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 --2- 172.21.15.63:0/1275326879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb0101ef0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- 172.21.15.63:0/1275326879 >> 172.21.15.63:0/1275326879 conn(0x7febb00fc490 msgr2=0x7febb00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- 172.21.15.63:0/1275326879 shutdown_connections 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- 172.21.15.63:0/1275326879 wait complete. 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 Processor -- start 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.232+0000 7febaf59e700 1 -- start start 2024-04-24T22:35:23.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febaf59e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febaf59e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb0113960 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47478/0 (socket says 172.21.15.63:47478) 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febae59c700 1 -- 172.21.15.63:0/1620765899 learned_addr learned my addr 172.21.15.63:0/1620765899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febae59c700 1 -- 172.21.15.63:0/1620765899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8009ae0 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febae59c700 1 --2- 172.21.15.63:0/1620765899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febb0063820 tx=0x7feba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feba8012b90 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8012d10 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb0113aa0 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feba801e400 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.233+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb0113f90 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.234+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7feba8019030 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.234+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7feba8026400 con 0x7febb0101ae0 2024-04-24T22:35:23.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.234+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb01095d0 con 0x7febb0101ae0 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.237+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7feba8017070 con 0x7febb0101ae0 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.307+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7febb0102880 con 0x7febb0101ae0 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.311+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7feba8038400 con 0x7febb0101ae0 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.311+0000 7feba6ffd700 1 -- 172.21.15.63:0/1620765899 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7feba80265c0 con 0x7febb0101ae0 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 msgr2=0x7febb01012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 --2- 172.21.15.63:0/1620765899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febb0063820 tx=0x7feba8017000 comp rx=0 tx=0).stop 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 shutdown_connections 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 --2- 172.21.15.63:0/1620765899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb0101ae0 0x7febb01012e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 >> 172.21.15.63:0/1620765899 conn(0x7febb00fc490 msgr2=0x7febb0103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 shutdown_connections 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:23.312+0000 7febaf59e700 1 -- 172.21.15.63:0/1620765899 wait complete. 2024-04-24T22:35:23.877 INFO:teuthology.orchestra.run.smithi063.stdout:Generating new minimal ceph.conf... 2024-04-24T22:35:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:23 smithi063 ceph-mon[26535]: from='client.? 172.21.15.63:0/1620765899' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T22:35:24.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:23 smithi063 ceph-mon[26535]: from='client.? 172.21.15.63:0/1620765899' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T22:35:24.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053c700700 1 Processor -- start 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053c700700 1 -- start start 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053c700700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053c700700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341031d0 con 0x7f0534102880 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053a49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053a49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47488/0 (socket says 172.21.15.63:47488) 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053a49c700 1 -- 172.21.15.63:0/3392128531 learned_addr learned my addr 172.21.15.63:0/3392128531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.292+0000 7f053a49c700 1 -- 172.21.15.63:0/3392128531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534103310 con 0x7f0534102880 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053a49c700 1 --2- 172.21.15.63:0/3392128531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f052c009e50 tx=0x7f052c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e478c87e59862ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053949a700 1 -- 172.21.15.63:0/3392128531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c0092e0 con 0x7f0534102880 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053949a700 1 -- 172.21.15.63:0/3392128531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f052c012430 con 0x7f0534102880 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053949a700 1 -- 172.21.15.63:0/3392128531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c0075c0 con 0x7f0534102880 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053c700700 1 -- 172.21.15.63:0/3392128531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 msgr2=0x7f0534102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:24.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.293+0000 7f053c700700 1 --2- 172.21.15.63:0/3392128531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f052c009e50 tx=0x7f052c009840 comp rx=0 tx=0).stop 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- 172.21.15.63:0/3392128531 shutdown_connections 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 --2- 172.21.15.63:0/3392128531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534102c90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- 172.21.15.63:0/3392128531 >> 172.21.15.63:0/3392128531 conn(0x7f05340fe060 msgr2=0x7f0534100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- 172.21.15.63:0/3392128531 shutdown_connections 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- 172.21.15.63:0/3392128531 wait complete. 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 Processor -- start 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- start start 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.294+0000 7f053c700700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534111b70 con 0x7f0534102880 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053a49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053a49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47504/0 (socket says 172.21.15.63:47504) 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053a49c700 1 -- 172.21.15.63:0/652797461 learned_addr learned my addr 172.21.15.63:0/652797461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053a49c700 1 -- 172.21.15.63:0/652797461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c009ae0 con 0x7f0534102880 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053a49c700 1 --2- 172.21.15.63:0/652797461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f052c009fd0 tx=0x7f052c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:24.979 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c0092e0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f052c007cc0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534111d70 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c01c9a0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341128e0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.295+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f052c019070 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.296+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f052c025ac0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.296+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053405f5c0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.298+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f052c02a060 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.369+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f053404bed0 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.369+0000 7f0532ffd700 1 -- 172.21.15.63:0/652797461 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f052c020070 con 0x7f0534102880 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 msgr2=0x7f0534111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 --2- 172.21.15.63:0/652797461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f052c009fd0 tx=0x7f052c009840 comp rx=0 tx=0).stop 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 shutdown_connections 2024-04-24T22:35:24.980 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 --2- 172.21.15.63:0/652797461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534102880 0x7f0534111630 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:24.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 >> 172.21.15.63:0/652797461 conn(0x7f05340fe060 msgr2=0x7f05340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:24.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 shutdown_connections 2024-04-24T22:35:24.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:24.370+0000 7f053c700700 1 -- 172.21.15.63:0/652797461 wait complete. 2024-04-24T22:35:24.981 INFO:teuthology.orchestra.run.smithi063.stdout:Restarting the monitor... 2024-04-24T22:35:24.994 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:24 smithi063 ceph-mon[26535]: from='client.? 172.21.15.63:0/652797461' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:35:24.994 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:24 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:35:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[26531]: 2024-04-24T22:35:25.256+0000 7f46bac0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:35:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[26531]: 2024-04-24T22:35:25.256+0000 7f46bac0f700 -1 mon.smithi063@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T22:35:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 podman[26782]: 2024-04-24 22:35:25.451440677 +0000 UTC m=+0.305761171 container died 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:35:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 podman[26782]: 2024-04-24 22:35:25.791208306 +0000 UTC m=+0.645528794 container cleanup 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T22:35:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 bash[26782]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063 2024-04-24T22:35:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:25 smithi063 podman[26796]: 2024-04-24 22:35:25.958610304 +0000 UTC m=+0.502049317 container remove 318c77083dc376774d5308087cddd506d5ee1859627038115fff99a5f583ac24 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-c61472c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T22:35:26.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service: Deactivated successfully. 2024-04-24T22:35:26.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:35:26.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 systemd[1]: Starting Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:35:26.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 podman[26893]: 2024-04-24 22:35:26.376068288 +0000 UTC m=+0.097530813 container create 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:35:26.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 podman[26893]: 2024-04-24 22:35:26.296943392 +0000 UTC m=+0.018405950 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-24T22:35:26.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 podman[26893]: 2024-04-24 22:35:26.595022701 +0000 UTC m=+0.316485227 container init 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:35:26.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 podman[26893]: 2024-04-24 22:35:26.598554001 +0000 UTC m=+0.320016526 container start 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T22:35:26.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: pidfile_write: ignore empty --pid-file 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: load: jerasure load: lrc 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: RocksDB version: 6.15.5 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Compile date Apr 24 2024 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: DB SUMMARY 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: DB Session ID: NTAIECZW5LHG39Z4UYNV 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: CURRENT file: CURRENT 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000010.log size: 98160 ; 2024-04-24T22:35:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.error_if_exists: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.create_if_missing: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.paranoid_checks: 1 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.env: 0x55dfec655b00 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.fs: Posix File System 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.info_log: 0x55dfecfb7620 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.statistics: (nil) 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.use_fsync: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_log_file_size: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T22:35:26.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_fallocate: 1 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.db_log_dir: 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi063/store.db 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:35:26.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.write_buffer_manager: 0x55dfedcaaa20 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:35:26.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.unordered_write: 0 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.row_cache: None 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.wal_filter: None 2024-04-24T22:35:26.632 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.preserve_deletes: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:35:26.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:35:26.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_open_files: -1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Compression algorithms supported: 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kZSTD supported: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:35:26.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000009 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.merge_operator: 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_filter: None 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dfecf8af58) 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-04-24T22:35:26.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: hash_index_allow_collision: 1 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 1 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x55dfecfbd090 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-04-24T22:35:26.637 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 4 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-04-24T22:35:26.638 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression: NoCompression 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.num_levels: 7 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:35:26.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:35:26.640 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:35:26.693 INFO:teuthology.orchestra.run.smithi063.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T22:35:26.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:35:26.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:35:26.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:35:26.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.arena_block_size: 4194304 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:35:26.914 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.ttl: 2592000 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:35:26.915 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998126625142, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998126626401, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93366, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91478, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84042, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1713998126, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "2b59f023-ef53-4379-9a33-e15e7967a456", "db_session_id": "NTAIECZW5LHG39Z4UYNV"}} 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998126626856, "job": 1, "event": "recovery_finished"} 2024-04-24T22:35:26.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55dfecf51180 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: DB pointer 0x55dfed026000 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:35:26.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:35:26.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 19.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 19.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 110.7 0.00 0.00 1 0.001 0 0 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:35:26.920 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 19.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???) e1 preinit fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).mds e1 new map 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).mds e1 print_map 2024-04-24T22:35:26.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e1 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: -1 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: No filesystems configured 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 bash[26893]: 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: monmap e1: 1 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: fsmap 2024-04-24T22:35:26.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T22:35:26.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 ceph-mon[26912]: mgrmap e1: no daemons active 2024-04-24T22:35:26.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:26 smithi063 systemd[1]: Started Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:35:27.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.117+0000 7f08ea20e700 1 Processor -- start 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.117+0000 7f08ea20e700 1 -- start start 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08ea20e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08ea20e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4105970 con 0x7f08e40712f0 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47520/0 (socket says 172.21.15.63:47520) 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08e37fe700 1 -- 172.21.15.63:0/3672980522 learned_addr learned my addr 172.21.15.63:0/3672980522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.118+0000 7f08e37fe700 1 -- 172.21.15.63:0/3672980522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4105ab0 con 0x7f08e40712f0 2024-04-24T22:35:27.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08e37fe700 1 --2- 172.21.15.63:0/3672980522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098e0 tx=0x7f08cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1ce8e3eb8a6995 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08e27fc700 1 -- 172.21.15.63:0/3672980522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08cc00b8f0 con 0x7f08e40712f0 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08e27fc700 1 -- 172.21.15.63:0/3672980522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08cc011420 con 0x7f08e40712f0 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08e27fc700 1 -- 172.21.15.63:0/3672980522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08cc010540 con 0x7f08e40712f0 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 -- 172.21.15.63:0/3672980522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 msgr2=0x7f08e4105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 --2- 172.21.15.63:0/3672980522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098e0 tx=0x7f08cc009250 comp rx=0 tx=0).stop 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 -- 172.21.15.63:0/3672980522 shutdown_connections 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 --2- 172.21.15.63:0/3672980522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4105430 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 -- 172.21.15.63:0/3672980522 >> 172.21.15.63:0/3672980522 conn(0x7f08e40fcec0 msgr2=0x7f08e40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 -- 172.21.15.63:0/3672980522 shutdown_connections 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.119+0000 7f08ea20e700 1 -- 172.21.15.63:0/3672980522 wait complete. 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08ea20e700 1 Processor -- start 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08ea20e700 1 -- start start 2024-04-24T22:35:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08ea20e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08ea20e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4111b90 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47524/0 (socket says 172.21.15.63:47524) 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.120+0000 7f08e37fe700 1 -- 172.21.15.63:0/3119550879 learned_addr learned my addr 172.21.15.63:0/3119550879 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08e37fe700 1 -- 172.21.15.63:0/3119550879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0094f0 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08e37fe700 1 --2- 172.21.15.63:0/3119550879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f08cc000c00 tx=0x7f08cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08cc0107b0 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08cc010da0 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e4111d90 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08cc01add0 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e4112900 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f08cc024460 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.121+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f08cc02e7d0 con 0x7f08e40712f0 2024-04-24T22:35:27.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.122+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e405f5c0 con 0x7f08e40712f0 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.124+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f08cc01a5c0 con 0x7f08e40712f0 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.193+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f08e4106830 con 0x7f08e40712f0 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.198+0000 7f08dbfff700 1 -- 172.21.15.63:0/3119550879 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f08cc010910 con 0x7f08e40712f0 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 msgr2=0x7f08e4111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 --2- 172.21.15.63:0/3119550879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f08cc000c00 tx=0x7f08cc009250 comp rx=0 tx=0).stop 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 shutdown_connections 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 --2- 172.21.15.63:0/3119550879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08e40712f0 0x7f08e4111650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 >> 172.21.15.63:0/3119550879 conn(0x7f08e40fcec0 msgr2=0x7f08e40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 shutdown_connections 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:27.199+0000 7f08ea20e700 1 -- 172.21.15.63:0/3119550879 wait complete. 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mgr... 2024-04-24T22:35:27.774 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 9283 ... 2024-04-24T22:35:27.956 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mgr.smithi063.upcnvd 2024-04-24T22:35:27.956 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mgr.smithi063.upcnvd.service: Unit ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mgr.smithi063.upcnvd.service not loaded. 2024-04-24T22:35:28.110 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968.target.wants/ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mgr.smithi063.upcnvd.service → /etc/systemd/system/ceph-c8a2676e-028a-11ef-bc93-c7b262605968@.service. 2024-04-24T22:35:28.453 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:28 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3119550879' entity='client.admin' 2024-04-24T22:35:28.622 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:35:28.622 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-04-24T22:35:28.639 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:35:28.639 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:35:28.639 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-04-24T22:35:28.639 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T22:35:28.641 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:35:28.641 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr to start... 2024-04-24T22:35:28.650 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr... 2024-04-24T22:35:29.388 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:29 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3348146878' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "c8a2676e-028a-11ef-bc93-c7b262605968", 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:35:29.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:29.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:35:29.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:29.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:35:21.765169+0000", 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.082+0000 7ffba1766700 1 Processor -- start 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.082+0000 7ffba1766700 1 -- start start 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.082+0000 7ffba1766700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.082+0000 7ffba1766700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c1015a0 con 0x7ffb9c100c50 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:29.771 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52676/0 (socket says 172.21.15.63:52676) 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb9affd700 1 -- 172.21.15.63:0/2192132961 learned_addr learned my addr 172.21.15.63:0/2192132961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb9affd700 1 -- 172.21.15.63:0/2192132961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c1016e0 con 0x7ffb9c100c50 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb9affd700 1 --2- 172.21.15.63:0/2192132961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb900098e0 tx=0x7ffb90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02dc3c7586d76ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb99ffb700 1 -- 172.21.15.63:0/2192132961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb9000b860 con 0x7ffb9c100c50 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb99ffb700 1 -- 172.21.15.63:0/2192132961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb9000be50 con 0x7ffb9c100c50 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.083+0000 7ffb99ffb700 1 -- 172.21.15.63:0/2192132961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb90010380 con 0x7ffb9c100c50 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 -- 172.21.15.63:0/2192132961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 msgr2=0x7ffb9c101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 --2- 172.21.15.63:0/2192132961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb900098e0 tx=0x7ffb90009250 comp rx=0 tx=0).stop 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 -- 172.21.15.63:0/2192132961 shutdown_connections 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 --2- 172.21.15.63:0/2192132961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c101060 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 -- 172.21.15.63:0/2192132961 >> 172.21.15.63:0/2192132961 conn(0x7ffb9c0fc6b0 msgr2=0x7ffb9c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 -- 172.21.15.63:0/2192132961 shutdown_connections 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.084+0000 7ffba1766700 1 -- 172.21.15.63:0/2192132961 wait complete. 2024-04-24T22:35:29.772 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffba1766700 1 Processor -- start 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffba1766700 1 -- start start 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffba1766700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffba1766700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c111ac0 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52678/0 (socket says 172.21.15.63:52678) 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.085+0000 7ffb9affd700 1 -- 172.21.15.63:0/3348146878 learned_addr learned my addr 172.21.15.63:0/3348146878 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb9affd700 1 -- 172.21.15.63:0/3348146878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900094f0 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb9affd700 1 --2- 172.21.15.63:0/3348146878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b440 tx=0x7ffb90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb90010760 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb90010d50 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c111cc0 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb9001ae20 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c112830 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ffb9002e440 con 0x7ffb9c100c50 2024-04-24T22:35:29.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.086+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ffb90024740 con 0x7ffb9c100c50 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.087+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c05f5c0 con 0x7ffb9c100c50 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.089+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7ffb90029070 con 0x7ffb9c100c50 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.163+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffb9c106830 con 0x7ffb9c100c50 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.163+0000 7ffb837fe700 1 -- 172.21.15.63:0/3348146878 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7ffb9001a6a0 con 0x7ffb9c100c50 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 msgr2=0x7ffb9c111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 --2- 172.21.15.63:0/3348146878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b440 tx=0x7ffb90009250 comp rx=0 tx=0).stop 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 shutdown_connections 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 --2- 172.21.15.63:0/3348146878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9c100c50 0x7ffb9c111580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 >> 172.21.15.63:0/3348146878 conn(0x7ffb9c0fc6b0 msgr2=0x7ffb9c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 shutdown_connections 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:29.164+0000 7ffba1766700 1 -- 172.21.15.63:0/3348146878 wait complete. 2024-04-24T22:35:29.774 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (1/15)... 2024-04-24T22:35:32.543 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:32 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1446760876' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:35:32.857 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:32.857 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:32.857 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "c8a2676e-028a-11ef-bc93-c7b262605968", 2024-04-24T22:35:32.857 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:35:32.857 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:32.858 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:35:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:35:32.867 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:32.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:35:21.765169+0000", 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb07b4a700 1 Processor -- start 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb07b4a700 1 -- start start 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb07b4a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb07b4a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00071640 con 0x7fcb00071230 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb058e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb058e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52682/0 (socket says 172.21.15.63:52682) 2024-04-24T22:35:32.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.153+0000 7fcb058e6700 1 -- 172.21.15.63:0/4001880884 learned_addr learned my addr 172.21.15.63:0/4001880884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb058e6700 1 -- 172.21.15.63:0/4001880884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00105a30 con 0x7fcb00071230 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb058e6700 1 --2- 172.21.15.63:0/4001880884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a1c63b1d34de8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb048e4700 1 -- 172.21.15.63:0/4001880884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc00b860 con 0x7fcb00071230 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb048e4700 1 -- 172.21.15.63:0/4001880884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcafc00be50 con 0x7fcb00071230 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb048e4700 1 -- 172.21.15.63:0/4001880884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc010380 con 0x7fcb00071230 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.154+0000 7fcb07b4a700 1 -- 172.21.15.63:0/4001880884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 msgr2=0x7fcb001054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 --2- 172.21.15.63:0/4001880884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc009250 comp rx=0 tx=0).stop 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 -- 172.21.15.63:0/4001880884 shutdown_connections 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 --2- 172.21.15.63:0/4001880884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb001054f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 -- 172.21.15.63:0/4001880884 >> 172.21.15.63:0/4001880884 conn(0x7fcb000fcec0 msgr2=0x7fcb000ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 -- 172.21.15.63:0/4001880884 shutdown_connections 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 -- 172.21.15.63:0/4001880884 wait complete. 2024-04-24T22:35:32.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.155+0000 7fcb07b4a700 1 Processor -- start 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb07b4a700 1 -- start start 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb07b4a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb07b4a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00103380 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb058e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb058e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52688/0 (socket says 172.21.15.63:52688) 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb058e6700 1 -- 172.21.15.63:0/1446760876 learned_addr learned my addr 172.21.15.63:0/1446760876 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb058e6700 1 -- 172.21.15.63:0/1446760876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0094f0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcb058e6700 1 --2- 172.21.15.63:0/1446760876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcafc006fd0 tx=0x7fcafc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.156+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc0107c0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcafc010db0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb001013f0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb001018e0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcafc01ada0 con 0x7fcb00071230 2024-04-24T22:35:32.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcafc019450 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.157+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcafc024750 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.158+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae40052e0 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.160+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fcafc029060 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.233+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcae4005860 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.234+0000 7fcaf67fc700 1 -- 172.21.15.63:0/1446760876 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fcafc020070 con 0x7fcb00071230 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 msgr2=0x7fcb00102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 --2- 172.21.15.63:0/1446760876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcafc006fd0 tx=0x7fcafc009250 comp rx=0 tx=0).stop 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 shutdown_connections 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 --2- 172.21.15.63:0/1446760876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb00071230 0x7fcb00102e40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 >> 172.21.15.63:0/1446760876 conn(0x7fcb000fcec0 msgr2=0x7fcb000fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 shutdown_connections 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:32.235+0000 7fcb07b4a700 1 -- 172.21.15.63:0/1446760876 wait complete. 2024-04-24T22:35:32.872 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (2/15)... 2024-04-24T22:35:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: mgrmap e2: smithi063.upcnvd(active, starting, since 0.00354863s) 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:35.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:35.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:35 smithi063 ceph-mon[26912]: from='mgr.14100 172.21.15.63:0/1775195917' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:36 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1540071786' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:35:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:36 smithi063 ceph-mon[26912]: mgrmap e3: smithi063.upcnvd(active, since 1.00754s) 2024-04-24T22:35:37.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:37.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "c8a2676e-028a-11ef-bc93-c7b262605968", 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:35:37.748 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:35:37.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:35:37.757 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:35:37.757 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:35:37.758 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:37.759 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:35:21.765169+0000", 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d3291700 1 Processor -- start 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d3291700 1 -- start start 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d3291700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d3291700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc071640 con 0x7f70cc071230 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d102d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d102d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52696/0 (socket says 172.21.15.63:52696) 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d102d700 1 -- 172.21.15.63:0/3021426621 learned_addr learned my addr 172.21.15.63:0/3021426621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.292+0000 7f70d102d700 1 -- 172.21.15.63:0/3021426621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc105a30 con 0x7f70cc071230 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d102d700 1 --2- 172.21.15.63:0/3021426621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70c80098e0 tx=0x7f70c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2e5f4ad62db9df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70c3fff700 1 -- 172.21.15.63:0/3021426621 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c800b860 con 0x7f70cc071230 2024-04-24T22:35:37.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70c3fff700 1 -- 172.21.15.63:0/3021426621 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70c800be50 con 0x7f70cc071230 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70c3fff700 1 -- 172.21.15.63:0/3021426621 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c8010380 con 0x7f70cc071230 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d3291700 1 -- 172.21.15.63:0/3021426621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 msgr2=0x7f70cc1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d3291700 1 --2- 172.21.15.63:0/3021426621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70c80098e0 tx=0x7f70c8009250 comp rx=0 tx=0).stop 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d3291700 1 -- 172.21.15.63:0/3021426621 shutdown_connections 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d3291700 1 --2- 172.21.15.63:0/3021426621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc1054f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.293+0000 7f70d3291700 1 -- 172.21.15.63:0/3021426621 >> 172.21.15.63:0/3021426621 conn(0x7f70cc0fcec0 msgr2=0x7f70cc0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 -- 172.21.15.63:0/3021426621 shutdown_connections 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 -- 172.21.15.63:0/3021426621 wait complete. 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 Processor -- start 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 -- start start 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.294+0000 7f70d3291700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc111b40 con 0x7f70cc071230 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d102d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d102d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52706/0 (socket says 172.21.15.63:52706) 2024-04-24T22:35:37.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d102d700 1 -- 172.21.15.63:0/1540071786 learned_addr learned my addr 172.21.15.63:0/1540071786 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d102d700 1 -- 172.21.15.63:0/1540071786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80094f0 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d102d700 1 --2- 172.21.15.63:0/1540071786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c8010760 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70c8010d50 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc111d40 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c801ae10 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.295+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc1128b0 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.296+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f70c8019410 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.296+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f70c8024740 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.296+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc05f5c0 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.298+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f70c8029060 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.312+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7f70c80108c0 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.378+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f70cc106870 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.378+0000 7f70c1ffb700 1 -- 172.21.15.63:0/1540071786 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f70c802d3e0 con 0x7f70cc071230 2024-04-24T22:35:37.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 msgr2=0x7f70cc111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 --2- 172.21.15.63:0/1540071786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8009250 comp rx=0 tx=0).stop 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 shutdown_connections 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 --2- 172.21.15.63:0/1540071786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70cc071230 0x7f70cc111600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 >> 172.21.15.63:0/1540071786 conn(0x7f70cc0fcec0 msgr2=0x7f70cc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 shutdown_connections 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:35.379+0000 7f70d3291700 1 -- 172.21.15.63:0/1540071786 wait complete. 2024-04-24T22:35:37.763 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (3/15)... 2024-04-24T22:35:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:38 smithi063 ceph-mon[26912]: mgrmap e4: smithi063.upcnvd(active, since 2s) 2024-04-24T22:35:43.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:43 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2014842842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "c8a2676e-028a-11ef-bc93-c7b262605968", 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T22:35:46.459 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T22:35:46.460 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.461 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T22:35:46.461 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:46.461 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T22:35:46.461 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T22:35:46.461 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T22:35:46.469 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T22:35:46.469 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T22:35:46.469 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T22:35:46.469 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.469 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.470 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T22:35:21.765169+0000", 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-04-24T22:35:46.471 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.873+0000 7f16fc67c700 1 Processor -- start 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.873+0000 7f16fc67c700 1 -- start start 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fc67c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fc67c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4103280 con 0x7f16f4071290 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fa418700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fa418700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41244/0 (socket says 172.21.15.63:41244) 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fa418700 1 -- 172.21.15.63:0/2977054194 learned_addr learned my addr 172.21.15.63:0/2977054194 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fa418700 1 -- 172.21.15.63:0/2977054194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4107dd0 con 0x7f16f4071290 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16fa418700 1 --2- 172.21.15.63:0/2977054194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098e0 tx=0x7f16e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01ef653a102a012 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16f9416700 1 -- 172.21.15.63:0/2977054194 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e400b860 con 0x7f16f4071290 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.874+0000 7f16f9416700 1 -- 172.21.15.63:0/2977054194 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f16e400be50 con 0x7f16f4071290 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16f9416700 1 -- 172.21.15.63:0/2977054194 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e4010380 con 0x7f16f4071290 2024-04-24T22:35:46.472 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 -- 172.21.15.63:0/2977054194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 msgr2=0x7f16f4107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2977054194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098e0 tx=0x7f16e4009250 comp rx=0 tx=0).stop 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 -- 172.21.15.63:0/2977054194 shutdown_connections 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2977054194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f4107890 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 -- 172.21.15.63:0/2977054194 >> 172.21.15.63:0/2977054194 conn(0x7f16f40fcec0 msgr2=0x7f16f40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 -- 172.21.15.63:0/2977054194 shutdown_connections 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.875+0000 7f16fc67c700 1 -- 172.21.15.63:0/2977054194 wait complete. 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fc67c700 1 Processor -- start 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fc67c700 1 -- start start 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fc67c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fc67c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4113d00 con 0x7f16f4071290 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fa418700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fa418700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41258/0 (socket says 172.21.15.63:41258) 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fa418700 1 -- 172.21.15.63:0/2014842842 learned_addr learned my addr 172.21.15.63:0/2014842842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.876+0000 7f16fa418700 1 -- 172.21.15.63:0/2014842842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40094f0 con 0x7f16f4071290 2024-04-24T22:35:46.473 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16fa418700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006b00 tx=0x7f16e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e40107c0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f16e4010db0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e401acf0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f4113f00 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.877+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f4114a70 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f16e40193c0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f405f5c0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16eaffd700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 0x7f16e0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f16e404a6b0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16f9c17700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 0x7f16e0038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.878+0000 7f16f9c17700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 0x7f16e0038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009990 tx=0x7f16f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:42.883+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f16e4018b80 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.158+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f16f404bed0 con 0x7f16f4071290 2024-04-24T22:35:46.474 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.158+0000 7f16eaffd700 1 -- 172.21.15.63:0/2014842842 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f16e4018350 con 0x7f16f4071290 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 msgr2=0x7f16e0038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 0x7f16e0038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009990 tx=0x7f16f0006e40 comp rx=0 tx=0).stop 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 msgr2=0x7f16f41137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006b00 tx=0x7f16e4009250 comp rx=0 tx=0).stop 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 shutdown_connections 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f16e0036850 0x7f16e0038d00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 --2- 172.21.15.63:0/2014842842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4071290 0x7f16f41137c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 >> 172.21.15.63:0/2014842842 conn(0x7f16f40fcec0 msgr2=0x7f16f40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 shutdown_connections 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:43.161+0000 7f16fc67c700 1 -- 172.21.15.63:0/2014842842 wait complete. 2024-04-24T22:35:46.475 INFO:teuthology.orchestra.run.smithi063.stdout:mgr is available 2024-04-24T22:35:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:47 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4173747304' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T22:35:47.670 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:47.671 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73eac32700 1 Processor -- start 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73eac32700 1 -- start start 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73eac32700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73eac32700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4105980 con 0x7f73e4076c10 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73e89ce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73e89ce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41260/0 (socket says 172.21.15.63:41260) 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73e89ce700 1 -- 172.21.15.63:0/3386986212 learned_addr learned my addr 172.21.15.63:0/3386986212 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73e89ce700 1 -- 172.21.15.63:0/3386986212 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4105ac0 con 0x7f73e4076c10 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.892+0000 7f73e89ce700 1 --2- 172.21.15.63:0/3386986212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f73d80098e0 tx=0x7f73d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cdc6c90e36a7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73e37fe700 1 -- 172.21.15.63:0/3386986212 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d800b860 con 0x7f73e4076c10 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73e37fe700 1 -- 172.21.15.63:0/3386986212 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f73d800be50 con 0x7f73e4076c10 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73e37fe700 1 -- 172.21.15.63:0/3386986212 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d8010380 con 0x7f73e4076c10 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 -- 172.21.15.63:0/3386986212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 msgr2=0x7f73e4105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 --2- 172.21.15.63:0/3386986212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f73d80098e0 tx=0x7f73d8009250 comp rx=0 tx=0).stop 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 -- 172.21.15.63:0/3386986212 shutdown_connections 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 --2- 172.21.15.63:0/3386986212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4105440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 -- 172.21.15.63:0/3386986212 >> 172.21.15.63:0/3386986212 conn(0x7f73e40fcef0 msgr2=0x7f73e40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 -- 172.21.15.63:0/3386986212 shutdown_connections 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.893+0000 7f73eac32700 1 -- 172.21.15.63:0/3386986212 wait complete. 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73eac32700 1 Processor -- start 2024-04-24T22:35:47.673 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73eac32700 1 -- start start 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73eac32700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73eac32700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4111ad0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73e89ce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73e89ce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41276/0 (socket says 172.21.15.63:41276) 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.894+0000 7f73e89ce700 1 -- 172.21.15.63:0/4173747304 learned_addr learned my addr 172.21.15.63:0/4173747304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73e89ce700 1 -- 172.21.15.63:0/4173747304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80094f0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73e89ce700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006b00 tx=0x7f73d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d80107c0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f73d8010db0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d801acf0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e4111d30 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.895+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e4112840 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.896+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f73d80193c0 con 0x7f73e4076c10 2024-04-24T22:35:47.674 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.896+0000 7f73e17fa700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 0x7f73d4038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.896+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e405f5c0 con 0x7f73e4076c10 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.897+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f73d804a5c0 con 0x7f73e4076c10 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.897+0000 7f73e3fff700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 0x7f73d4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.897+0000 7f73e3fff700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 0x7f73d4038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009990 tx=0x7f73d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:46.901+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f73d8019670 con 0x7f73e4076c10 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.096+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f73e404bed0 con 0x7f73e4076c10 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.097+0000 7f73e17fa700 1 -- 172.21.15.63:0/4173747304 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f73d8049020 con 0x7f73e4076c10 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 msgr2=0x7f73d4038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 0x7f73d4038cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009990 tx=0x7f73d0006e40 comp rx=0 tx=0).stop 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 msgr2=0x7f73e4111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006b00 tx=0x7f73d8009250 comp rx=0 tx=0).stop 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 shutdown_connections 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f73d4036800 0x7f73d4038cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 --2- 172.21.15.63:0/4173747304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e4076c10 0x7f73e4111590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:47.675 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 >> 172.21.15.63:0/4173747304 conn(0x7f73e40fcef0 msgr2=0x7f73e40fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:47.676 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 shutdown_connections 2024-04-24T22:35:47.676 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:47.099+0000 7f73eac32700 1 -- 172.21.15.63:0/4173747304 wait complete. 2024-04-24T22:35:47.676 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling cephadm module... 2024-04-24T22:35:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:48 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1525345843' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c87a8700 1 Processor -- start 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c87a8700 1 -- start start 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c87a8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c87a8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c0102610 con 0x7f90c0101cc0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c6544700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c6544700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41292/0 (socket says 172.21.15.63:41292) 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c6544700 1 -- 172.21.15.63:0/21528024 learned_addr learned my addr 172.21.15.63:0/21528024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.036+0000 7f90c6544700 1 -- 172.21.15.63:0/21528024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0102750 con 0x7f90c0101cc0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c6544700 1 --2- 172.21.15.63:0/21528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00bb80 tx=0x7f90bc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b801e86eaa80c706 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c5542700 1 -- 172.21.15.63:0/21528024 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90bc00dd00 con 0x7f90c0101cc0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c5542700 1 -- 172.21.15.63:0/21528024 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f90bc00de60 con 0x7f90c0101cc0 2024-04-24T22:35:48.941 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c5542700 1 -- 172.21.15.63:0/21528024 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90bc0188a0 con 0x7f90c0101cc0 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c87a8700 1 -- 172.21.15.63:0/21528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 msgr2=0x7f90c01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c87a8700 1 --2- 172.21.15.63:0/21528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00bb80 tx=0x7f90bc00b4f0 comp rx=0 tx=0).stop 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c87a8700 1 -- 172.21.15.63:0/21528024 shutdown_connections 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c87a8700 1 --2- 172.21.15.63:0/21528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c01020d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.037+0000 7f90c87a8700 1 -- 172.21.15.63:0/21528024 >> 172.21.15.63:0/21528024 conn(0x7f90c00fc6f0 msgr2=0x7f90c00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 -- 172.21.15.63:0/21528024 shutdown_connections 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 -- 172.21.15.63:0/21528024 wait complete. 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 Processor -- start 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 -- start start 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c87a8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c010f8b0 con 0x7f90c0101cc0 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c6544700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c6544700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41296/0 (socket says 172.21.15.63:41296) 2024-04-24T22:35:48.942 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.038+0000 7f90c6544700 1 -- 172.21.15.63:0/1525345843 learned_addr learned my addr 172.21.15.63:0/1525345843 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90c6544700 1 -- 172.21.15.63:0/1525345843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc00b790 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90c6544700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90bc018600 tx=0x7f90bc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90bc010bd0 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c010fab0 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f90bc01d050 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90bc022590 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.039+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c0110620 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.040+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f90bc023470 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.040+0000 7f90b6ffd700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 0x7f90ac038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.040+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c005f5c0 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.040+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f90bc014070 con 0x7f90c0101cc0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.040+0000 7f90c5d43700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 0x7f90ac038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.041+0000 7f90c5d43700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 0x7f90ac038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009990 tx=0x7f90b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:48.943 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.045+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f90bc022ca0 con 0x7f90c0101cc0 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.293+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f90c004bed0 con 0x7f90c0101cc0 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.346+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f90bc00f3d0 con 0x7f90c0101cc0 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.347+0000 7f90b6ffd700 1 -- 172.21.15.63:0/1525345843 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f90bc01c6c0 con 0x7f90c0101cc0 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.349+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 msgr2=0x7f90ac038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.349+0000 7f90c87a8700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 0x7f90ac038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009990 tx=0x7f90b0006e40 comp rx=0 tx=0).stop 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.349+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 msgr2=0x7f90c010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.349+0000 7f90c87a8700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90bc018600 tx=0x7f90bc00b4f0 comp rx=0 tx=0).stop 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 shutdown_connections 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f90ac036850 0x7f90ac038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 --2- 172.21.15.63:0/1525345843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0101cc0 0x7f90c010f370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 >> 172.21.15.63:0/1525345843 conn(0x7f90c00fc6f0 msgr2=0x7f90c0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 shutdown_connections 2024-04-24T22:35:48.944 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:48.350+0000 7f90c87a8700 1 -- 172.21.15.63:0/1525345843 wait complete. 2024-04-24T22:35:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:49 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1525345843' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T22:35:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:49 smithi063 ceph-mon[26912]: mgrmap e5: smithi063.upcnvd(active, since 13s) 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.upcnvd", 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.366+0000 7fd9cfca7700 1 Processor -- start 2024-04-24T22:35:50.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.366+0000 7fd9cfca7700 1 -- start start 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cfca7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cfca7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8105900 con 0x7fd9c8076ca0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cda43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cda43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33020/0 (socket says 172.21.15.63:33020) 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cda43700 1 -- 172.21.15.63:0/501702948 learned_addr learned my addr 172.21.15.63:0/501702948 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cda43700 1 -- 172.21.15.63:0/501702948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8105a40 con 0x7fd9c8076ca0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cda43700 1 --2- 172.21.15.63:0/501702948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40098e0 tx=0x7fd9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1115d96d1e93f3af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cca41700 1 -- 172.21.15.63:0/501702948 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c400b860 con 0x7fd9c8076ca0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cca41700 1 -- 172.21.15.63:0/501702948 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd9c400be50 con 0x7fd9c8076ca0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.367+0000 7fd9cca41700 1 -- 172.21.15.63:0/501702948 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c4010380 con 0x7fd9c8076ca0 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 -- 172.21.15.63:0/501702948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 msgr2=0x7fd9c81053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/501702948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40098e0 tx=0x7fd9c4009250 comp rx=0 tx=0).stop 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 -- 172.21.15.63:0/501702948 shutdown_connections 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/501702948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81053c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:50.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 -- 172.21.15.63:0/501702948 >> 172.21.15.63:0/501702948 conn(0x7fd9c80fced0 msgr2=0x7fd9c80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 -- 172.21.15.63:0/501702948 shutdown_connections 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.368+0000 7fd9cfca7700 1 -- 172.21.15.63:0/501702948 wait complete. 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cfca7700 1 Processor -- start 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cfca7700 1 -- start start 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cfca7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cfca7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8111b10 con 0x7fd9c8076ca0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cda43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cda43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33034/0 (socket says 172.21.15.63:33034) 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.369+0000 7fd9cda43700 1 -- 172.21.15.63:0/2439769209 learned_addr learned my addr 172.21.15.63:0/2439769209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9cda43700 1 -- 172.21.15.63:0/2439769209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40094f0 con 0x7fd9c8076ca0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9cda43700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4003050 tx=0x7fd9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c4010760 con 0x7fd9c8076ca0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c8111d10 con 0x7fd9c8076ca0 2024-04-24T22:35:50.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd9c4010d50 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c401ae20 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.370+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c8112880 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c805f5c0 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd9c402e440 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9be7fc700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 0x7fd9b4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd9c404a500 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9cd242700 1 -- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 msgr2=0x7fd9b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.371+0000 7fd9cd242700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 0x7fd9b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.379+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd9c4018760 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.572+0000 7fd9cd242700 1 -- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 msgr2=0x7fd9b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.572+0000 7fd9cd242700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 0x7fd9b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.678+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd9c8112bc0 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.678+0000 7fd9be7fc700 1 -- 172.21.15.63:0/2439769209 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fd9c4018350 con 0x7fd9c8076ca0 2024-04-24T22:35:50.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 msgr2=0x7fd9b4038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 0x7fd9b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 msgr2=0x7fd9c81115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4003050 tx=0x7fd9c4009250 comp rx=0 tx=0).stop 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 shutdown_connections 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7fd9b4036830 0x7fd9b4038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 --2- 172.21.15.63:0/2439769209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8076ca0 0x7fd9c81115d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 >> 172.21.15.63:0/2439769209 conn(0x7fd9c80fced0 msgr2=0x7fd9c80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:50.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.680+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 shutdown_connections 2024-04-24T22:35:50.387 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:49.681+0000 7fd9cfca7700 1 -- 172.21.15.63:0/2439769209 wait complete. 2024-04-24T22:35:50.387 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-04-24T22:35:50.387 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 5... 2024-04-24T22:35:50.419 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:50 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2439769209' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T22:35:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:35:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:35:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T22:35:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: mgrmap e6: smithi063.upcnvd(active, starting, since 0.00368508s) 2024-04-24T22:35:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:35:55.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:55 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:35:56.412 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:56 smithi063 ceph-mon[26912]: Found migration_current of "None". Setting to last migration. 2024-04-24T22:35:56.412 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:56 smithi063 ceph-mon[26912]: mgrmap e7: smithi063.upcnvd(active, since 1.00498s) 2024-04-24T22:35:56.656 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.815+0000 7f74ac099700 1 Processor -- start 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.815+0000 7f74ac099700 1 -- start start 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.815+0000 7f74ac099700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.815+0000 7f74ac099700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4105970 con 0x7f74a4071310 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a9e35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a9e35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33040/0 (socket says 172.21.15.63:33040) 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a9e35700 1 -- 172.21.15.63:0/2993709045 learned_addr learned my addr 172.21.15.63:0/2993709045 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a9e35700 1 -- 172.21.15.63:0/2993709045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4105ab0 con 0x7f74a4071310 2024-04-24T22:35:56.657 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a9e35700 1 --2- 172.21.15.63:0/2993709045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74940098e0 tx=0x7f7494009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556037446a196579 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a8e33700 1 -- 172.21.15.63:0/2993709045 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749400b860 con 0x7f74a4071310 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a8e33700 1 -- 172.21.15.63:0/2993709045 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f749400be50 con 0x7f74a4071310 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.816+0000 7f74a8e33700 1 -- 172.21.15.63:0/2993709045 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7494010380 con 0x7f74a4071310 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 -- 172.21.15.63:0/2993709045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 msgr2=0x7f74a4105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 --2- 172.21.15.63:0/2993709045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74940098e0 tx=0x7f7494009250 comp rx=0 tx=0).stop 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 -- 172.21.15.63:0/2993709045 shutdown_connections 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 --2- 172.21.15.63:0/2993709045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4105430 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:56.658 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 -- 172.21.15.63:0/2993709045 >> 172.21.15.63:0/2993709045 conn(0x7f74a40fcea0 msgr2=0x7f74a40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 -- 172.21.15.63:0/2993709045 shutdown_connections 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.817+0000 7f74ac099700 1 -- 172.21.15.63:0/2993709045 wait complete. 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74ac099700 1 Processor -- start 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74ac099700 1 -- start start 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74ac099700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74ac099700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4111b40 con 0x7f74a4071310 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74a9e35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74a9e35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33044/0 (socket says 172.21.15.63:33044) 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74a9e35700 1 -- 172.21.15.63:0/1037210850 learned_addr learned my addr 172.21.15.63:0/1037210850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74a9e35700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940094f0 con 0x7f74a4071310 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f74a9e35700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7494006b00 tx=0x7f7494009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74940107c0 con 0x7f74a4071310 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.818+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7494010db0 con 0x7f74a4071310 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.819+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a4111d40 con 0x7f74a4071310 2024-04-24T22:35:56.659 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.819+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749401acf0 con 0x7f74a4071310 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.819+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a41128b0 con 0x7f74a4071310 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f7494010920 con 0x7f74a4071310 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f749a7fc700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7490039410 con 0x7f7490036850 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f749404a5e0 con 0x7f74a4071310 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f74a9634700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:50.820+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:35:56.660 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:51.021+0000 7f74a9634700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:51.021+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:51.422+0000 7f74a9634700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:51.422+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:52.223+0000 7f74a9634700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:52.223+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:53.824+0000 7f74a9634700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2614941819 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:53.824+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:55.099+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f7494016d00 con 0x7f74a4071310 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:55.099+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 msgr2=0x7f7490038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:55.099+0000 7f749a7fc700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.099+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f74940193c0 con 0x7f74a4071310 2024-04-24T22:35:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.100+0000 7f749a7fc700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 0x7f749003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.100+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7490039410 con 0x7f7490039ff0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.100+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 0x7f749003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.101+0000 7f74a9634700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 0x7f749003c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74a00039d0 tx=0x7f74a0009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.101+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f7490039410 con 0x7f7490039ff0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f74a41121d0 con 0x7f7490039ff0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f749a7fc700 1 -- 172.21.15.63:0/1037210850 <== mgr.14120 v2:172.21.15.63:6800/2691264506 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f74a41121d0 con 0x7f7490039ff0 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 msgr2=0x7f749003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f74ac099700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 0x7f749003c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74a00039d0 tx=0x7f74a0009350 comp rx=0 tx=0).stop 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 msgr2=0x7f74a4111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.112+0000 7f74ac099700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7494006b00 tx=0x7f7494009250 comp rx=0 tx=0).stop 2024-04-24T22:35:56.662 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 shutdown_connections 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f7490039ff0 0x7f749003c3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74a4071310 0x7f74a4111600 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 --2- 172.21.15.63:0/1037210850 >> [v2:172.21.15.63:6800/2614941819,v1:172.21.15.63:6801/2614941819] conn(0x7f7490036850 0x7f7490038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 >> 172.21.15.63:0/1037210850 conn(0x7f74a40fcea0 msgr2=0x7f74a40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 shutdown_connections 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:56.113+0000 7f74ac099700 1 -- 172.21.15.63:0/1037210850 wait complete. 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 5 is available 2024-04-24T22:35:56.663 INFO:teuthology.orchestra.run.smithi063.stdout:Setting orchestrator backend to cephadm... 2024-04-24T22:35:57.299 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:57 smithi063 ceph-mon[26912]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T22:35:57.299 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:57 smithi063 ceph-mon[26912]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f342aaa9700 1 Processor -- start 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f342aaa9700 1 -- start start 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f342aaa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f342aaa9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241031c0 con 0x7f3424102870 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f3428845700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f3428845700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33104/0 (socket says 172.21.15.63:33104) 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.063+0000 7f3428845700 1 -- 172.21.15.63:0/106162724 learned_addr learned my addr 172.21.15.63:0/106162724 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:57.868 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.064+0000 7f3428845700 1 -- 172.21.15.63:0/106162724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424103300 con 0x7f3424102870 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.064+0000 7f3428845700 1 --2- 172.21.15.63:0/106162724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f34180098e0 tx=0x7f3418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a9f60aca0a1985 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.064+0000 7f34237fe700 1 -- 172.21.15.63:0/106162724 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f341800b860 con 0x7f3424102870 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.064+0000 7f34237fe700 1 -- 172.21.15.63:0/106162724 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f341800be50 con 0x7f3424102870 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.064+0000 7f34237fe700 1 -- 172.21.15.63:0/106162724 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3418010380 con 0x7f3424102870 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- 172.21.15.63:0/106162724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 msgr2=0x7f3424102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 --2- 172.21.15.63:0/106162724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f34180098e0 tx=0x7f3418009250 comp rx=0 tx=0).stop 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- 172.21.15.63:0/106162724 shutdown_connections 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 --2- 172.21.15.63:0/106162724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424102c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:57.869 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- 172.21.15.63:0/106162724 >> 172.21.15.63:0/106162724 conn(0x7f34240fe030 msgr2=0x7f3424100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- 172.21.15.63:0/106162724 shutdown_connections 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- 172.21.15.63:0/106162724 wait complete. 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 Processor -- start 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.065+0000 7f342aaa9700 1 -- start start 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f342aaa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f342aaa9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3424079160 con 0x7f3424102870 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f3428845700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f3428845700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33110/0 (socket says 172.21.15.63:33110) 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f3428845700 1 -- 172.21.15.63:0/1539894310 learned_addr learned my addr 172.21.15.63:0/1539894310 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f3428845700 1 -- 172.21.15.63:0/1539894310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34180094f0 con 0x7f3424102870 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f3428845700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34180107c0 con 0x7f3424102870 2024-04-24T22:35:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3418010db0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240771d0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f341801ad00 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.066+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240776c0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.067+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f34180193c0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.067+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342404bed0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.068+0000 7f34217fa700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 0x7f3414038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.068+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f341804a4b0 con 0x7f3424102870 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.068+0000 7f3423fff700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 0x7f3414038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.068+0000 7f3423fff700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 0x7f3414038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3410009990 tx=0x7f3410006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.076+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3418022930 con 0x7f3424102870 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.153+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f341801a530 con 0x7f3424102870 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.288+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3424077e80 con 0x7f3414036720 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.297+0000 7f34217fa700 1 -- 172.21.15.63:0/1539894310 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3424077e80 con 0x7f3414036720 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 msgr2=0x7f3414038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 0x7f3414038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3410009990 tx=0x7f3410006e40 comp rx=0 tx=0).stop 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 msgr2=0x7f3424078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418009250 comp rx=0 tx=0).stop 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 shutdown_connections 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f3414036720 0x7f3414038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 --2- 172.21.15.63:0/1539894310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424102870 0x7f3424078c20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 >> 172.21.15.63:0/1539894310 conn(0x7f34240fe030 msgr2=0x7f34240feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 shutdown_connections 2024-04-24T22:35:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:57.300+0000 7f342aaa9700 1 -- 172.21.15.63:0/1539894310 wait complete. 2024-04-24T22:35:58.491 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: mgrmap e8: smithi063.upcnvd(active, since 2s) 2024-04-24T22:35:58.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:35:58.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:58.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:58.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:35:58.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:58 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:35:59.086 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T22:35:59.086 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6b08cd700 1 Processor -- start 2024-04-24T22:35:59.086 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6b08cd700 1 -- start start 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6b08cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6b08cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac105a40 con 0x7fd6ac076b70 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33116/0 (socket says 172.21.15.63:33116) 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.285+0000 7fd6aa59c700 1 -- 172.21.15.63:0/3821731198 learned_addr learned my addr 172.21.15.63:0/3821731198 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.286+0000 7fd6aa59c700 1 -- 172.21.15.63:0/3821731198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac105b80 con 0x7fd6ac076b70 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.286+0000 7fd6aa59c700 1 --2- 172.21.15.63:0/3821731198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0098e0 tx=0x7fd69c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595898d529d2d076 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.286+0000 7fd6a959a700 1 -- 172.21.15.63:0/3821731198 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd69c00b860 con 0x7fd6ac076b70 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.286+0000 7fd6a959a700 1 -- 172.21.15.63:0/3821731198 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd69c00be50 con 0x7fd6ac076b70 2024-04-24T22:35:59.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.286+0000 7fd6a959a700 1 -- 172.21.15.63:0/3821731198 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd69c010380 con 0x7fd6ac076b70 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- 172.21.15.63:0/3821731198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 msgr2=0x7fd6ac105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/3821731198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0098e0 tx=0x7fd69c009250 comp rx=0 tx=0).stop 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- 172.21.15.63:0/3821731198 shutdown_connections 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/3821731198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac105500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- 172.21.15.63:0/3821731198 >> 172.21.15.63:0/3821731198 conn(0x7fd6ac0fcef0 msgr2=0x7fd6ac0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- 172.21.15.63:0/3821731198 shutdown_connections 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- 172.21.15.63:0/3821731198 wait complete. 2024-04-24T22:35:59.088 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 Processor -- start 2024-04-24T22:35:59.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- start start 2024-04-24T22:35:59.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:59.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.287+0000 7fd6b08cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac103390 con 0x7fd6ac076b70 2024-04-24T22:35:59.089 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33126/0 (socket says 172.21.15.63:33126) 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6aa59c700 1 -- 172.21.15.63:0/4063164014 learned_addr learned my addr 172.21.15.63:0/4063164014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6aa59c700 1 -- 172.21.15.63:0/4063164014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0094f0 con 0x7fd6ac076b70 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6aa59c700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003050 tx=0x7fd69c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd69c010760 con 0x7fd6ac076b70 2024-04-24T22:35:59.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd69c010d50 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac101400 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd69c01ae20 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.288+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1018f0 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.289+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd69c0108c0 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.289+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac04bed0 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.289+0000 7fd69affd700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 0x7fd694038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.289+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd69c04a510 con 0x7fd6ac076b70 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.289+0000 7fd6a9d9b700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 0x7fd694038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.290+0000 7fd6a9d9b700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 0x7fd694038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009990 tx=0x7fd6a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:35:59.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.294+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd69c01a6a0 con 0x7fd6ac076b70 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.489+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ac102000 con 0x7fd694036830 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.490+0000 7fd69affd700 1 -- 172.21.15.63:0/4063164014 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd6ac102000 con 0x7fd694036830 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 msgr2=0x7fd694038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 0x7fd694038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009990 tx=0x7fd6a0006e40 comp rx=0 tx=0).stop 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 msgr2=0x7fd6ac102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003050 tx=0x7fd69c009250 comp rx=0 tx=0).stop 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 shutdown_connections 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fd694036830 0x7fd694038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 --2- 172.21.15.63:0/4063164014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac076b70 0x7fd6ac102e50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:35:59.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.492+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 >> 172.21.15.63:0/4063164014 conn(0x7fd6ac0fcef0 msgr2=0x7fd6ac0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:35:59.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.493+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 shutdown_connections 2024-04-24T22:35:59.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:58.493+0000 7fd6b08cd700 1 -- 172.21.15.63:0/4063164014 wait complete. 2024-04-24T22:35:59.093 INFO:teuthology.orchestra.run.smithi063.stdout:Generating ssh key... 2024-04-24T22:35:59.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:59 smithi063 ceph-mon[26912]: [24/Apr/2024:22:35:57] ENGINE Bus STARTING 2024-04-24T22:35:59.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:59 smithi063 ceph-mon[26912]: [24/Apr/2024:22:35:58] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:35:59.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:35:59 smithi063 ceph-mon[26912]: [24/Apr/2024:22:35:58] ENGINE Bus STARTED 2024-04-24T22:36:00.322 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:00 smithi063 ceph-mon[26912]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:00.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:00 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:00.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:00 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ecae1d700 1 Processor -- start 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ecae1d700 1 -- start start 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ecae1d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ecae1d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4105940 con 0x7f6ec4076c50 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ec8bb9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ec8bb9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60970/0 (socket says 172.21.15.63:60970) 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.483+0000 7f6ec8bb9700 1 -- 172.21.15.63:0/1559476508 learned_addr learned my addr 172.21.15.63:0/1559476508 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.484+0000 7f6ec8bb9700 1 -- 172.21.15.63:0/1559476508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4105a80 con 0x7f6ec4076c50 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.484+0000 7f6ec8bb9700 1 --2- 172.21.15.63:0/1559476508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80098e0 tx=0x7f6eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731dc1b300ba5511 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.484+0000 7f6ec37fe700 1 -- 172.21.15.63:0/1559476508 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eb800b860 con 0x7f6ec4076c50 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.484+0000 7f6ec37fe700 1 -- 172.21.15.63:0/1559476508 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6eb800be50 con 0x7f6ec4076c50 2024-04-24T22:36:00.331 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.484+0000 7f6ec37fe700 1 -- 172.21.15.63:0/1559476508 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eb8010380 con 0x7f6ec4076c50 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- 172.21.15.63:0/1559476508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 msgr2=0x7f6ec4105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/1559476508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80098e0 tx=0x7f6eb8009250 comp rx=0 tx=0).stop 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- 172.21.15.63:0/1559476508 shutdown_connections 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/1559476508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec4105400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- 172.21.15.63:0/1559476508 >> 172.21.15.63:0/1559476508 conn(0x7f6ec40fcef0 msgr2=0x7f6ec40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- 172.21.15.63:0/1559476508 shutdown_connections 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- 172.21.15.63:0/1559476508 wait complete. 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 Processor -- start 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.485+0000 7f6ecae1d700 1 -- start start 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ecae1d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ecae1d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4111b20 con 0x7f6ec4076c50 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec8bb9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec8bb9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60974/0 (socket says 172.21.15.63:60974) 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec8bb9700 1 -- 172.21.15.63:0/2085779516 learned_addr learned my addr 172.21.15.63:0/2085779516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:00.332 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec8bb9700 1 -- 172.21.15.63:0/2085779516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80094f0 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec8bb9700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8003050 tx=0x7f6eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eb8010760 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6eb8010d50 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.486+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec4111d20 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.487+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eb801ae20 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.487+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec4112890 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.487+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f6eb802e440 con 0x7f6ec4076c50 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.488+0000 7f6ec17fa700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 0x7f6eb4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:00.333 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.488+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6eb804a6a0 con 0x7f6ec4076c50 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.488+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec405f5c0 con 0x7f6ec4076c50 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.488+0000 7f6ec3fff700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 0x7f6eb4038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.488+0000 7f6ec3fff700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 0x7f6eb4038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009990 tx=0x7f6eb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.493+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6eb8018350 con 0x7f6ec4076c50 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.688+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ec40fe0b0 con 0x7f6eb4036830 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.809+0000 7f6ec17fa700 1 -- 172.21.15.63:0/2085779516 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6ec40fe0b0 con 0x7f6eb4036830 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 msgr2=0x7f6eb4038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 0x7f6eb4038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009990 tx=0x7f6eb0006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 msgr2=0x7f6ec41115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8003050 tx=0x7f6eb8009250 comp rx=0 tx=0).stop 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 shutdown_connections 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f6eb4036830 0x7f6eb4038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:00.334 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 --2- 172.21.15.63:0/2085779516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ec4076c50 0x7f6ec41115e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:00.335 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 >> 172.21.15.63:0/2085779516 conn(0x7f6ec40fcef0 msgr2=0x7f6ec40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:00.335 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 shutdown_connections 2024-04-24T22:36:00.335 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:35:59.812+0000 7f6ecae1d700 1 -- 172.21.15.63:0/2085779516 wait complete. 2024-04-24T22:36:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:01 smithi063 ceph-mon[26912]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:01 smithi063 ceph-mon[26912]: Generating ssh key... 2024-04-24T22:36:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLwh9MGMTi/2yiWovGCgAro3bTNvzNF5lKqQ7elb/BB5DLXAE27TuHUQA/VkhG/mirh9RA7oPkmU4m8Y1OsLL6VVAXFYMscflJ5knCH7kvjyekNRSX1WPXJZjmgw66ZlFcElFtz2Pr5NzllkUy0k96fhGwTgSG1CryLTrS2u+ikseJaV4M0DplhoXw6Wzrb1+AJkt3mBCwX/oH+eyFOy35L3N6l25SxGd/dKScfbqGar9J2FH6Vvlwfzqu/+mpaCmZwJBAGSbSNFGbmeNulbMeCPPXmlfGgYkIiFdqqB1w9Tc+B+jYWNi5klLp4GtLf2i7KFIUH5Ol9DB0rEVSK3Q6mSm0qdRfKp6vG08IrbBPj2BvTr888oEV1BxYKH/KhKmoLre37BFbcL29XtoiemgFvXdTpXFOnirYriDox2dwfTZqqP3PPLyHKW+nq1QXi5KWq/iYnAD028X7nyknuruQGBxQybu1r+cVi/Lz8t09llQF1XGw1IZk/fsfPLj7z5U= ceph-c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c7d316700 1 Processor -- start 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c7d316700 1 -- start start 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c7d316700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c7d316700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781031d0 con 0x7f9c78102880 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60990/0 (socket says 172.21.15.63:60990) 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c76ffd700 1 -- 172.21.15.63:0/834532239 learned_addr learned my addr 172.21.15.63:0/834532239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.727+0000 7f9c76ffd700 1 -- 172.21.15.63:0/834532239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78103310 con 0x7f9c78102880 2024-04-24T22:36:01.495 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c76ffd700 1 --2- 172.21.15.63:0/834532239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098e0 tx=0x7f9c68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f8e430b4b87b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c75ffb700 1 -- 172.21.15.63:0/834532239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c6800b860 con 0x7f9c78102880 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c75ffb700 1 -- 172.21.15.63:0/834532239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9c6800be50 con 0x7f9c78102880 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c75ffb700 1 -- 172.21.15.63:0/834532239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c68010380 con 0x7f9c78102880 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 -- 172.21.15.63:0/834532239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 msgr2=0x7f9c78102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 --2- 172.21.15.63:0/834532239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098e0 tx=0x7f9c68009250 comp rx=0 tx=0).stop 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 -- 172.21.15.63:0/834532239 shutdown_connections 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 --2- 172.21.15.63:0/834532239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78102c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 -- 172.21.15.63:0/834532239 >> 172.21.15.63:0/834532239 conn(0x7f9c780fe060 msgr2=0x7f9c78100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 -- 172.21.15.63:0/834532239 shutdown_connections 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.728+0000 7f9c7d316700 1 -- 172.21.15.63:0/834532239 wait complete. 2024-04-24T22:36:01.496 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.729+0000 7f9c7d316700 1 Processor -- start 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.729+0000 7f9c7d316700 1 -- start start 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.729+0000 7f9c7d316700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.729+0000 7f9c7d316700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78079160 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.729+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32774/0 (socket says 172.21.15.63:32774) 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c76ffd700 1 -- 172.21.15.63:0/3813252900 learned_addr learned my addr 172.21.15.63:0/3813252900 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c76ffd700 1 -- 172.21.15.63:0/3813252900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c680094f0 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c76ffd700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b440 tx=0x7f9c68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c68010760 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9c68010d50 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c780771d0 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c6801ae20 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.730+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c780776c0 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.731+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9c6802e440 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.731+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7804bed0 con 0x7f9c78102880 2024-04-24T22:36:01.497 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.731+0000 7f9c5f7fe700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 0x7f9c60038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.731+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9c6804a7e0 con 0x7f9c78102880 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.731+0000 7f9c767fc700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 0x7f9c60038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.732+0000 7f9c767fc700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 0x7f9c60038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009990 tx=0x7f9c6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.736+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9c68018940 con 0x7f9c78102880 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.932+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c78077dd0 con 0x7f9c60036830 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.933+0000 7f9c5f7fe700 1 -- 172.21.15.63:0/3813252900 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9c78077dd0 con 0x7f9c60036830 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 msgr2=0x7f9c60038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 0x7f9c60038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009990 tx=0x7f9c6c006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 msgr2=0x7f9c78078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b440 tx=0x7f9c68009250 comp rx=0 tx=0).stop 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 shutdown_connections 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f9c60036830 0x7f9c60038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 --2- 172.21.15.63:0/3813252900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c78102880 0x7f9c78078c20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:01.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 >> 172.21.15.63:0/3813252900 conn(0x7f9c780fe060 msgr2=0x7f9c780feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 shutdown_connections 2024-04-24T22:36:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:00.935+0000 7f9c7d316700 1 -- 172.21.15.63:0/3813252900 wait complete. 2024-04-24T22:36:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T22:36:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T22:36:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:Adding host smithi063... 2024-04-24T22:36:02.436 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:02 smithi063 ceph-mon[26912]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:03 smithi063 ceph-mon[26912]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:04 smithi063 ceph-mon[26912]: Deploying cephadm binary to smithi063 2024-04-24T22:36:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:04 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:04 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:04.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Added host 'smithi063' with addr '172.21.15.63' 2024-04-24T22:36:04.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d4768700 1 Processor -- start 2024-04-24T22:36:04.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d4768700 1 -- start start 2024-04-24T22:36:04.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d4768700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d4768700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1031c0 con 0x7f57cc102870 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d2504700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d2504700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32784/0 (socket says 172.21.15.63:32784) 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.939+0000 7f57d2504700 1 -- 172.21.15.63:0/3290966874 learned_addr learned my addr 172.21.15.63:0/3290966874 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d2504700 1 -- 172.21.15.63:0/3290966874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc103300 con 0x7f57cc102870 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d2504700 1 --2- 172.21.15.63:0/3290966874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098e0 tx=0x7f57c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585510c9902f57f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d1502700 1 -- 172.21.15.63:0/3290966874 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57c800b860 con 0x7f57cc102870 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d1502700 1 -- 172.21.15.63:0/3290966874 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f57c800be50 con 0x7f57cc102870 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d1502700 1 -- 172.21.15.63:0/3290966874 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57c8010380 con 0x7f57cc102870 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d4768700 1 -- 172.21.15.63:0/3290966874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 msgr2=0x7f57cc102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d4768700 1 --2- 172.21.15.63:0/3290966874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57c80098e0 tx=0x7f57c8009250 comp rx=0 tx=0).stop 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d4768700 1 -- 172.21.15.63:0/3290966874 shutdown_connections 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d4768700 1 --2- 172.21.15.63:0/3290966874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc102c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:04.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.940+0000 7f57d4768700 1 -- 172.21.15.63:0/3290966874 >> 172.21.15.63:0/3290966874 conn(0x7f57cc0fe030 msgr2=0x7f57cc100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 -- 172.21.15.63:0/3290966874 shutdown_connections 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 -- 172.21.15.63:0/3290966874 wait complete. 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 Processor -- start 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 -- start start 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d4768700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc079160 con 0x7f57cc102870 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d2504700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d2504700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32792/0 (socket says 172.21.15.63:32792) 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.941+0000 7f57d2504700 1 -- 172.21.15.63:0/3973473477 learned_addr learned my addr 172.21.15.63:0/3973473477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57d2504700 1 -- 172.21.15.63:0/3973473477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80094f0 con 0x7f57cc102870 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57d2504700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57c8000c00 tx=0x7f57c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57c80107c0 con 0x7f57cc102870 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f57c8010db0 con 0x7f57cc102870 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc0771d0 con 0x7f57cc102870 2024-04-24T22:36:04.597 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57c801ad00 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.942+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc0776c0 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.943+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f57c80193c0 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.943+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc04bed0 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.943+0000 7f57c2ffd700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 0x7f57b8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.943+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f57c804a630 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.943+0000 7f57d1d03700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 0x7f57b8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.944+0000 7f57d1d03700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 0x7f57b8038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009990 tx=0x7f57bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:01.948+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f57c8022930 con 0x7f57cc102870 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:02.147+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}) v1 -- 0x7f57cc077dd0 con 0x7f57b80368a0 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.037+0000 7f57c2ffd700 1 -- 172.21.15.63:0/3973473477 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f57cc077dd0 con 0x7f57b80368a0 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 msgr2=0x7f57b8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 0x7f57b8038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009990 tx=0x7f57bc006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 msgr2=0x7f57cc078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:04.598 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57c8000c00 tx=0x7f57c8009250 comp rx=0 tx=0).stop 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 shutdown_connections 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f57b80368a0 0x7f57b8038d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 --2- 172.21.15.63:0/3973473477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57cc102870 0x7f57cc078c20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.039+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 >> 172.21.15.63:0/3973473477 conn(0x7f57cc0fe030 msgr2=0x7f57cc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.040+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 shutdown_connections 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:04.040+0000 7f57d4768700 1 -- 172.21.15.63:0/3973473477 wait complete. 2024-04-24T22:36:04.599 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mon service with default placement... 2024-04-24T22:36:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:05 smithi063 ceph-mon[26912]: Added host smithi063 2024-04-24T22:36:06.018 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T22:36:06.018 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.038+0000 7f68170d5700 1 Processor -- start 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.038+0000 7f68170d5700 1 -- start start 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.038+0000 7f68170d5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.038+0000 7f68170d5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68101020a0 con 0x7f6810101750 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f6814e71700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f6814e71700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32808/0 (socket says 172.21.15.63:32808) 2024-04-24T22:36:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f6814e71700 1 -- 172.21.15.63:0/2287568102 learned_addr learned my addr 172.21.15.63:0/2287568102 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:06.020 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f6814e71700 1 -- 172.21.15.63:0/2287568102 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68101021e0 con 0x7f6810101750 2024-04-24T22:36:06.020 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f6814e71700 1 --2- 172.21.15.63:0/2287568102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f680400bb80 tx=0x7f680400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d38d71aa8827570 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:06.020 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f680f7fe700 1 -- 172.21.15.63:0/2287568102 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f680400dd00 con 0x7f6810101750 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f680f7fe700 1 -- 172.21.15.63:0/2287568102 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f680400de60 con 0x7f6810101750 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.039+0000 7f680f7fe700 1 -- 172.21.15.63:0/2287568102 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68040188a0 con 0x7f6810101750 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 -- 172.21.15.63:0/2287568102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 msgr2=0x7f6810101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 --2- 172.21.15.63:0/2287568102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f680400bb80 tx=0x7f680400b4f0 comp rx=0 tx=0).stop 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 -- 172.21.15.63:0/2287568102 shutdown_connections 2024-04-24T22:36:06.021 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 --2- 172.21.15.63:0/2287568102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f6810101b60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 -- 172.21.15.63:0/2287568102 >> 172.21.15.63:0/2287568102 conn(0x7f68100fcf30 msgr2=0x7f68100ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 -- 172.21.15.63:0/2287568102 shutdown_connections 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.040+0000 7f68170d5700 1 -- 172.21.15.63:0/2287568102 wait complete. 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f68170d5700 1 Processor -- start 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f68170d5700 1 -- start start 2024-04-24T22:36:06.022 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f68170d5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f68170d5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681010f890 con 0x7f6810101750 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f6814e71700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f6814e71700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32822/0 (socket says 172.21.15.63:32822) 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f6814e71700 1 -- 172.21.15.63:0/378635995 learned_addr learned my addr 172.21.15.63:0/378635995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f6814e71700 1 -- 172.21.15.63:0/378635995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680400b790 con 0x7f6810101750 2024-04-24T22:36:06.023 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.041+0000 7f6814e71700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f680400d650 tx=0x7f680400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.042+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6804010b50 con 0x7f6810101750 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.042+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f680401d050 con 0x7f6810101750 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.042+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681010fa90 con 0x7f6810101750 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.042+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810110600 con 0x7f6810101750 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.042+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6804022590 con 0x7f6810101750 2024-04-24T22:36:06.024 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f68040227a0 con 0x7f6810101750 2024-04-24T22:36:06.025 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f680d7fa700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 0x7f6800038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:06.025 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6804014070 con 0x7f6810101750 2024-04-24T22:36:06.025 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f40052e0 con 0x7f6810101750 2024-04-24T22:36:06.025 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f680ffff700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 0x7f6800038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:06.025 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.043+0000 7f680ffff700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 0x7f6800038d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009990 tx=0x7f67fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.048+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6804021d70 con 0x7f6810101750 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.280+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f67f4000bf0 con 0x7f68000368a0 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.288+0000 7f680d7fa700 1 -- 172.21.15.63:0/378635995 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f67f4000bf0 con 0x7f68000368a0 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 msgr2=0x7f6800038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 0x7f6800038d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009990 tx=0x7f67fc006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:06.026 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 msgr2=0x7f681010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f680400d650 tx=0x7f680400b4f0 comp rx=0 tx=0).stop 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 shutdown_connections 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f68000368a0 0x7f6800038d50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 --2- 172.21.15.63:0/378635995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6810101750 0x7f681010f350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 >> 172.21.15.63:0/378635995 conn(0x7f68100fcf30 msgr2=0x7f6810105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 shutdown_connections 2024-04-24T22:36:06.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:05.290+0000 7f68170d5700 1 -- 172.21.15.63:0/378635995 wait complete. 2024-04-24T22:36:06.028 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mgr service with default placement... 2024-04-24T22:36:06.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:06 smithi063 ceph-mon[26912]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:06.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:06 smithi063 ceph-mon[26912]: Saving service mon spec with placement count:5 2024-04-24T22:36:06.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:06 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:08.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T22:36:08.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbfde37700 1 Processor -- start 2024-04-24T22:36:08.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbfde37700 1 -- start start 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbfde37700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbfde37700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf81026b0 con 0x7fcbf8101d60 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbf77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbf77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32836/0 (socket says 172.21.15.63:32836) 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.664+0000 7fcbf77fe700 1 -- 172.21.15.63:0/1125182965 learned_addr learned my addr 172.21.15.63:0/1125182965 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.665+0000 7fcbf77fe700 1 -- 172.21.15.63:0/1125182965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf81027f0 con 0x7fcbf8101d60 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.665+0000 7fcbf77fe700 1 --2- 172.21.15.63:0/1125182965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bb80 tx=0x7fcbe800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf90b126ea7cfbf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.665+0000 7fcbf67fc700 1 -- 172.21.15.63:0/1125182965 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe800dd00 con 0x7fcbf8101d60 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.665+0000 7fcbf67fc700 1 -- 172.21.15.63:0/1125182965 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcbe800de60 con 0x7fcbf8101d60 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.665+0000 7fcbf67fc700 1 -- 172.21.15.63:0/1125182965 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe80188a0 con 0x7fcbf8101d60 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- 172.21.15.63:0/1125182965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 msgr2=0x7fcbf8102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1125182965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bb80 tx=0x7fcbe800b4f0 comp rx=0 tx=0).stop 2024-04-24T22:36:08.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- 172.21.15.63:0/1125182965 shutdown_connections 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1125182965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8102170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- 172.21.15.63:0/1125182965 >> 172.21.15.63:0/1125182965 conn(0x7fcbf80fc6f0 msgr2=0x7fcbf80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- 172.21.15.63:0/1125182965 shutdown_connections 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- 172.21.15.63:0/1125182965 wait complete. 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 Processor -- start 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.666+0000 7fcbfde37700 1 -- start start 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbfde37700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbfde37700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8113cb0 con 0x7fcbf8101d60 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbf77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbf77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32850/0 (socket says 172.21.15.63:32850) 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbf77fe700 1 -- 172.21.15.63:0/1138964830 learned_addr learned my addr 172.21.15.63:0/1138964830 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbf77fe700 1 -- 172.21.15.63:0/1138964830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe800b790 con 0x7fcbf8101d60 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbf77fe700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8010fa0 tx=0x7fcbe800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:08.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.667+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe8018e10 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.668+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcbe8010b80 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.668+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf8113eb0 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.668+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe8021550 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.668+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf8114a20 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.668+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcbe8021780 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.669+0000 7fcbdffff700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 0x7fcbe0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.669+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf805f5c0 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.669+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcbe8014070 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.669+0000 7fcbf6ffd700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 0x7fcbe0038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.669+0000 7fcbf6ffd700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 0x7fcbe0038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009990 tx=0x7fcbec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.674+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcbe800ed90 con 0x7fcbf8101d60 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.884+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbf81155a0 con 0x7fcbe00368a0 2024-04-24T22:36:08.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.888+0000 7fcbdffff700 1 -- 172.21.15.63:0/1138964830 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcbf81155a0 con 0x7fcbe00368a0 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 msgr2=0x7fcbe0038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 0x7fcbe0038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009990 tx=0x7fcbec006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 msgr2=0x7fcbf8113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8010fa0 tx=0x7fcbe800b4f0 comp rx=0 tx=0).stop 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 shutdown_connections 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fcbe00368a0 0x7fcbe0038d50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 --2- 172.21.15.63:0/1138964830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbf8101d60 0x7fcbf8113770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.890+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 >> 172.21.15.63:0/1138964830 conn(0x7fcbf80fc6f0 msgr2=0x7fcbf810a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.891+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 shutdown_connections 2024-04-24T22:36:08.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:06.891+0000 7fcbfde37700 1 -- 172.21.15.63:0/1138964830 wait complete. 2024-04-24T22:36:08.112 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying crash service with default placement... 2024-04-24T22:36:08.229 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:07 smithi063 ceph-mon[26912]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:08.229 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:07 smithi063 ceph-mon[26912]: Saving service mgr spec with placement count:2 2024-04-24T22:36:08.229 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:07 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:08.229 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:07 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:09 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:09 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:09.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-24T22:36:09.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f967de2b700 1 Processor -- start 2024-04-24T22:36:09.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f967de2b700 1 -- start start 2024-04-24T22:36:09.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f967de2b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:09.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f967de2b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96781031b0 con 0x7f9678102860 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f96777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f96777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32866/0 (socket says 172.21.15.63:32866) 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.533+0000 7f96777fe700 1 -- 172.21.15.63:0/2796916574 learned_addr learned my addr 172.21.15.63:0/2796916574 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f96777fe700 1 -- 172.21.15.63:0/2796916574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96781032f0 con 0x7f9678102860 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f96777fe700 1 --2- 172.21.15.63:0/2796916574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9670009e50 tx=0x7f9670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61727ab9bfe61252 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f96767fc700 1 -- 172.21.15.63:0/2796916574 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96700092e0 con 0x7f9678102860 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f96767fc700 1 -- 172.21.15.63:0/2796916574 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9670012430 con 0x7f9678102860 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f96767fc700 1 -- 172.21.15.63:0/2796916574 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96700075a0 con 0x7f9678102860 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f967de2b700 1 -- 172.21.15.63:0/2796916574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 msgr2=0x7f9678102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.534+0000 7f967de2b700 1 --2- 172.21.15.63:0/2796916574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9670009e50 tx=0x7f9670009840 comp rx=0 tx=0).stop 2024-04-24T22:36:09.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 -- 172.21.15.63:0/2796916574 shutdown_connections 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 --2- 172.21.15.63:0/2796916574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f9678102c70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 -- 172.21.15.63:0/2796916574 >> 172.21.15.63:0/2796916574 conn(0x7f96780fe060 msgr2=0x7f9678100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 -- 172.21.15.63:0/2796916574 shutdown_connections 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 -- 172.21.15.63:0/2796916574 wait complete. 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 Processor -- start 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.535+0000 7f967de2b700 1 -- start start 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f967de2b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f967de2b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678111b10 con 0x7f9678102860 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f96777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f96777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32882/0 (socket says 172.21.15.63:32882) 2024-04-24T22:36:09.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f96777fe700 1 -- 172.21.15.63:0/2110191795 learned_addr learned my addr 172.21.15.63:0/2110191795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f96777fe700 1 -- 172.21.15.63:0/2110191795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670009ae0 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f96777fe700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f96700127d0 tx=0x7f9670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96700092e0 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9670007d30 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9678111d10 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9670025960 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.536+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9678112120 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.537+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9670019050 con 0x7f9678102860 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.537+0000 7f966ffff700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 0x7f966403d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.538+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f967004a000 con 0x7f9678102860 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.538+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967804bed0 con 0x7f9678102860 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.538+0000 7f9676ffd700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 0x7f966403d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.538+0000 7f9676ffd700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 0x7f966403d160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9660009990 tx=0x7f9660006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.542+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9670017070 con 0x7f9678102860 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.769+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f96780ff200 con 0x7f966403acb0 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.774+0000 7f966ffff700 1 -- 172.21.15.63:0/2110191795 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f96780ff200 con 0x7f966403acb0 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 msgr2=0x7f966403d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 0x7f966403d160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9660009990 tx=0x7f9660006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:09.923 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 msgr2=0x7f96781115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f96700127d0 tx=0x7f9670009840 comp rx=0 tx=0).stop 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 shutdown_connections 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f966403acb0 0x7f966403d160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 --2- 172.21.15.63:0/2110191795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9678102860 0x7f96781115d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 >> 172.21.15.63:0/2110191795 conn(0x7f96780fe060 msgr2=0x7f96780feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 shutdown_connections 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:08.776+0000 7f967de2b700 1 -- 172.21.15.63:0/2110191795 wait complete. 2024-04-24T22:36:09.924 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying prometheus service with default placement... 2024-04-24T22:36:10.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:10 smithi063 ceph-mon[26912]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:10.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:10 smithi063 ceph-mon[26912]: Saving service crash spec with placement * 2024-04-24T22:36:10.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:10 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-24T22:36:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.424+0000 7fe215f64700 1 Processor -- start 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.424+0000 7fe215f64700 1 -- start start 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.424+0000 7fe215f64700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.424+0000 7fe215f64700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210102690 con 0x7fe210101d40 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54312/0 (socket says 172.21.15.63:54312) 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20f7fe700 1 -- 172.21.15.63:0/2368327522 learned_addr learned my addr 172.21.15.63:0/2368327522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20f7fe700 1 -- 172.21.15.63:0/2368327522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2101027d0 con 0x7fe210101d40 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20f7fe700 1 --2- 172.21.15.63:0/2368327522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098e0 tx=0x7fe200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2933827144a43539 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20e7fc700 1 -- 172.21.15.63:0/2368327522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20000b860 con 0x7fe210101d40 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20e7fc700 1 -- 172.21.15.63:0/2368327522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe20000be90 con 0x7fe210101d40 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.425+0000 7fe20e7fc700 1 -- 172.21.15.63:0/2368327522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2000103c0 con 0x7fe210101d40 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 -- 172.21.15.63:0/2368327522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 msgr2=0x7fe210102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 --2- 172.21.15.63:0/2368327522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098e0 tx=0x7fe200009250 comp rx=0 tx=0).stop 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 -- 172.21.15.63:0/2368327522 shutdown_connections 2024-04-24T22:36:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 --2- 172.21.15.63:0/2368327522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe210102150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 -- 172.21.15.63:0/2368327522 >> 172.21.15.63:0/2368327522 conn(0x7fe2100fc690 msgr2=0x7fe2100feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 -- 172.21.15.63:0/2368327522 shutdown_connections 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.426+0000 7fe215f64700 1 -- 172.21.15.63:0/2368327522 wait complete. 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe215f64700 1 Processor -- start 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe215f64700 1 -- start start 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe215f64700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe215f64700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210111b30 con 0x7fe210101d40 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54326/0 (socket says 172.21.15.63:54326) 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.427+0000 7fe20f7fe700 1 -- 172.21.15.63:0/1745826897 learned_addr learned my addr 172.21.15.63:0/1745826897 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe20f7fe700 1 -- 172.21.15.63:0/1745826897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000094f0 con 0x7fe210101d40 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe20f7fe700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe200011680 tx=0x7fe200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe200010890 con 0x7fe210101d40 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe200010ec0 con 0x7fe210101d40 2024-04-24T22:36:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe210111d30 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2000243f0 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.428+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101128a0 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.429+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe20002d3f0 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.430+0000 7fe1f7fff700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 0x7fe1f8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.430+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe20004a220 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.430+0000 7fe20effd700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 0x7fe1f8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.430+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21005f5c0 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.430+0000 7fe20effd700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 0x7fe1f8038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe204009990 tx=0x7fe204006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.435+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe20002b8e0 con 0x7fe210101d40 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.666+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2100fd850 con 0x7fe1f80368a0 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.670+0000 7fe1f7fff700 1 -- 172.21.15.63:0/1745826897 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe2100fd850 con 0x7fe1f80368a0 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 msgr2=0x7fe1f8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:11.277 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 0x7fe1f8038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe204009990 tx=0x7fe204006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 msgr2=0x7fe2101115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe200011680 tx=0x7fe200009250 comp rx=0 tx=0).stop 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 shutdown_connections 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fe1f80368a0 0x7fe1f8038d50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.672+0000 7fe215f64700 1 --2- 172.21.15.63:0/1745826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe210101d40 0x7fe2101115f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.673+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 >> 172.21.15.63:0/1745826897 conn(0x7fe2100fc690 msgr2=0x7fe2100fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.673+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 shutdown_connections 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:10.673+0000 7fe215f64700 1 -- 172.21.15.63:0/1745826897 wait complete. 2024-04-24T22:36:11.278 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying grafana service with default placement... 2024-04-24T22:36:11.676 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:11 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:11.677 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:11 smithi063 ceph-mon[26912]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:11.677 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:11 smithi063 ceph-mon[26912]: Saving service prometheus spec with placement count:1 2024-04-24T22:36:11.677 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:11 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:13.219 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:12 smithi063 ceph-mon[26912]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:13.219 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:12 smithi063 ceph-mon[26912]: Saving service grafana spec with placement count:1 2024-04-24T22:36:13.219 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:12 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.710+0000 7f36d2b7c700 1 Processor -- start 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d2b7c700 1 -- start start 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d2b7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d2b7c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc1055c0 con 0x7f36cc104be0 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d0918700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d0918700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54338/0 (socket says 172.21.15.63:54338) 2024-04-24T22:36:13.477 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d0918700 1 -- 172.21.15.63:0/1949131018 learned_addr learned my addr 172.21.15.63:0/1949131018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.711+0000 7f36d0918700 1 -- 172.21.15.63:0/1949131018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc105700 con 0x7f36cc104be0 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.712+0000 7f36d0918700 1 --2- 172.21.15.63:0/1949131018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098e0 tx=0x7f36c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69bc4161e614c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.712+0000 7f36cb7fe700 1 -- 172.21.15.63:0/1949131018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36c000b860 con 0x7f36cc104be0 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.712+0000 7f36cb7fe700 1 -- 172.21.15.63:0/1949131018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c000be90 con 0x7f36cc104be0 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.712+0000 7f36cb7fe700 1 -- 172.21.15.63:0/1949131018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36c00103c0 con 0x7f36cc104be0 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 -- 172.21.15.63:0/1949131018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 msgr2=0x7f36cc104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/1949131018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098e0 tx=0x7f36c0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 -- 172.21.15.63:0/1949131018 shutdown_connections 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/1949131018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc104ff0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 -- 172.21.15.63:0/1949131018 >> 172.21.15.63:0/1949131018 conn(0x7f36cc078300 msgr2=0x7f36cc078700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 -- 172.21.15.63:0/1949131018 shutdown_connections 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.713+0000 7f36d2b7c700 1 -- 172.21.15.63:0/1949131018 wait complete. 2024-04-24T22:36:13.478 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d2b7c700 1 Processor -- start 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d2b7c700 1 -- start start 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d2b7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d2b7c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc113cc0 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d0918700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d0918700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54344/0 (socket says 172.21.15.63:54344) 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d0918700 1 -- 172.21.15.63:0/2743944226 learned_addr learned my addr 172.21.15.63:0/2743944226 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.714+0000 7f36d0918700 1 -- 172.21.15.63:0/2743944226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00094f0 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36d0918700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36c00090b0 tx=0x7f36c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36c0010840 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c0010e70 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc113ec0 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36c0024e40 con 0x7f36cc104be0 2024-04-24T22:36:13.479 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.715+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc114a30 con 0x7f36cc104be0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.716+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f36c001a570 con 0x7f36cc104be0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.716+0000 7f36c97fa700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 0x7f36bc038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.716+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f36c004f590 con 0x7f36cc104be0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.716+0000 7f36cbfff700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 0x7f36bc038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.717+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc05f5c0 con 0x7f36cc104be0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.717+0000 7f36cbfff700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 0x7f36bc038cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009990 tx=0x7f36b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.721+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36c0050060 con 0x7f36cc104be0 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.930+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f36cc102b10 con 0x7f36bc036800 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.935+0000 7f36c97fa700 1 -- 172.21.15.63:0/2743944226 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f36cc102b10 con 0x7f36bc036800 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 msgr2=0x7f36bc038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 0x7f36bc038cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009990 tx=0x7f36b8006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 msgr2=0x7f36cc113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36c00090b0 tx=0x7f36c0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:13.480 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 shutdown_connections 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f36bc036800 0x7f36bc038cb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 --2- 172.21.15.63:0/2743944226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36cc104be0 0x7f36cc113780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 >> 172.21.15.63:0/2743944226 conn(0x7f36cc078300 msgr2=0x7f36cc102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 shutdown_connections 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:11.937+0000 7f36d2b7c700 1 -- 172.21.15.63:0/2743944226 wait complete. 2024-04-24T22:36:13.481 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying node-exporter service with default placement... 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.864+0000 7f46e5787700 1 Processor -- start 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.864+0000 7f46e5787700 1 -- start start 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46e5787700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46e5787700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0103160 con 0x7f46e0102810 2024-04-24T22:36:14.519 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46deffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46deffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54346/0 (socket says 172.21.15.63:54346) 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46deffd700 1 -- 172.21.15.63:0/654448180 learned_addr learned my addr 172.21.15.63:0/654448180 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46deffd700 1 -- 172.21.15.63:0/654448180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e01032a0 con 0x7f46e0102810 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.865+0000 7f46deffd700 1 --2- 172.21.15.63:0/654448180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46d00098e0 tx=0x7f46d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b41fea9474ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46ddffb700 1 -- 172.21.15.63:0/654448180 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d000b860 con 0x7f46e0102810 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46ddffb700 1 -- 172.21.15.63:0/654448180 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46d000be90 con 0x7f46e0102810 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46ddffb700 1 -- 172.21.15.63:0/654448180 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d00103c0 con 0x7f46e0102810 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 -- 172.21.15.63:0/654448180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 msgr2=0x7f46e0102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 --2- 172.21.15.63:0/654448180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46d00098e0 tx=0x7f46d0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 -- 172.21.15.63:0/654448180 shutdown_connections 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 --2- 172.21.15.63:0/654448180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0102c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 -- 172.21.15.63:0/654448180 >> 172.21.15.63:0/654448180 conn(0x7f46e00fdfd0 msgr2=0x7f46e01003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 -- 172.21.15.63:0/654448180 shutdown_connections 2024-04-24T22:36:14.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.866+0000 7f46e5787700 1 -- 172.21.15.63:0/654448180 wait complete. 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46e5787700 1 Processor -- start 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46e5787700 1 -- start start 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46e5787700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46e5787700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0079160 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46deffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46deffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54350/0 (socket says 172.21.15.63:54350) 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.867+0000 7f46deffd700 1 -- 172.21.15.63:0/2428431409 learned_addr learned my addr 172.21.15.63:0/2428431409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46deffd700 1 -- 172.21.15.63:0/2428431409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00094f0 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46deffd700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46d0006b00 tx=0x7f46d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d0010840 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46d0010e70 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d001ae70 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e00771d0 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.868+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e00776c0 con 0x7f46e0102810 2024-04-24T22:36:14.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f46d00244d0 con 0x7f46e0102810 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46c77fe700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 0x7f46c8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e004bed0 con 0x7f46e0102810 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f46d004ac40 con 0x7f46e0102810 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46de7fc700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 0x7f46c8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.869+0000 7f46de7fc700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 0x7f46c8038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009990 tx=0x7f46d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:13.874+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f46d0018350 con 0x7f46e0102810 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.081+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f46e0077dd0 con 0x7f46c8036850 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.085+0000 7f46c77fe700 1 -- 172.21.15.63:0/2428431409 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f46e0077dd0 con 0x7f46c8036850 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 msgr2=0x7f46c8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 0x7f46c8038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009990 tx=0x7f46d4006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 msgr2=0x7f46e0078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46d0006b00 tx=0x7f46d0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 shutdown_connections 2024-04-24T22:36:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f46c8036850 0x7f46c8038d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:14.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 --2- 172.21.15.63:0/2428431409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0102810 0x7f46e0078c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:14.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 >> 172.21.15.63:0/2428431409 conn(0x7f46e00fdfd0 msgr2=0x7f46e00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:14.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 shutdown_connections 2024-04-24T22:36:14.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.087+0000 7f46e5787700 1 -- 172.21.15.63:0/2428431409 wait complete. 2024-04-24T22:36:14.523 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying alertmanager service with default placement... 2024-04-24T22:36:15.166 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:15 smithi063 ceph-mon[26912]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:15.166 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:15 smithi063 ceph-mon[26912]: Saving service node-exporter spec with placement * 2024-04-24T22:36:15.167 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:15 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.945+0000 7ffab11a7700 1 Processor -- start 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.945+0000 7ffab11a7700 1 -- start start 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffab11a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffab11a7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac103690 con 0x7ffaac102cb0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaaad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaaad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54362/0 (socket says 172.21.15.63:54362) 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaaad9d700 1 -- 172.21.15.63:0/3639747245 learned_addr learned my addr 172.21.15.63:0/3639747245 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaaad9d700 1 -- 172.21.15.63:0/3639747245 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac1037d0 con 0x7ffaac102cb0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaaad9d700 1 --2- 172.21.15.63:0/3639747245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00098e0 tx=0x7ffaa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acbed89d9820e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaa9d9b700 1 -- 172.21.15.63:0/3639747245 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa000b860 con 0x7ffaac102cb0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaa9d9b700 1 -- 172.21.15.63:0/3639747245 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaa000be90 con 0x7ffaac102cb0 2024-04-24T22:36:15.720 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.946+0000 7ffaa9d9b700 1 -- 172.21.15.63:0/3639747245 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa00103c0 con 0x7ffaac102cb0 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 -- 172.21.15.63:0/3639747245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 msgr2=0x7ffaac1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 --2- 172.21.15.63:0/3639747245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00098e0 tx=0x7ffaa0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 -- 172.21.15.63:0/3639747245 shutdown_connections 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 --2- 172.21.15.63:0/3639747245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac1030c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 -- 172.21.15.63:0/3639747245 >> 172.21.15.63:0/3639747245 conn(0x7ffaac0fc6f0 msgr2=0x7ffaac0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 -- 172.21.15.63:0/3639747245 shutdown_connections 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.947+0000 7ffab11a7700 1 -- 172.21.15.63:0/3639747245 wait complete. 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffab11a7700 1 Processor -- start 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffab11a7700 1 -- start start 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffab11a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffab11a7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac113d80 con 0x7ffaac102cb0 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffaaad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffaaad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54372/0 (socket says 172.21.15.63:54372) 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffaaad9d700 1 -- 172.21.15.63:0/1598346856 learned_addr learned my addr 172.21.15.63:0/1598346856 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:15.721 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffaaad9d700 1 -- 172.21.15.63:0/1598346856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00094f0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.948+0000 7ffaaad9d700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00094c0 tx=0x7ffaa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.949+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa0010890 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.949+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaa0010ec0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.949+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac113f80 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.949+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa00243f0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.949+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac114af0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ffaa00109f0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac05f5c0 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffa937fe700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 0x7ffa94038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffaa004a230 con 0x7ffaac102cb0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffaaa59c700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 0x7ffa94038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.950+0000 7ffaaa59c700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 0x7ffa94038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009990 tx=0x7ffa9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:15.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:14.954+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffaa002b8c0 con 0x7ffaac102cb0 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.164+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaac0fd8e0 con 0x7ffa940368a0 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.169+0000 7ffa937fe700 1 -- 172.21.15.63:0/1598346856 <== mgr.14120 v2:172.21.15.63:6800/2691264506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7ffaac0fd8e0 con 0x7ffa940368a0 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 msgr2=0x7ffa94038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 0x7ffa94038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009990 tx=0x7ffa9c006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 msgr2=0x7ffaac113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00094c0 tx=0x7ffaa0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 shutdown_connections 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7ffa940368a0 0x7ffa94038d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 --2- 172.21.15.63:0/1598346856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaac102cb0 0x7ffaac113840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 >> 172.21.15.63:0/1598346856 conn(0x7ffaac0fc6f0 msgr2=0x7ffaac0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 shutdown_connections 2024-04-24T22:36:15.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:15.171+0000 7ffab11a7700 1 -- 172.21.15.63:0/1598346856 wait complete. 2024-04-24T22:36:16.332 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:16 smithi063 ceph-mon[26912]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:16.332 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:16 smithi063 ceph-mon[26912]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:16.332 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:16 smithi063 ceph-mon[26912]: Saving service alertmanager spec with placement count:1 2024-04-24T22:36:16.332 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:16 smithi063 ceph-mon[26912]: from='mgr.14120 172.21.15.63:0/3832152146' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:16.899 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.106+0000 7f1737bd0700 1 Processor -- start 2024-04-24T22:36:16.899 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.106+0000 7f1737bd0700 1 -- start start 2024-04-24T22:36:16.899 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.106+0000 7f1737bd0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:16.899 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.106+0000 7f1737bd0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1730079070 con 0x7f1730078c60 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.106+0000 7f173596c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173596c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54378/0 (socket says 172.21.15.63:54378) 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173596c700 1 -- 172.21.15.63:0/3789514436 learned_addr learned my addr 172.21.15.63:0/3789514436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173596c700 1 -- 172.21.15.63:0/3789514436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1730077750 con 0x7f1730078c60 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173596c700 1 --2- 172.21.15.63:0/3789514436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1728009e50 tx=0x7f1728009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d517a272b11f2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173496a700 1 -- 172.21.15.63:0/3789514436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17280092e0 con 0x7f1730078c60 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173496a700 1 -- 172.21.15.63:0/3789514436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1728012430 con 0x7f1730078c60 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.107+0000 7f173496a700 1 -- 172.21.15.63:0/3789514436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17280075a0 con 0x7f1730078c60 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- 172.21.15.63:0/3789514436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 msgr2=0x7f1730077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 --2- 172.21.15.63:0/3789514436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1728009e50 tx=0x7f1728009840 comp rx=0 tx=0).stop 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- 172.21.15.63:0/3789514436 shutdown_connections 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 --2- 172.21.15.63:0/3789514436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730077210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- 172.21.15.63:0/3789514436 >> 172.21.15.63:0/3789514436 conn(0x7f17300fe060 msgr2=0x7f1730100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- 172.21.15.63:0/3789514436 shutdown_connections 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- 172.21.15.63:0/3789514436 wait complete. 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 Processor -- start 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.108+0000 7f1737bd0700 1 -- start start 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f1737bd0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f1737bd0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1730113d80 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f173596c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f173596c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54386/0 (socket says 172.21.15.63:54386) 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f173596c700 1 -- 172.21.15.63:0/139266634 learned_addr learned my addr 172.21.15.63:0/139266634 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f173596c700 1 -- 172.21.15.63:0/139266634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009ae0 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f173596c700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f1728009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17280092e0 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1728007ce0 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1730113f80 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17280259c0 con 0x7f1730078c60 2024-04-24T22:36:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.109+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1730114af0 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.110+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1728019050 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.110+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173005f5c0 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.110+0000 7f172e7fc700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 0x7f1718038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.111+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f172804a0c0 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.111+0000 7f173516b700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 0x7f1718038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.111+0000 7f173516b700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 0x7f1718038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1724009990 tx=0x7f1724006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.115+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f172800be50 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.312+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f173004bed0 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.317+0000 7f172e7fc700 1 -- 172.21.15.63:0/139266634 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1728020030 con 0x7f1730078c60 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 msgr2=0x7f1718038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 0x7f1718038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1724009990 tx=0x7f1724006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 msgr2=0x7f1730113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:16.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f1728009840 comp rx=0 tx=0).stop 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 shutdown_connections 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f1718036850 0x7f1718038d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 --2- 172.21.15.63:0/139266634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1730078c60 0x7f1730113840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 >> 172.21.15.63:0/139266634 conn(0x7f17300fe060 msgr2=0x7f17300feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.319+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 shutdown_connections 2024-04-24T22:36:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:16.320+0000 7f1737bd0700 1 -- 172.21.15.63:0/139266634 wait complete. 2024-04-24T22:36:17.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:17 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/139266634' entity='client.admin' 2024-04-24T22:36:18.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.382+0000 7fdca5d19700 1 Processor -- start 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.382+0000 7fdca5d19700 1 -- start start 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.382+0000 7fdca5d19700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.382+0000 7fdca5d19700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca0079040 con 0x7fdca0078c30 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54398/0 (socket says 172.21.15.63:54398) 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9f7fe700 1 -- 172.21.15.63:0/3995967496 learned_addr learned my addr 172.21.15.63:0/3995967496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9f7fe700 1 -- 172.21.15.63:0/3995967496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0077770 con 0x7fdca0078c30 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9f7fe700 1 --2- 172.21.15.63:0/3995967496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009e50 tx=0x7fdc98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f78b51e7dc0078 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9e7fc700 1 -- 172.21.15.63:0/3995967496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc980092e0 con 0x7fdca0078c30 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9e7fc700 1 -- 172.21.15.63:0/3995967496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc98012430 con 0x7fdca0078c30 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.383+0000 7fdc9e7fc700 1 -- 172.21.15.63:0/3995967496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc980075a0 con 0x7fdca0078c30 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 -- 172.21.15.63:0/3995967496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 msgr2=0x7fdca0077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:18.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 --2- 172.21.15.63:0/3995967496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009e50 tx=0x7fdc98009840 comp rx=0 tx=0).stop 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 -- 172.21.15.63:0/3995967496 shutdown_connections 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 --2- 172.21.15.63:0/3995967496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0077230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 -- 172.21.15.63:0/3995967496 >> 172.21.15.63:0/3995967496 conn(0x7fdca00fe030 msgr2=0x7fdca0100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 -- 172.21.15.63:0/3995967496 shutdown_connections 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 -- 172.21.15.63:0/3995967496 wait complete. 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.384+0000 7fdca5d19700 1 Processor -- start 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdca5d19700 1 -- start start 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdca5d19700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdca5d19700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca0113d80 con 0x7fdca0078c30 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54400/0 (socket says 172.21.15.63:54400) 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc9f7fe700 1 -- 172.21.15.63:0/442972185 learned_addr learned my addr 172.21.15.63:0/442972185 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:18.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc9f7fe700 1 -- 172.21.15.63:0/442972185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009ae0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc9f7fe700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009fd0 tx=0x7fdc98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.385+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc980092e0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca0113f80 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc98007ce0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc980259c0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca0114af0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdc98019050 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.386+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca005f5c0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.387+0000 7fdc97fff700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 0x7fdc8c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.387+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdc9804a060 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.387+0000 7fdc9effd700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 0x7fdc8c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.387+0000 7fdc9effd700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 0x7fdc8c038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009990 tx=0x7fdc88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.391+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdc9802d9a0 con 0x7fdca0078c30 2024-04-24T22:36:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.585+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fdca004bed0 con 0x7fdca0078c30 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.591+0000 7fdc97fff700 1 -- 172.21.15.63:0/442972185 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fdc98020030 con 0x7fdca0078c30 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 msgr2=0x7fdc8c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 0x7fdc8c038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009990 tx=0x7fdc88006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 msgr2=0x7fdca0113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009fd0 tx=0x7fdc98009840 comp rx=0 tx=0).stop 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 shutdown_connections 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fdc8c036850 0x7fdc8c038d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 --2- 172.21.15.63:0/442972185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca0078c30 0x7fdca0113840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 >> 172.21.15.63:0/442972185 conn(0x7fdca00fe030 msgr2=0x7fdca00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 shutdown_connections 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:17.593+0000 7fdca5d19700 1 -- 172.21.15.63:0/442972185 wait complete. 2024-04-24T22:36:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling the dashboard module... 2024-04-24T22:36:18.601 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:18 smithi063 ceph-mon[26912]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:18.601 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:18 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/442972185' entity='client.admin' 2024-04-24T22:36:19.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:19 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3497459326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T22:36:19.890 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826eb19700 1 Processor -- start 2024-04-24T22:36:19.890 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826eb19700 1 -- start start 2024-04-24T22:36:19.890 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826eb19700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.890 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826eb19700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268103b40 con 0x7f8268100c90 2024-04-24T22:36:19.890 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826c8b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826c8b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40188/0 (socket says 172.21.15.63:40188) 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826c8b5700 1 -- 172.21.15.63:0/1077244752 learned_addr learned my addr 172.21.15.63:0/1077244752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.626+0000 7f826c8b5700 1 -- 172.21.15.63:0/1077244752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82681015e0 con 0x7f8268100c90 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826c8b5700 1 --2- 172.21.15.63:0/1077244752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098e0 tx=0x7f825c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b300cc1f26c5a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f82677fe700 1 -- 172.21.15.63:0/1077244752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c00b860 con 0x7f8268100c90 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f82677fe700 1 -- 172.21.15.63:0/1077244752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f825c00be90 con 0x7f8268100c90 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f82677fe700 1 -- 172.21.15.63:0/1077244752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0103c0 con 0x7f8268100c90 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826eb19700 1 -- 172.21.15.63:0/1077244752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 msgr2=0x7f82681010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826eb19700 1 --2- 172.21.15.63:0/1077244752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098e0 tx=0x7f825c009250 comp rx=0 tx=0).stop 2024-04-24T22:36:19.891 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826eb19700 1 -- 172.21.15.63:0/1077244752 shutdown_connections 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826eb19700 1 --2- 172.21.15.63:0/1077244752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681010a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.627+0000 7f826eb19700 1 -- 172.21.15.63:0/1077244752 >> 172.21.15.63:0/1077244752 conn(0x7f82680fc6b0 msgr2=0x7f82680feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 -- 172.21.15.63:0/1077244752 shutdown_connections 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 -- 172.21.15.63:0/1077244752 wait complete. 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 Processor -- start 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 -- start start 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.628+0000 7f826eb19700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268113cf0 con 0x7f8268100c90 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826c8b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826c8b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40204/0 (socket says 172.21.15.63:40204) 2024-04-24T22:36:19.904 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826c8b5700 1 -- 172.21.15.63:0/3497459326 learned_addr learned my addr 172.21.15.63:0/3497459326 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826c8b5700 1 -- 172.21.15.63:0/3497459326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0094f0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826c8b5700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f825c011680 tx=0x7f825c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c010890 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8268113ef0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f825c010ec0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0243f0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.629+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8268114a60 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.630+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f825c02d3f0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.630+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826805f5c0 con 0x7f8268100c90 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.630+0000 7f82657fa700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 0x7f8258038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.630+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f825c04a410 con 0x7f8268100c90 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.630+0000 7f8267fff700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 0x7f8258038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.631+0000 7f8267fff700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 0x7f8258038d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.635+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f825c02b8e0 con 0x7f8268100c90 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:18.883+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f826804bed0 con 0x7f8268100c90 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.326+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f825c01a8c0 con 0x7f8268100c90 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.328+0000 7f82657fa700 1 -- 172.21.15.63:0/3497459326 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f825c02d7b0 con 0x7f8268100c90 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 msgr2=0x7f8258038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.906 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 0x7f8258038d50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 msgr2=0x7f82681137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f825c011680 tx=0x7f825c009250 comp rx=0 tx=0).stop 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 shutdown_connections 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f82580368a0 0x7f8258038d50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 --2- 172.21.15.63:0/3497459326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8268100c90 0x7f82681137b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 >> 172.21.15.63:0/3497459326 conn(0x7f82680fc6b0 msgr2=0x7f82680fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 shutdown_connections 2024-04-24T22:36:19.907 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:19.330+0000 7f826eb19700 1 -- 172.21.15.63:0/3497459326 wait complete. 2024-04-24T22:36:20.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:20 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3497459326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T22:36:20.513 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:20 smithi063 ceph-mon[26912]: mgrmap e9: smithi063.upcnvd(active, since 24s) 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.upcnvd", 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac43f6d700 1 Processor -- start 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac43f6d700 1 -- start start 2024-04-24T22:36:21.170 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac43f6d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac43f6d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c079060 con 0x7fac3c078c50 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac41d09700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac41d09700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40234/0 (socket says 172.21.15.63:40234) 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.304+0000 7fac41d09700 1 -- 172.21.15.63:0/3312528832 learned_addr learned my addr 172.21.15.63:0/3312528832 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.305+0000 7fac41d09700 1 -- 172.21.15.63:0/3312528832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c077790 con 0x7fac3c078c50 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.305+0000 7fac41d09700 1 --2- 172.21.15.63:0/3312528832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac380098e0 tx=0x7fac38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99af81b4bc1d8682 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.305+0000 7fac40d07700 1 -- 172.21.15.63:0/3312528832 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac3800b860 con 0x7fac3c078c50 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.305+0000 7fac40d07700 1 -- 172.21.15.63:0/3312528832 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac3800be90 con 0x7fac3c078c50 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.305+0000 7fac40d07700 1 -- 172.21.15.63:0/3312528832 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac380103c0 con 0x7fac3c078c50 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 -- 172.21.15.63:0/3312528832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 msgr2=0x7fac3c077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 --2- 172.21.15.63:0/3312528832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac380098e0 tx=0x7fac38009250 comp rx=0 tx=0).stop 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 -- 172.21.15.63:0/3312528832 shutdown_connections 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 --2- 172.21.15.63:0/3312528832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c077250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 -- 172.21.15.63:0/3312528832 >> 172.21.15.63:0/3312528832 conn(0x7fac3c0fe090 msgr2=0x7fac3c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 -- 172.21.15.63:0/3312528832 shutdown_connections 2024-04-24T22:36:21.171 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 -- 172.21.15.63:0/3312528832 wait complete. 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.306+0000 7fac43f6d700 1 Processor -- start 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac43f6d700 1 -- start start 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac43f6d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac43f6d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c113d80 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac41d09700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac41d09700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40250/0 (socket says 172.21.15.63:40250) 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac41d09700 1 -- 172.21.15.63:0/2302185261 learned_addr learned my addr 172.21.15.63:0/2302185261 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac41d09700 1 -- 172.21.15.63:0/2302185261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac380094f0 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac41d09700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac38006d40 tx=0x7fac38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac38010840 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.307+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac38010e70 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.308+0000 7fac43f6d700 1 -- 172.21.15.63:0/2302185261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c113f80 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.308+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac38024ea0 con 0x7fac3c078c50 2024-04-24T22:36:21.172 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.308+0000 7fac43f6d700 1 -- 172.21.15.63:0/2302185261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c114af0 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.308+0000 7fac43f6d700 1 -- 172.21.15.63:0/2302185261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c05f5c0 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.309+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fac38024780 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.309+0000 7fac327fc700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 0x7fac28038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.309+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fac3804aa50 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.309+0000 7fac41508700 1 -- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 msgr2=0x7fac28038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.309+0000 7fac41508700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 0x7fac28038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.313+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fac38018b80 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.510+0000 7fac41508700 1 -- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 msgr2=0x7fac28038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.510+0000 7fac41508700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 0x7fac28038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.577+0000 7fac43f6d700 1 -- 172.21.15.63:0/2302185261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fac3c114e30 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.584+0000 7fac327fc700 1 -- 172.21.15.63:0/2302185261 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fac38018770 con 0x7fac3c078c50 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.587+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 msgr2=0x7fac28038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.587+0000 7fac1ffff700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 0x7fac28038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.587+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 msgr2=0x7fac3c113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.587+0000 7fac1ffff700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac38006d40 tx=0x7fac38009250 comp rx=0 tx=0).stop 2024-04-24T22:36:21.173 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 shutdown_connections 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7fac28036850 0x7fac28038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 --2- 172.21.15.63:0/2302185261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3c078c50 0x7fac3c113840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 >> 172.21.15.63:0/2302185261 conn(0x7fac3c0fe090 msgr2=0x7fac3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 shutdown_connections 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:20.588+0000 7fac1ffff700 1 -- 172.21.15.63:0/2302185261 wait complete. 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-04-24T22:36:21.174 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 9... 2024-04-24T22:36:21.415 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:21 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2302185261' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T22:36:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:36:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:36:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T22:36:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: mgrmap e10: smithi063.upcnvd(active, starting, since 0.0040072s) 2024-04-24T22:36:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:36:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:26 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:36:27.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-04-24T22:36:27.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T22:36:27.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T22:36:27.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-04-24T22:36:27.749 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.612+0000 7f4bd50cd700 1 Processor -- start 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.612+0000 7f4bd50cd700 1 -- start start 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.612+0000 7f4bd50cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.612+0000 7f4bd50cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0102080 con 0x7f4bd0101730 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.612+0000 7f4bced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40252/0 (socket says 172.21.15.63:40252) 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bced9d700 1 -- 172.21.15.63:0/326148539 learned_addr learned my addr 172.21.15.63:0/326148539 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bced9d700 1 -- 172.21.15.63:0/326148539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd01021c0 con 0x7f4bd0101730 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bced9d700 1 --2- 172.21.15.63:0/326148539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00098e0 tx=0x7f4bc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7e7edc4d7dfcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bcdd9b700 1 -- 172.21.15.63:0/326148539 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc000b860 con 0x7f4bd0101730 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bcdd9b700 1 -- 172.21.15.63:0/326148539 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bc000be90 con 0x7f4bd0101730 2024-04-24T22:36:27.750 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.613+0000 7f4bcdd9b700 1 -- 172.21.15.63:0/326148539 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc00103c0 con 0x7f4bd0101730 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 -- 172.21.15.63:0/326148539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 msgr2=0x7f4bd0101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/326148539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00098e0 tx=0x7f4bc0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 -- 172.21.15.63:0/326148539 shutdown_connections 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/326148539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0101b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 -- 172.21.15.63:0/326148539 >> 172.21.15.63:0/326148539 conn(0x7f4bd00fced0 msgr2=0x7f4bd00ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 -- 172.21.15.63:0/326148539 shutdown_connections 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.614+0000 7f4bd50cd700 1 -- 172.21.15.63:0/326148539 wait complete. 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bd50cd700 1 Processor -- start 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bd50cd700 1 -- start start 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bd50cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:27.751 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bd50cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0112100 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40262/0 (socket says 172.21.15.63:40262) 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bced9d700 1 -- 172.21.15.63:0/2301383437 learned_addr learned my addr 172.21.15.63:0/2301383437 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.615+0000 7f4bced9d700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00094f0 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bced9d700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00090b0 tx=0x7f4bc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc0010840 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd0116710 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bc0010e70 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0116ae0 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.616+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc0024e40 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f4bc001a4c0 con 0x7f4bd0101730 2024-04-24T22:36:27.752 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bb77fe700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bb8039460 con 0x7f4bb80368a0 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4bc004a820 con 0x7f4bd0101730 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bce59c700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.617+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.818+0000 7f4bce59c700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:21.818+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:22.219+0000 7f4bce59c700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:22.219+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:23.020+0000 7f4bce59c700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:23.020+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:36:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:24.621+0000 7f4bce59c700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2691264506 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:24.621+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:26.080+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f4bc0024430 con 0x7f4bd0101730 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:26.081+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 msgr2=0x7f4bb8038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:26.081+0000 7f4bb77fe700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.081+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f4bc0024710 con 0x7f4bd0101730 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.082+0000 7f4bb77fe700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 0x7f4bb803c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.082+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bb8039460 con 0x7f4bb803a1d0 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.082+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 0x7f4bb803c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.083+0000 7f4bce59c700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 0x7f4bb803c5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40039d0 tx=0x7f4bc4009350 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.083+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f4bb8039460 con 0x7f4bb803a1d0 2024-04-24T22:36:27.754 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.089+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4bd004bed0 con 0x7f4bb803a1d0 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.089+0000 7f4bb77fe700 1 -- 172.21.15.63:0/2301383437 <== mgr.14162 v2:172.21.15.63:6800/201758584 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4bd004bed0 con 0x7f4bb803a1d0 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 msgr2=0x7f4bb803c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 0x7f4bb803c5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40039d0 tx=0x7f4bc4009350 comp rx=0 tx=0).stop 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 msgr2=0x7f4bd0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00090b0 tx=0x7f4bc0009250 comp rx=0 tx=0).stop 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 shutdown_connections 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4bb803a1d0 0x7f4bb803c5b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bd0101730 0x7f4bd0076b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 --2- 172.21.15.63:0/2301383437 >> [v2:172.21.15.63:6800/2691264506,v1:172.21.15.63:6801/2691264506] conn(0x7f4bb80368a0 0x7f4bb8038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 >> 172.21.15.63:0/2301383437 conn(0x7f4bd00fced0 msgr2=0x7f4bd00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 shutdown_connections 2024-04-24T22:36:27.755 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:27.090+0000 7f4bd50cd700 1 -- 172.21.15.63:0/2301383437 wait complete. 2024-04-24T22:36:27.756 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 9 is available 2024-04-24T22:36:27.756 INFO:teuthology.orchestra.run.smithi063.stdout:Generating a dashboard self-signed certificate... 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: [24/Apr/2024:22:36:26] ENGINE Bus STARTING 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: [24/Apr/2024:22:36:26] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: [24/Apr/2024:22:36:26] ENGINE Bus STARTED 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: mgrmap e11: smithi063.upcnvd(active, since 1.00541s) 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T22:36:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:27 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2703d0d700 1 Processor -- start 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2703d0d700 1 -- start start 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2703d0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2703d0d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc1031d0 con 0x7f26fc102880 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2701aa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2701aa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40338/0 (socket says 172.21.15.63:40338) 2024-04-24T22:36:29.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.170+0000 7f2701aa9700 1 -- 172.21.15.63:0/3283208953 learned_addr learned my addr 172.21.15.63:0/3283208953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2701aa9700 1 -- 172.21.15.63:0/3283208953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc103310 con 0x7f26fc102880 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2701aa9700 1 --2- 172.21.15.63:0/3283208953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ea7bffa73c94a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2700aa7700 1 -- 172.21.15.63:0/3283208953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec00b860 con 0x7f26fc102880 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2700aa7700 1 -- 172.21.15.63:0/3283208953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26ec00be90 con 0x7f26fc102880 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2700aa7700 1 -- 172.21.15.63:0/3283208953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec0103c0 con 0x7f26fc102880 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 -- 172.21.15.63:0/3283208953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 msgr2=0x7f26fc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 --2- 172.21.15.63:0/3283208953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec009250 comp rx=0 tx=0).stop 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 -- 172.21.15.63:0/3283208953 shutdown_connections 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 --2- 172.21.15.63:0/3283208953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc102c90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 -- 172.21.15.63:0/3283208953 >> 172.21.15.63:0/3283208953 conn(0x7f26fc0fe060 msgr2=0x7f26fc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.171+0000 7f2703d0d700 1 -- 172.21.15.63:0/3283208953 shutdown_connections 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2703d0d700 1 -- 172.21.15.63:0/3283208953 wait complete. 2024-04-24T22:36:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2703d0d700 1 Processor -- start 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2703d0d700 1 -- start start 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2703d0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2703d0d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc111b60 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2701aa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2701aa9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40346/0 (socket says 172.21.15.63:40346) 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.172+0000 7f2701aa9700 1 -- 172.21.15.63:0/1022085081 learned_addr learned my addr 172.21.15.63:0/1022085081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f2701aa9700 1 -- 172.21.15.63:0/1022085081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0094f0 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f2701aa9700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26ec011650 tx=0x7f26ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec010890 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26ec010ec0 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc111d60 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec0193c0 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.173+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc1128d0 con 0x7f26fc102880 2024-04-24T22:36:29.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.174+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f26ec01a7a0 con 0x7f26fc102880 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.174+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc05f5c0 con 0x7f26fc102880 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.174+0000 7f26fa7fc700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 0x7f26e4038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.174+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f26ec04a2a0 con 0x7f26fc102880 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.175+0000 7f27012a8700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 0x7f26e4038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.175+0000 7f27012a8700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 0x7f26e4038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009990 tx=0x7f26f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.179+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f26ec006d50 con 0x7f26fc102880 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.383+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f26fc0ff250 con 0x7f26e4036770 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.548+0000 7f26fa7fc700 1 -- 172.21.15.63:0/1022085081 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f26fc0ff250 con 0x7f26e4036770 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 msgr2=0x7f26e4038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 0x7f26e4038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009990 tx=0x7f26f0006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 msgr2=0x7f26fc111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26ec011650 tx=0x7f26ec009250 comp rx=0 tx=0).stop 2024-04-24T22:36:29.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 shutdown_connections 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f26e4036770 0x7f26e4038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 --2- 172.21.15.63:0/1022085081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26fc102880 0x7f26fc111620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 >> 172.21.15.63:0/1022085081 conn(0x7f26fc0fe060 msgr2=0x7f26fc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 shutdown_connections 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:28.550+0000 7f2703d0d700 1 -- 172.21.15.63:0/1022085081 wait complete. 2024-04-24T22:36:29.141 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial admin user... 2024-04-24T22:36:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:29 smithi063 ceph-mon[26912]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:29 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:29 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:29 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:29 smithi063 ceph-mon[26912]: mgrmap e12: smithi063.upcnvd(active, since 2s) 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ozOesORT96ReoK86P65tXeiD2c2PNKjLFvEi6cWXmoweg0TRD/j5a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713998190, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.585+0000 7f973454c700 1 Processor -- start 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.585+0000 7f973454c700 1 -- start start 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f973454c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f973454c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c102090 con 0x7f972c101740 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97322e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97322e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43400/0 (socket says 172.21.15.63:43400) 2024-04-24T22:36:30.731 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97322e8700 1 -- 172.21.15.63:0/15912423 learned_addr learned my addr 172.21.15.63:0/15912423 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97322e8700 1 -- 172.21.15.63:0/15912423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c1021d0 con 0x7f972c101740 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97322e8700 1 --2- 172.21.15.63:0/15912423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97280098e0 tx=0x7f9728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f221155ceb828e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97312e6700 1 -- 172.21.15.63:0/15912423 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f972800b860 con 0x7f972c101740 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97312e6700 1 -- 172.21.15.63:0/15912423 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f972800be90 con 0x7f972c101740 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.586+0000 7f97312e6700 1 -- 172.21.15.63:0/15912423 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97280103c0 con 0x7f972c101740 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.587+0000 7f973454c700 1 -- 172.21.15.63:0/15912423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 msgr2=0x7f972c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.587+0000 7f973454c700 1 --2- 172.21.15.63:0/15912423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97280098e0 tx=0x7f9728009250 comp rx=0 tx=0).stop 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.587+0000 7f973454c700 1 -- 172.21.15.63:0/15912423 shutdown_connections 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.587+0000 7f973454c700 1 --2- 172.21.15.63:0/15912423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c101b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.587+0000 7f973454c700 1 -- 172.21.15.63:0/15912423 >> 172.21.15.63:0/15912423 conn(0x7f972c0fcec0 msgr2=0x7f972c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 -- 172.21.15.63:0/15912423 shutdown_connections 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 -- 172.21.15.63:0/15912423 wait complete. 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 Processor -- start 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 -- start start 2024-04-24T22:36:30.732 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.588+0000 7f973454c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c1109f0 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f97322e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f97322e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43412/0 (socket says 172.21.15.63:43412) 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f97322e8700 1 -- 172.21.15.63:0/2541552362 learned_addr learned my addr 172.21.15.63:0/2541552362 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f97322e8700 1 -- 172.21.15.63:0/2541552362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280094f0 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f97322e8700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97280090b0 tx=0x7f9728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9728010840 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9728010e70 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9728024e40 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c110bf0 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.589+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c111760 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.590+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f972801a510 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.590+0000 7f9722ffd700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 0x7f9718038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.591+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f972804aaf0 con 0x7f972c101740 2024-04-24T22:36:30.733 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.591+0000 7f9731ae7700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 0x7f9718038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.591+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c05f5c0 con 0x7f972c101740 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.591+0000 7f9731ae7700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 0x7f9718038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f971c009990 tx=0x7f971c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.596+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9728018710 con 0x7f972c101740 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:29.817+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f972c05f3b0 con 0x7f9718036850 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.063+0000 7f9722ffd700 1 -- 172.21.15.63:0/2541552362 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f972c05f3b0 con 0x7f9718036850 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.065+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 msgr2=0x7f9718038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.065+0000 7f973454c700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 0x7f9718038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f971c009990 tx=0x7f971c006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.065+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 msgr2=0x7f972c1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.065+0000 7f973454c700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97280090b0 tx=0x7f9728009250 comp rx=0 tx=0).stop 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 shutdown_connections 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f9718036850 0x7f9718038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 --2- 172.21.15.63:0/2541552362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f972c101740 0x7f972c1104b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:30.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 >> 172.21.15.63:0/2541552362 conn(0x7f972c0fcec0 msgr2=0x7f972c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:30.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 shutdown_connections 2024-04-24T22:36:30.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:30.066+0000 7f973454c700 1 -- 172.21.15.63:0/2541552362 wait complete. 2024-04-24T22:36:30.735 INFO:teuthology.orchestra.run.smithi063.stdout:Fetching dashboard port number... 2024-04-24T22:36:31.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:31 smithi063 ceph-mon[26912]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:31.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:31 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.163+0000 7f55ff584700 1 Processor -- start 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.163+0000 7f55ff584700 1 -- start start 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55ff584700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55ff584700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8103200 con 0x7f55f81028b0 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55fd320700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55fd320700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43424/0 (socket says 172.21.15.63:43424) 2024-04-24T22:36:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55fd320700 1 -- 172.21.15.63:0/1578729668 learned_addr learned my addr 172.21.15.63:0/1578729668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55fd320700 1 -- 172.21.15.63:0/1578729668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8103340 con 0x7f55f81028b0 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.164+0000 7f55fd320700 1 --2- 172.21.15.63:0/1578729668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098e0 tx=0x7f55f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae43afc302abd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55effff700 1 -- 172.21.15.63:0/1578729668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f400b860 con 0x7f55f81028b0 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55effff700 1 -- 172.21.15.63:0/1578729668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f400be90 con 0x7f55f81028b0 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55effff700 1 -- 172.21.15.63:0/1578729668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f40103c0 con 0x7f55f81028b0 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55ff584700 1 -- 172.21.15.63:0/1578729668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 msgr2=0x7f55f8102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55ff584700 1 --2- 172.21.15.63:0/1578729668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098e0 tx=0x7f55f4009250 comp rx=0 tx=0).stop 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55ff584700 1 -- 172.21.15.63:0/1578729668 shutdown_connections 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55ff584700 1 --2- 172.21.15.63:0/1578729668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f8102cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.165+0000 7f55ff584700 1 -- 172.21.15.63:0/1578729668 >> 172.21.15.63:0/1578729668 conn(0x7f55f80fe030 msgr2=0x7f55f8100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 -- 172.21.15.63:0/1578729668 shutdown_connections 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 -- 172.21.15.63:0/1578729668 wait complete. 2024-04-24T22:36:32.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 Processor -- start 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 -- start start 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.166+0000 7f55ff584700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8111b10 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55fd320700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55fd320700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43432/0 (socket says 172.21.15.63:43432) 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55fd320700 1 -- 172.21.15.63:0/1486901611 learned_addr learned my addr 172.21.15.63:0/1486901611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55fd320700 1 -- 172.21.15.63:0/1486901611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40094f0 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55fd320700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011680 tx=0x7f55f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f4010890 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f4010ec0 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f8111d10 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f40243f0 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.167+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f8112880 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.168+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f55f402d3f0 con 0x7f55f81028b0 2024-04-24T22:36:32.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.168+0000 7f55edffb700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 0x7f55e4038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.168+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f805f5c0 con 0x7f55f81028b0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.169+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f55f404a380 con 0x7f55f81028b0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.169+0000 7f55fcb1f700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 0x7f55e4038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.169+0000 7f55fcb1f700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 0x7f55e4038da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009990 tx=0x7f55e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.174+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f55f402b8e0 con 0x7f55f81028b0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.373+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f55f804bed0 con 0x7f55f81028b0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.373+0000 7f55edffb700 1 -- 172.21.15.63:0/1486901611 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f55f4006d70 con 0x7f55f81028b0 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 msgr2=0x7f55e4038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 0x7f55e4038da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009990 tx=0x7f55e8006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 msgr2=0x7f55f81115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011680 tx=0x7f55f4009250 comp rx=0 tx=0).stop 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 shutdown_connections 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f55e40368f0 0x7f55e4038da0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:32.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 --2- 172.21.15.63:0/1486901611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f81028b0 0x7f55f81115d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:32.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 >> 172.21.15.63:0/1486901611 conn(0x7f55f80fe030 msgr2=0x7f55f80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:32.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 shutdown_connections 2024-04-24T22:36:32.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:31.376+0000 7f55ff584700 1 -- 172.21.15.63:0/1486901611 wait complete. 2024-04-24T22:36:32.140 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T22:36:32.140 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-04-24T22:36:32.149 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T22:36:32.149 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-04-24T22:36:32.149 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-04-24T22:36:32.149 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T22:36:32.150 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph Dashboard is now available at: 2024-04-24T22:36:32.151 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:32.151 INFO:teuthology.orchestra.run.smithi063.stdout: URL: https://smithi063:8443/ 2024-04-24T22:36:32.151 INFO:teuthology.orchestra.run.smithi063.stdout: User: admin 2024-04-24T22:36:32.151 INFO:teuthology.orchestra.run.smithi063.stdout: Password: v19yqgbt68 2024-04-24T22:36:32.151 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:32.152 INFO:teuthology.orchestra.run.smithi063.stdout:Saving cluster configuration to /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config directory 2024-04-24T22:36:32.152 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling autotune for osd_memory_target 2024-04-24T22:36:32.250 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:32 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1486901611' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T22:36:33.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.554+0000 7f2e1c8ac700 1 Processor -- start 2024-04-24T22:36:33.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.554+0000 7f2e1c8ac700 1 -- start start 2024-04-24T22:36:33.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1c8ac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:33.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1c8ac700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181031d0 con 0x7f2e18102880 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43440/0 (socket says 172.21.15.63:43440) 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1659c700 1 -- 172.21.15.63:0/1509023270 learned_addr learned my addr 172.21.15.63:0/1509023270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1659c700 1 -- 172.21.15.63:0/1509023270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18103310 con 0x7f2e18102880 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1659c700 1 --2- 172.21.15.63:0/1509023270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e080098e0 tx=0x7f2e08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becb31b8213099bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1559a700 1 -- 172.21.15.63:0/1509023270 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e0800b860 con 0x7f2e18102880 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.555+0000 7f2e1559a700 1 -- 172.21.15.63:0/1509023270 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e0800be90 con 0x7f2e18102880 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1559a700 1 -- 172.21.15.63:0/1509023270 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e080103c0 con 0x7f2e18102880 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1509023270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 msgr2=0x7f2e18102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1509023270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e080098e0 tx=0x7f2e08009250 comp rx=0 tx=0).stop 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1509023270 shutdown_connections 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1509023270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18102c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1509023270 >> 172.21.15.63:0/1509023270 conn(0x7f2e180fe060 msgr2=0x7f2e18100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1509023270 shutdown_connections 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.556+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1509023270 wait complete. 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1c8ac700 1 Processor -- start 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1c8ac700 1 -- start start 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1c8ac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1c8ac700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18111b60 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43444/0 (socket says 172.21.15.63:43444) 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1659c700 1 -- 172.21.15.63:0/1801557030 learned_addr learned my addr 172.21.15.63:0/1801557030 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1659c700 1 -- 172.21.15.63:0/1801557030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e080094f0 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e1659c700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011680 tx=0x7f2e08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.557+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e08010890 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e18111d60 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e08010ec0 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e080243f0 con 0x7f2e18102880 2024-04-24T22:36:33.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181128d0 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f2e0802d3f0 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.558+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1805f5c0 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.559+0000 7f2e06ffd700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 0x7f2e00038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.559+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2e0804a500 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.559+0000 7f2e15d9b700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 0x7f2e00038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.559+0000 7f2e15d9b700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 0x7f2e00038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009990 tx=0x7f2e0c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.563+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2e08018d20 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.757+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2e1804bed0 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.757+0000 7f2e06ffd700 1 -- 172.21.15.63:0/1801557030 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2e08006d70 con 0x7f2e18102880 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 msgr2=0x7f2e00038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 0x7f2e00038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009990 tx=0x7f2e0c006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 msgr2=0x7f2e18111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011680 tx=0x7f2e08009250 comp rx=0 tx=0).stop 2024-04-24T22:36:33.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 shutdown_connections 2024-04-24T22:36:33.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f2e000368a0 0x7f2e00038d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:33.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 --2- 172.21.15.63:0/1801557030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e18102880 0x7f2e18111620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:33.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 >> 172.21.15.63:0/1801557030 conn(0x7f2e180fe060 msgr2=0x7f2e180feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:33.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 shutdown_connections 2024-04-24T22:36:33.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:32.759+0000 7f2e1c8ac700 1 -- 172.21.15.63:0/1801557030 wait complete. 2024-04-24T22:36:34.428 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:34 smithi063 ceph-mon[26912]: mgrmap e13: smithi063.upcnvd(active, since 6s) 2024-04-24T22:36:34.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbf2456700 1 Processor -- start 2024-04-24T22:36:34.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbf2456700 1 -- start start 2024-04-24T22:36:34.704 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbf2456700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbf2456700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec102080 con 0x7fcbec101730 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43454/0 (socket says 172.21.15.63:43454) 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbebfff700 1 -- 172.21.15.63:0/4045005770 learned_addr learned my addr 172.21.15.63:0/4045005770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.800+0000 7fcbebfff700 1 -- 172.21.15.63:0/4045005770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec1021c0 con 0x7fcbec101730 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbebfff700 1 --2- 172.21.15.63:0/4045005770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0098e0 tx=0x7fcbdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173f4ebb9d970e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbeaffd700 1 -- 172.21.15.63:0/4045005770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbdc00b860 con 0x7fcbec101730 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbeaffd700 1 -- 172.21.15.63:0/4045005770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbdc00be90 con 0x7fcbec101730 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbeaffd700 1 -- 172.21.15.63:0/4045005770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbdc0103c0 con 0x7fcbec101730 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbf2456700 1 -- 172.21.15.63:0/4045005770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 msgr2=0x7fcbec101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.801+0000 7fcbf2456700 1 --2- 172.21.15.63:0/4045005770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0098e0 tx=0x7fcbdc009250 comp rx=0 tx=0).stop 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 -- 172.21.15.63:0/4045005770 shutdown_connections 2024-04-24T22:36:34.705 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 --2- 172.21.15.63:0/4045005770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec101b40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 -- 172.21.15.63:0/4045005770 >> 172.21.15.63:0/4045005770 conn(0x7fcbec0fcef0 msgr2=0x7fcbec0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 -- 172.21.15.63:0/4045005770 shutdown_connections 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 -- 172.21.15.63:0/4045005770 wait complete. 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 Processor -- start 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.802+0000 7fcbf2456700 1 -- start start 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbf2456700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbf2456700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec110a70 con 0x7fcbec101730 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43458/0 (socket says 172.21.15.63:43458) 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbebfff700 1 -- 172.21.15.63:0/68576276 learned_addr learned my addr 172.21.15.63:0/68576276 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbebfff700 1 -- 172.21.15.63:0/68576276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0094f0 con 0x7fcbec101730 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbebfff700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0090b0 tx=0x7fcbdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbdc010840 con 0x7fcbec101730 2024-04-24T22:36:34.706 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbdc010e70 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec110c70 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbdc024e90 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.803+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec1117e0 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.804+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fcbdc01a510 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.804+0000 7fcbe8ff9700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 0x7fcbd4038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.804+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec05f5c0 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.804+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcbdc04a9e0 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.805+0000 7fcbeb7fe700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 0x7fcbd4038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.805+0000 7fcbeb7fe700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 0x7fcbd4038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009990 tx=0x7fcbe0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:33.809+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcbdc018940 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.116+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fcbec103520 con 0x7fcbec101730 2024-04-24T22:36:34.707 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.121+0000 7fcbe8ff9700 1 -- 172.21.15.63:0/68576276 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fcbdc018350 con 0x7fcbec101730 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 msgr2=0x7fcbd4038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 0x7fcbd4038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009990 tx=0x7fcbe0006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 msgr2=0x7fcbec110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0090b0 tx=0x7fcbdc009250 comp rx=0 tx=0).stop 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 shutdown_connections 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fcbd40368f0 0x7fcbd4038da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 --2- 172.21.15.63:0/68576276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbec101730 0x7fcbec110530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 >> 172.21.15.63:0/68576276 conn(0x7fcbec0fcef0 msgr2=0x7fcbec0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 shutdown_connections 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-04-24T22:36:34.123+0000 7fcbf2456700 1 -- 172.21.15.63:0/68576276 wait complete. 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.708 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: ceph telemetry on 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout:For more information see: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:34.709 INFO:teuthology.orchestra.run.smithi063.stdout:Bootstrap complete. 2024-04-24T22:36:34.741 INFO:tasks.cephadm:Fetching config... 2024-04-24T22:36:34.741 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:36:34.741 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T22:36:34.797 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T22:36:34.797 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:36:34.797 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T22:36:34.853 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T22:36:34.853 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:36:34.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/keyring of=/dev/stdout 2024-04-24T22:36:34.918 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T22:36:34.918 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:36:34.918 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T22:36:34.973 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T22:36:34.974 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLwh9MGMTi/2yiWovGCgAro3bTNvzNF5lKqQ7elb/BB5DLXAE27TuHUQA/VkhG/mirh9RA7oPkmU4m8Y1OsLL6VVAXFYMscflJ5knCH7kvjyekNRSX1WPXJZjmgw66ZlFcElFtz2Pr5NzllkUy0k96fhGwTgSG1CryLTrS2u+ikseJaV4M0DplhoXw6Wzrb1+AJkt3mBCwX/oH+eyFOy35L3N6l25SxGd/dKScfbqGar9J2FH6Vvlwfzqu/+mpaCmZwJBAGSbSNFGbmeNulbMeCPPXmlfGgYkIiFdqqB1w9Tc+B+jYWNi5klLp4GtLf2i7KFIUH5Ol9DB0rEVSK3Q6mSm0qdRfKp6vG08IrbBPj2BvTr888oEV1BxYKH/KhKmoLre37BFbcL29XtoiemgFvXdTpXFOnirYriDox2dwfTZqqP3PPLyHKW+nq1QXi5KWq/iYnAD028X7nyknuruQGBxQybu1r+cVi/Lz8t09llQF1XGw1IZk/fsfPLj7z5U= ceph-c8a2676e-028a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T22:36:35.046 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLwh9MGMTi/2yiWovGCgAro3bTNvzNF5lKqQ7elb/BB5DLXAE27TuHUQA/VkhG/mirh9RA7oPkmU4m8Y1OsLL6VVAXFYMscflJ5knCH7kvjyekNRSX1WPXJZjmgw66ZlFcElFtz2Pr5NzllkUy0k96fhGwTgSG1CryLTrS2u+ikseJaV4M0DplhoXw6Wzrb1+AJkt3mBCwX/oH+eyFOy35L3N6l25SxGd/dKScfbqGar9J2FH6Vvlwfzqu/+mpaCmZwJBAGSbSNFGbmeNulbMeCPPXmlfGgYkIiFdqqB1w9Tc+B+jYWNi5klLp4GtLf2i7KFIUH5Ol9DB0rEVSK3Q6mSm0qdRfKp6vG08IrbBPj2BvTr888oEV1BxYKH/KhKmoLre37BFbcL29XtoiemgFvXdTpXFOnirYriDox2dwfTZqqP3PPLyHKW+nq1QXi5KWq/iYnAD028X7nyknuruQGBxQybu1r+cVi/Lz8t09llQF1XGw1IZk/fsfPLj7z5U= ceph-c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:36:35.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLwh9MGMTi/2yiWovGCgAro3bTNvzNF5lKqQ7elb/BB5DLXAE27TuHUQA/VkhG/mirh9RA7oPkmU4m8Y1OsLL6VVAXFYMscflJ5knCH7kvjyekNRSX1WPXJZjmgw66ZlFcElFtz2Pr5NzllkUy0k96fhGwTgSG1CryLTrS2u+ikseJaV4M0DplhoXw6Wzrb1+AJkt3mBCwX/oH+eyFOy35L3N6l25SxGd/dKScfbqGar9J2FH6Vvlwfzqu/+mpaCmZwJBAGSbSNFGbmeNulbMeCPPXmlfGgYkIiFdqqB1w9Tc+B+jYWNi5klLp4GtLf2i7KFIUH5Ol9DB0rEVSK3Q6mSm0qdRfKp6vG08IrbBPj2BvTr888oEV1BxYKH/KhKmoLre37BFbcL29XtoiemgFvXdTpXFOnirYriDox2dwfTZqqP3PPLyHKW+nq1QXi5KWq/iYnAD028X7nyknuruQGBxQybu1r+cVi/Lz8t09llQF1XGw1IZk/fsfPLj7z5U= ceph-c8a2676e-028a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T22:36:35.095 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLwh9MGMTi/2yiWovGCgAro3bTNvzNF5lKqQ7elb/BB5DLXAE27TuHUQA/VkhG/mirh9RA7oPkmU4m8Y1OsLL6VVAXFYMscflJ5knCH7kvjyekNRSX1WPXJZjmgw66ZlFcElFtz2Pr5NzllkUy0k96fhGwTgSG1CryLTrS2u+ikseJaV4M0DplhoXw6Wzrb1+AJkt3mBCwX/oH+eyFOy35L3N6l25SxGd/dKScfbqGar9J2FH6Vvlwfzqu/+mpaCmZwJBAGSbSNFGbmeNulbMeCPPXmlfGgYkIiFdqqB1w9Tc+B+jYWNi5klLp4GtLf2i7KFIUH5Ol9DB0rEVSK3Q6mSm0qdRfKp6vG08IrbBPj2BvTr888oEV1BxYKH/KhKmoLre37BFbcL29XtoiemgFvXdTpXFOnirYriDox2dwfTZqqP3PPLyHKW+nq1QXi5KWq/iYnAD028X7nyknuruQGBxQybu1r+cVi/Lz8t09llQF1XGw1IZk/fsfPLj7z5U= ceph-c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:36:35.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T22:36:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:35 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/68576276' entity='client.admin' 2024-04-24T22:36:35.368 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:36:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:36:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:36:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T22:36:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:36:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:37 smithi063 ceph-mon[26912]: Deploying daemon crash.smithi063 on smithi063 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 -- 172.21.15.63:0/3844806598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 msgr2=0x7f8a740ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 --2- 172.21.15.63:0/3844806598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a740ff630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009c60 tx=0x7f8a6c004d10 comp rx=0 tx=0).stop 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 -- 172.21.15.63:0/3844806598 shutdown_connections 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 --2- 172.21.15.63:0/3844806598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a740ff630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 -- 172.21.15.63:0/3844806598 >> 172.21.15.63:0/3844806598 conn(0x7f8a740f70a0 msgr2=0x7f8a740f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 -- 172.21.15.63:0/3844806598 shutdown_connections 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.130+0000 7f8a7b537700 1 -- 172.21.15.63:0/3844806598 wait complete. 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a7b537700 1 Processor -- start 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a7b537700 1 -- start start 2024-04-24T22:36:38.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a7b537700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a7b537700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7410d5d0 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a792d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a792d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43476/0 (socket says 172.21.15.63:43476) 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.131+0000 7f8a792d3700 1 -- 172.21.15.63:0/4134549761 learned_addr learned my addr 172.21.15.63:0/4134549761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a792d3700 1 -- 172.21.15.63:0/4134549761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009940 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a792d3700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0097b0 tx=0x7f8a6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a6c012410 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a6c012a40 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7410d7d0 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a6c01b980 con 0x7f8a740665c0 2024-04-24T22:36:38.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.132+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7410e340 con 0x7f8a740665c0 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.133+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f8a6c019040 con 0x7f8a740665c0 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.133+0000 7f8a71ffb700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 0x7f8a5c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.133+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8a6c04ac60 con 0x7f8a740665c0 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.133+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a74102f40 con 0x7f8a740665c0 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.133+0000 7f8a78ad2700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 0x7f8a5c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:38.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.134+0000 7f8a78ad2700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 0x7f8a5c038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009990 tx=0x7f8a68006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.138+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a6c017080 con 0x7f8a740665c0 2024-04-24T22:36:38.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.334+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8a7405f5c0 con 0x7f8a740665c0 2024-04-24T22:36:38.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.337+0000 7f8a71ffb700 1 -- 172.21.15.63:0/4134549761 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8a6c01f020 con 0x7f8a740665c0 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 msgr2=0x7f8a5c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 0x7f8a5c038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009990 tx=0x7f8a68006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 msgr2=0x7f8a7410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0097b0 tx=0x7f8a6c004d10 comp rx=0 tx=0).stop 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 shutdown_connections 2024-04-24T22:36:38.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f8a5c0368d0 0x7f8a5c038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 --2- 172.21.15.63:0/4134549761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a740665c0 0x7f8a7410d090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:38.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.339+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 >> 172.21.15.63:0/4134549761 conn(0x7f8a740f70a0 msgr2=0x7f8a74103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:38.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.340+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 shutdown_connections 2024-04-24T22:36:38.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:38.340+0000 7f8a7b537700 1 -- 172.21.15.63:0/4134549761 wait complete. 2024-04-24T22:36:39.450 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T22:36:39.451 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T22:36:39.639 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:39 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4134549761' entity='client.admin' 2024-04-24T22:36:40.196 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- 172.21.15.63:0/3534761862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 msgr2=0x7f18dc0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3534761862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc0fbd70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009c60 tx=0x7f18cc004d10 comp rx=0 tx=0).stop 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- 172.21.15.63:0/3534761862 shutdown_connections 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3534761862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc0fbd70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- 172.21.15.63:0/3534761862 >> 172.21.15.63:0/3534761862 conn(0x7f18dc0f70c0 msgr2=0x7f18dc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- 172.21.15.63:0/3534761862 shutdown_connections 2024-04-24T22:36:42.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- 172.21.15.63:0/3534761862 wait complete. 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 Processor -- start 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.251+0000 7f18e11c9700 1 -- start start 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18e11c9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18e11c9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc10d5c0 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18dad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18dad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52550/0 (socket says 172.21.15.63:52550) 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18dad9d700 1 -- 172.21.15.63:0/3429111108 learned_addr learned my addr 172.21.15.63:0/3429111108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18dad9d700 1 -- 172.21.15.63:0/3429111108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009940 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18dad9d700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009500 tx=0x7f18cc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18cc007c80 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18cc01d910 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc10d7c0 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18cc0129d0 con 0x7f18dc0fb960 2024-04-24T22:36:42.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.252+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc10e330 con 0x7f18dc0fb960 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.253+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f18cc019040 con 0x7f18dc0fb960 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.253+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc101e70 con 0x7f18dc0fb960 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.253+0000 7f18d37fe700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 0x7f18c8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.253+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f18cc04a340 con 0x7f18dc0fb960 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.253+0000 7f18da59c700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 0x7f18c8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.254+0000 7f18da59c700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 0x7f18c8038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009990 tx=0x7f18c4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:42.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.258+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f18cc02eab0 con 0x7f18dc0fb960 2024-04-24T22:36:42.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.464+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f18dc0663b0 con 0x7f18c8036880 2024-04-24T22:36:42.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.468+0000 7f18d37fe700 1 -- 172.21.15.63:0/3429111108 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f18dc0663b0 con 0x7f18c8036880 2024-04-24T22:36:42.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.469+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 msgr2=0x7f18c8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:42.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.469+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 0x7f18c8038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009990 tx=0x7f18c4006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:42.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.469+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 msgr2=0x7f18dc10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:42.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.469+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009500 tx=0x7f18cc004d10 comp rx=0 tx=0).stop 2024-04-24T22:36:42.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 shutdown_connections 2024-04-24T22:36:42.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f18c8036880 0x7f18c8038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:42.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 --2- 172.21.15.63:0/3429111108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18dc0fb960 0x7f18dc10d080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:42.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 >> 172.21.15.63:0/3429111108 conn(0x7f18dc0f70c0 msgr2=0x7f18dc065ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:42.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 shutdown_connections 2024-04-24T22:36:42.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:42.470+0000 7f18e11c9700 1 -- 172.21.15.63:0/3429111108 wait complete. 2024-04-24T22:36:43.415 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-04-24T22:36:43.415 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:36:43.415 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-04-24T22:36:43.434 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:36:43.434 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:36:43.492 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-04-24T22:36:43.492 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch host add smithi192 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:43.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:43 smithi063 ceph-mon[26912]: Deploying daemon node-exporter.smithi063 on smithi063 2024-04-24T22:36:43.772 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 -- 172.21.15.63:0/106881936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 msgr2=0x7f45480ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 --2- 172.21.15.63:0/106881936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f45480ff6f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f453c0048c0 tx=0x7f453c009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 -- 172.21.15.63:0/106881936 shutdown_connections 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 --2- 172.21.15.63:0/106881936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f45480ff6f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 -- 172.21.15.63:0/106881936 >> 172.21.15.63:0/106881936 conn(0x7f45480f70a0 msgr2=0x7f45480f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 -- 172.21.15.63:0/106881936 shutdown_connections 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 -- 172.21.15.63:0/106881936 wait complete. 2024-04-24T22:36:45.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.325+0000 7f454ec0b700 1 Processor -- start 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454ec0b700 1 -- start start 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454ec0b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454ec0b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454810d5d0 con 0x7f4548065c70 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454c9a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454c9a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52596/0 (socket says 172.21.15.63:52596) 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454c9a7700 1 -- 172.21.15.63:0/1906237534 learned_addr learned my addr 172.21.15.63:0/1906237534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454c9a7700 1 -- 172.21.15.63:0/1906237534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c005040 con 0x7f4548065c70 2024-04-24T22:36:45.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.326+0000 7f454c9a7700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f453c009680 tx=0x7f453c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f453c00bed0 con 0x7f4548065c70 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f453c00b5c0 con 0x7f4548065c70 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454810d7d0 con 0x7f4548065c70 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f453c01c5b0 con 0x7f4548065c70 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454810e340 con 0x7f4548065c70 2024-04-24T22:36:45.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.327+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f453c01c710 con 0x7f4548065c70 2024-04-24T22:36:45.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.328+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548102f40 con 0x7f4548065c70 2024-04-24T22:36:45.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.328+0000 7f45457fa700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 0x7f4538038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:45.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.328+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f453c049990 con 0x7f4548065c70 2024-04-24T22:36:45.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.328+0000 7f4547fff700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 0x7f4538038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:45.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.328+0000 7f4547fff700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 0x7f4538038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4534009990 tx=0x7f4534006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:45.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.333+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f453c025400 con 0x7f4548065c70 2024-04-24T22:36:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:45.536+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f454810e580 con 0x7f4538036880 2024-04-24T22:36:47.406 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:47 smithi063 ceph-mon[26912]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:36:47.406 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:47 smithi063 ceph-mon[26912]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:47.406 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:47 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:47.407 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:47 smithi063 ceph-mon[26912]: Deploying cephadm binary to smithi192 2024-04-24T22:36:47.523 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-04-24T22:36:47.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.521+0000 7f45457fa700 1 -- 172.21.15.63:0/1906237534 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f454810e580 con 0x7f4538036880 2024-04-24T22:36:47.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.523+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 msgr2=0x7f4538038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:47.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 0x7f4538038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4534009990 tx=0x7f4534006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:47.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 msgr2=0x7f454810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f453c009680 tx=0x7f453c009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 shutdown_connections 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4538036880 0x7f4538038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 --2- 172.21.15.63:0/1906237534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4548065c70 0x7f454810d090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 >> 172.21.15.63:0/1906237534 conn(0x7f45480f70a0 msgr2=0x7f4548103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 shutdown_connections 2024-04-24T22:36:47.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:47.524+0000 7f454ec0b700 1 -- 172.21.15.63:0/1906237534 wait complete. 2024-04-24T22:36:48.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T22:36:48.361 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:36:48.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:48.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: Added host smithi192 2024-04-24T22:36:48.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:48.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:48.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:48.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:48.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:48.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:48 smithi063 ceph-mon[26912]: Deploying daemon alertmanager.smithi063 on smithi063 2024-04-24T22:36:49.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 -- 172.21.15.63:0/1177770465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 msgr2=0x7fd2700ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:49.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 --2- 172.21.15.63:0/1177770465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd2700ff630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0048c0 tx=0x7fd26c009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:49.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 -- 172.21.15.63:0/1177770465 shutdown_connections 2024-04-24T22:36:49.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 --2- 172.21.15.63:0/1177770465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd2700ff630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:49.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 -- 172.21.15.63:0/1177770465 >> 172.21.15.63:0/1177770465 conn(0x7fd2700f70a0 msgr2=0x7fd2700f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:49.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 -- 172.21.15.63:0/1177770465 shutdown_connections 2024-04-24T22:36:49.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.909+0000 7fd27775b700 1 -- 172.21.15.63:0/1177770465 wait complete. 2024-04-24T22:36:49.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd27775b700 1 Processor -- start 2024-04-24T22:36:49.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd27775b700 1 -- start start 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd27775b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd27775b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27010d5a0 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd2754f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd2754f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48500/0 (socket says 172.21.15.63:48500) 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.910+0000 7fd2754f7700 1 -- 172.21.15.63:0/358433190 learned_addr learned my addr 172.21.15.63:0/358433190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd2754f7700 1 -- 172.21.15.63:0/358433190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c005040 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd2754f7700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00b900 tx=0x7fd26c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd26c00b5c0 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26c00be70 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27010d7a0 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd26c01c590 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.911+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27010e310 con 0x7fd270065d30 2024-04-24T22:36:49.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.912+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fd26c004d30 con 0x7fd270065d30 2024-04-24T22:36:49.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.912+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd270102f40 con 0x7fd270065d30 2024-04-24T22:36:49.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.912+0000 7fd265ffb700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 0x7fd25c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:49.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.912+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd26c049380 con 0x7fd270065d30 2024-04-24T22:36:49.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.912+0000 7fd274cf6700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 0x7fd25c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:49.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.913+0000 7fd274cf6700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 0x7fd25c038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd260009990 tx=0x7fd260006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:49.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:49.917+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd26c01b350 con 0x7fd270065d30 2024-04-24T22:36:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.124+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd27010e550 con 0x7fd25c0368d0 2024-04-24T22:36:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.125+0000 7fd265ffb700 1 -- 172.21.15.63:0/358433190 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fd27010e550 con 0x7fd25c0368d0 2024-04-24T22:36:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:36:50.127 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 msgr2=0x7fd25c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 0x7fd25c038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd260009990 tx=0x7fd260006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 msgr2=0x7fd27010d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00b900 tx=0x7fd26c009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 shutdown_connections 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fd25c0368d0 0x7fd25c038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 --2- 172.21.15.63:0/358433190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270065d30 0x7fd27010d060 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 >> 172.21.15.63:0/358433190 conn(0x7fd2700f70a0 msgr2=0x7fd270103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:50.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.127+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 shutdown_connections 2024-04-24T22:36:50.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:50.128+0000 7fd27775b700 1 -- 172.21.15.63:0/358433190 wait complete. 2024-04-24T22:36:51.194 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T22:36:51.194 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T22:36:51.453 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:36:51.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:51 smithi063 ceph-mon[26912]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:51.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:51 smithi063 ceph-mon[26912]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:36:51.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:51 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:53.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:53 smithi063 ceph-mon[26912]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:54.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 -- 172.21.15.63:0/4198502226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 msgr2=0x7f5dc80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:54.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 --2- 172.21.15.63:0/4198502226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc80fce30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5db80048c0 tx=0x7f5db8009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:54.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 -- 172.21.15.63:0/4198502226 shutdown_connections 2024-04-24T22:36:54.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 --2- 172.21.15.63:0/4198502226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc80fce30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 -- 172.21.15.63:0/4198502226 >> 172.21.15.63:0/4198502226 conn(0x7f5dc80f81a0 msgr2=0x7f5dc80fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 -- 172.21.15.63:0/4198502226 shutdown_connections 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.613+0000 7f5dcd121700 1 -- 172.21.15.63:0/4198502226 wait complete. 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dcd121700 1 Processor -- start 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dcd121700 1 -- start start 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dcd121700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dcd121700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc810d5c0 con 0x7f5dc80fca20 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dc6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dc6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48506/0 (socket says 172.21.15.63:48506) 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.614+0000 7f5dc6d9d700 1 -- 172.21.15.63:0/1483024621 learned_addr learned my addr 172.21.15.63:0/1483024621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dc6d9d700 1 -- 172.21.15.63:0/1483024621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8005040 con 0x7f5dc80fca20 2024-04-24T22:36:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dc6d9d700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009680 tx=0x7f5db8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db800b5c0 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc810d7c0 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5db800be70 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db80135a0 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.615+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc810e330 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.616+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5db8013860 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.616+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc8102f30 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.616+0000 7f5dbf7fe700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 0x7f5db4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.616+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5db80493a0 con 0x7f5dc80fca20 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.616+0000 7f5dc659c700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 0x7f5db4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:54.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.617+0000 7f5dc659c700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 0x7f5db4038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009990 tx=0x7f5db0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:54.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.621+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5db8011960 con 0x7f5dc80fca20 2024-04-24T22:36:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:54.825+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5dc810ed40 con 0x7f5dc80fca20 2024-04-24T22:36:55.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.145+0000 7f5dbf7fe700 1 -- 172.21.15.63:0/1483024621 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5db8021020 con 0x7f5dc80fca20 2024-04-24T22:36:55.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 msgr2=0x7f5db4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 0x7f5db4038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009990 tx=0x7f5db0006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 msgr2=0x7f5dc810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009680 tx=0x7f5db8009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 shutdown_connections 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5db4036880 0x7f5db4038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 --2- 172.21.15.63:0/1483024621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc80fca20 0x7f5dc810d080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 >> 172.21.15.63:0/1483024621 conn(0x7f5dc80f81a0 msgr2=0x7f5dc8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 shutdown_connections 2024-04-24T22:36:55.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:36:55.148+0000 7f5dcd121700 1 -- 172.21.15.63:0/1483024621 wait complete. 2024-04-24T22:36:55.151 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-04-24T22:36:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:55 smithi063 ceph-mon[26912]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:55 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1483024621' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T22:36:56.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:56 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1483024621' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T22:36:56.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:56 smithi063 ceph-mon[26912]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:36:57.264 INFO:tasks.cephadm:Adding mon.smithi063 on smithi063 2024-04-24T22:36:57.264 INFO:tasks.cephadm:Adding mon.smithi192 on smithi192 2024-04-24T22:36:57.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192' 2024-04-24T22:36:57.513 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:36:57.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:57 smithi063 ceph-mon[26912]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:57.561 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 -- 172.21.15.192:0/415316381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 msgr2=0x7f57b40fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 --2- 172.21.15.192:0/415316381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b40fbd40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f57a80048c0 tx=0x7f57a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 -- 172.21.15.192:0/415316381 shutdown_connections 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 --2- 172.21.15.192:0/415316381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b40fbd40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 -- 172.21.15.192:0/415316381 >> 172.21.15.192:0/415316381 conn(0x7f57b40f7070 msgr2=0x7f57b40f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 -- 172.21.15.192:0/415316381 shutdown_connections 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.134+0000 7f57b919f700 1 -- 172.21.15.192:0/415316381 wait complete. 2024-04-24T22:36:59.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b919f700 1 Processor -- start 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b919f700 1 -- start start 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b919f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b919f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b410d5f0 con 0x7f57b40fb930 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b2d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b2d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:60764/0 (socket says 172.21.15.192:60764) 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.135+0000 7f57b2d9d700 1 -- 172.21.15.192:0/1265174798 learned_addr learned my addr 172.21.15.192:0/1265174798 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f57b2d9d700 1 -- 172.21.15.192:0/1265174798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8005040 con 0x7f57b40fb930 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f57b2d9d700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57a80096b0 tx=0x7f57a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57a800bed0 con 0x7f57b40fb930 2024-04-24T22:36:59.141 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b410d7f0 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57a800b5c0 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57a801c5b0 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.136+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b410e360 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.137+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f57a8013420 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.137+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4101e40 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.137+0000 7f579b7fe700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 0x7f579c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.137+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f57a8049ae0 con 0x7f57b40fb930 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.137+0000 7f57b259c700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 0x7f579c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:36:59.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.138+0000 7f57b259c700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 0x7f579c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:36:59.144 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.142+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f57a8025400 con 0x7f57b40fb930 2024-04-24T22:36:59.337 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.335+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 --> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b40663d0 con 0x7f579c0368d0 2024-04-24T22:36:59.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.340+0000 7f579b7fe700 1 -- 172.21.15.192:0/1265174798 <== mgr.14162 v2:172.21.15.63:6800/201758584 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f57b40663d0 con 0x7f579c0368d0 2024-04-24T22:36:59.343 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 msgr2=0x7f579c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 0x7f579c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a4006e40 comp rx=0 tx=0).stop 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 msgr2=0x7f57b410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57a80096b0 tx=0x7f57a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 shutdown_connections 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f579c0368d0 0x7f579c038d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 --2- 172.21.15.192:0/1265174798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57b40fb930 0x7f57b410d0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 >> 172.21.15.192:0/1265174798 conn(0x7f57b40f7070 msgr2=0x7f57b4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.342+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 shutdown_connections 2024-04-24T22:36:59.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:36:59.343+0000 7f57b919f700 1 -- 172.21.15.192:0/1265174798 wait complete. 2024-04-24T22:36:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T22:36:59.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:36:59 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:36:59.875 DEBUG:teuthology.orchestra.run.smithi192:mon.smithi192> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi192.service 2024-04-24T22:36:59.877 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:36:59.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:00.150 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:00.197 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:00.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:00 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T22:37:00.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:00 smithi063 ceph-mon[26912]: Deploying daemon grafana.smithi063 on smithi063 2024-04-24T22:37:00.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:00 smithi063 ceph-mon[26912]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:37:00.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:00 smithi063 ceph-mon[26912]: Saving service mon spec with placement smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192;count:2 2024-04-24T22:37:00.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:00 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.445+0000 7f4bc4447700 1 -- 172.21.15.192:0/2995526008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 msgr2=0x7f4bbc0fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.445+0000 7f4bc4447700 1 --2- 172.21.15.192:0/2995526008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc0fbd40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80048c0 tx=0x7f4bb8009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.445+0000 7f4bc4447700 1 -- 172.21.15.192:0/2995526008 shutdown_connections 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.445+0000 7f4bc4447700 1 --2- 172.21.15.192:0/2995526008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc0fbd40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.445+0000 7f4bc4447700 1 -- 172.21.15.192:0/2995526008 >> 172.21.15.192:0/2995526008 conn(0x7f4bbc0f7070 msgr2=0x7f4bbc0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 -- 172.21.15.192:0/2995526008 shutdown_connections 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 -- 172.21.15.192:0/2995526008 wait complete. 2024-04-24T22:37:01.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 Processor -- start 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 -- start start 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc4447700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc10c4f0 con 0x7f4bbc0fb930 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc21e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc21e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:60784/0 (socket says 172.21.15.192:60784) 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.446+0000 7f4bc21e3700 1 -- 172.21.15.192:0/181165690 learned_addr learned my addr 172.21.15.192:0/181165690 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bc21e3700 1 -- 172.21.15.192:0/181165690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8005040 con 0x7f4bbc0fb930 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bc21e3700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80096b0 tx=0x7f4bb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:01.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb800bed0 con 0x7f4bbc0fb930 2024-04-24T22:37:01.451 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bb800b5c0 con 0x7f4bbc0fb930 2024-04-24T22:37:01.451 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc10c6f0 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb801c5b0 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.447+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc10d260 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.448+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f4bb8013420 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.448+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc101e40 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.448+0000 7f4bb2ffd700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 0x7f4ba8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.448+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4bb8049a00 con 0x7f4bbc0fb930 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.449+0000 7f4bc19e2700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 0x7f4ba8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:01.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.449+0000 7f4bc19e2700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 0x7f4ba8038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009990 tx=0x7f4bac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:01.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.454+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4bb8013a80 con 0x7f4bbc0fb930 2024-04-24T22:37:01.715 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.713+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4bbc10dc70 con 0x7f4bbc0fb930 2024-04-24T22:37:01.716 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.714+0000 7f4bb2ffd700 1 -- 172.21.15.192:0/181165690 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4bb8011370 con 0x7f4bbc0fb930 2024-04-24T22:37:01.716 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:01.717 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:01.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 msgr2=0x7f4ba8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:01.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 0x7f4ba8038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009990 tx=0x7f4bac006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:01.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 msgr2=0x7f4bbc10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:01.718 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80096b0 tx=0x7f4bb8009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 shutdown_connections 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f4ba8036880 0x7f4ba8038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 --2- 172.21.15.192:0/181165690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bbc0fb930 0x7f4bbc10bfb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 >> 172.21.15.192:0/181165690 conn(0x7f4bbc0f7070 msgr2=0x7f4bbc102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 shutdown_connections 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:01.716+0000 7f4bc4447700 1 -- 172.21.15.192:0/181165690 wait complete. 2024-04-24T22:37:01.719 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:01.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:01 smithi063 ceph-mon[26912]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:01.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:01 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:02.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:02 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/181165690' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:03.253 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:03.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:03.502 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:03.550 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:03 smithi063 ceph-mon[26912]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 -- 172.21.15.192:0/2109887344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 msgr2=0x7f5cc80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 --2- 172.21.15.192:0/2109887344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc80fce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40048c0 tx=0x7f5cc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 -- 172.21.15.192:0/2109887344 shutdown_connections 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 --2- 172.21.15.192:0/2109887344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc80fce40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 -- 172.21.15.192:0/2109887344 >> 172.21.15.192:0/2109887344 conn(0x7f5cc80f8170 msgr2=0x7f5cc80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 -- 172.21.15.192:0/2109887344 shutdown_connections 2024-04-24T22:37:04.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 -- 172.21.15.192:0/2109887344 wait complete. 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.865+0000 7f5cd0694700 1 Processor -- start 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cd0694700 1 -- start start 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cd0694700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cd0694700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8072240 con 0x7f5cc80fca30 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cce430700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cce430700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:60790/0 (socket says 172.21.15.192:60790) 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cce430700 1 -- 172.21.15.192:0/148403256 learned_addr learned my addr 172.21.15.192:0/148403256 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cce430700 1 -- 172.21.15.192:0/148403256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4005040 con 0x7f5cc80fca30 2024-04-24T22:37:04.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cce430700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009680 tx=0x7f5cc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:04.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cc400b5c0 con 0x7f5cc80fca30 2024-04-24T22:37:04.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cc400be70 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.866+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8070310 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.867+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cc40135a0 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.867+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8070800 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.867+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc805f5c0 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.867+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5cc4004d30 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.868+0000 7f5cbeffd700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 0x7f5cb4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.868+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5cc4049390 con 0x7f5cc80fca30 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.868+0000 7f5ccdc2f700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 0x7f5cb4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.868+0000 7f5ccdc2f700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 0x7f5cb4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:04.876+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5cc4011960 con 0x7f5cc80fca30 2024-04-24T22:37:05.139 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.137+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5cc8106410 con 0x7f5cc80fca30 2024-04-24T22:37:05.139 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.137+0000 7f5cbeffd700 1 -- 172.21.15.192:0/148403256 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5cc4021070 con 0x7f5cc80fca30 2024-04-24T22:37:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:05.140 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 msgr2=0x7f5cb4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 0x7f5cb4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 msgr2=0x7f5cc8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009680 tx=0x7f5cc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 shutdown_connections 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f5cb4036880 0x7f5cb4038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 --2- 172.21.15.192:0/148403256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc80fca30 0x7f5cc8071d00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 >> 172.21.15.192:0/148403256 conn(0x7f5cc80f8170 msgr2=0x7f5cc8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:05.142 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 shutdown_connections 2024-04-24T22:37:05.143 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:05.140+0000 7f5cd0694700 1 -- 172.21.15.192:0/148403256 wait complete. 2024-04-24T22:37:05.143 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:05 smithi063 ceph-mon[26912]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:05 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/148403256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:06.703 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:06.703 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:06.942 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:06.989 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:07 smithi063 ceph-mon[26912]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:08.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 -- 172.21.15.192:0/1429771175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 msgr2=0x7f083c0fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:08.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 --2- 172.21.15.192:0/1429771175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c0fb290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f08380048c0 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:08.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 -- 172.21.15.192:0/1429771175 shutdown_connections 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 --2- 172.21.15.192:0/1429771175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c0fb290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 -- 172.21.15.192:0/1429771175 >> 172.21.15.192:0/1429771175 conn(0x7f083c0f6820 msgr2=0x7f083c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 -- 172.21.15.192:0/1429771175 shutdown_connections 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 -- 172.21.15.192:0/1429771175 wait complete. 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.593+0000 7f08442b7700 1 Processor -- start 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f08442b7700 1 -- start start 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f08442b7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f08442b7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c10d5e0 con 0x7f083c0fae60 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0842053700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0842053700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:33748/0 (socket says 172.21.15.192:33748) 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0842053700 1 -- 172.21.15.192:0/3902867511 learned_addr learned my addr 172.21.15.192:0/3902867511 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0842053700 1 -- 172.21.15.192:0/3902867511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838005040 con 0x7f083c0fae60 2024-04-24T22:37:08.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0842053700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0838009680 tx=0x7f0838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.594+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f083800bed0 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.595+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c10d7e0 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.595+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f083800b5c0 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.595+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f083801c5b0 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.595+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c10e350 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f0838004d30 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c102f50 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f0832ffd700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 0x7f0828038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0838049900 con 0x7f083c0fae60 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f0841852700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 0x7f0828038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.596+0000 7f0841852700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 0x7f0828038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f082c009990 tx=0x7f082c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:08.602 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.601+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0838011550 con 0x7f083c0fae60 2024-04-24T22:37:08.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.859+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f083c10ed60 con 0x7f083c0fae60 2024-04-24T22:37:08.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.860+0000 7f0832ffd700 1 -- 172.21.15.192:0/3902867511 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0838011370 con 0x7f083c0fae60 2024-04-24T22:37:08.861 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:08.861 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 msgr2=0x7f0828038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 0x7f0828038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f082c009990 tx=0x7f082c006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 msgr2=0x7f083c10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0838009680 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 shutdown_connections 2024-04-24T22:37:08.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f0828036880 0x7f0828038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 --2- 172.21.15.192:0/3902867511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c0fae60 0x7f083c10d0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 >> 172.21.15.192:0/3902867511 conn(0x7f083c0f6820 msgr2=0x7f083c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 shutdown_connections 2024-04-24T22:37:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:08.862+0000 7f08442b7700 1 -- 172.21.15.192:0/3902867511 wait complete. 2024-04-24T22:37:08.866 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:09.706 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:09 smithi063 ceph-mon[26912]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:09.706 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:09 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3902867511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:10.434 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:10.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:10.682 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:10.730 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:11.700 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:11 smithi063 ceph-mon[26912]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 -- 172.21.15.192:0/2789405009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 msgr2=0x7f63cc0fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 --2- 172.21.15.192:0/2789405009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fc3a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0048c0 tx=0x7f63bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 -- 172.21.15.192:0/2789405009 shutdown_connections 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 --2- 172.21.15.192:0/2789405009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fc3a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 -- 172.21.15.192:0/2789405009 >> 172.21.15.192:0/2789405009 conn(0x7f63cc0f6820 msgr2=0x7f63cc0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 -- 172.21.15.192:0/2789405009 shutdown_connections 2024-04-24T22:37:12.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.035+0000 7f63d2670700 1 -- 172.21.15.192:0/2789405009 wait complete. 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63d2670700 1 Processor -- start 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63d2670700 1 -- start start 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63d2670700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63d2670700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc10fdb0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:33766/0 (socket says 172.21.15.192:33766) 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63cbfff700 1 -- 172.21.15.192:0/2103267077 learned_addr learned my addr 172.21.15.192:0/2103267077 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63cbfff700 1 -- 172.21.15.192:0/2103267077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc005040 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.036+0000 7f63cbfff700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0096b0 tx=0x7f63bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63bc00bed0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc10fef0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63bc00b5c0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63bc01c5b0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.041 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc1103e0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.037+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f63bc013420 con 0x7f63cc0fbf90 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.038+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc105920 con 0x7f63cc0fbf90 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.038+0000 7f63c8ff9700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 0x7f63b4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.038+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f63bc049ac0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.038+0000 7f63cb7fe700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 0x7f63b4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:12.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.038+0000 7f63cb7fe700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 0x7f63b4038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009990 tx=0x7f63c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:12.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.043+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63bc025400 con 0x7f63cc0fbf90 2024-04-24T22:37:12.304 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.303+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f63cc05f5c0 con 0x7f63cc0fbf90 2024-04-24T22:37:12.305 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.304+0000 7f63c8ff9700 1 -- 172.21.15.192:0/2103267077 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f63bc004d30 con 0x7f63cc0fbf90 2024-04-24T22:37:12.305 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:12.305 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 msgr2=0x7f63b4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 0x7f63b4038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009990 tx=0x7f63c0006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 msgr2=0x7f63cc0fb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0096b0 tx=0x7f63bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 shutdown_connections 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f63b40368d0 0x7f63b4038d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:12.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 --2- 172.21.15.192:0/2103267077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63cc0fbf90 0x7f63cc0fb790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:12.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 >> 172.21.15.192:0/2103267077 conn(0x7f63cc0f6820 msgr2=0x7f63cc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:12.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 shutdown_connections 2024-04-24T22:37:12.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:12.306+0000 7f63d2670700 1 -- 172.21.15.192:0/2103267077 wait complete. 2024-04-24T22:37:12.308 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:12.662 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:12 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/2103267077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:13.413 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:13.413 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.413 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:13 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:13.869 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:13.870 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:14.120 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:14.167 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:14.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:14 smithi063 ceph-mon[26912]: Deploying daemon prometheus.smithi063 on smithi063 2024-04-24T22:37:15.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 -- 172.21.15.192:0/888286865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 msgr2=0x7fa0880fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:15.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 --2- 172.21.15.192:0/888286865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa0880fbd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa080009c60 tx=0x7fa080004d10 comp rx=0 tx=0).stop 2024-04-24T22:37:15.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 -- 172.21.15.192:0/888286865 shutdown_connections 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 --2- 172.21.15.192:0/888286865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa0880fbd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 -- 172.21.15.192:0/888286865 >> 172.21.15.192:0/888286865 conn(0x7fa0880f70a0 msgr2=0x7fa0880f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 -- 172.21.15.192:0/888286865 shutdown_connections 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.563+0000 7fa090637700 1 -- 172.21.15.192:0/888286865 wait complete. 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa090637700 1 Processor -- start 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa090637700 1 -- start start 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa090637700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa090637700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08810d5d0 con 0x7fa0880fb940 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa08e3d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa08e3d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:33772/0 (socket says 172.21.15.192:33772) 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa08e3d3700 1 -- 172.21.15.192:0/2211976962 learned_addr learned my addr 172.21.15.192:0/2211976962 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:15.566 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.564+0000 7fa08e3d3700 1 -- 172.21.15.192:0/2211976962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080009940 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa08e3d3700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa080009500 tx=0x7fa080004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa080007c80 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08001d910 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08810d7d0 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0800129d0 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.565+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08810e340 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.566+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fa080019040 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.566+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa088102f50 con 0x7fa0880fb940 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.566+0000 7fa07effd700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 0x7fa074038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:15.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.566+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa08004a400 con 0x7fa0880fb940 2024-04-24T22:37:15.569 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.566+0000 7fa08dbd2700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 0x7fa074038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:15.569 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.567+0000 7fa08dbd2700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 0x7fa074038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa078009990 tx=0x7fa078006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:15.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.575+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa08001da80 con 0x7fa0880fb940 2024-04-24T22:37:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:15 smithi063 ceph-mon[26912]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:15.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.832+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa08810eca0 con 0x7fa0880fb940 2024-04-24T22:37:15.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.832+0000 7fa07effd700 1 -- 172.21.15.192:0/2211976962 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa080017020 con 0x7fa0880fb940 2024-04-24T22:37:15.834 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:15.834 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 msgr2=0x7fa074038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 0x7fa074038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa078009990 tx=0x7fa078006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 msgr2=0x7fa08810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa080009500 tx=0x7fa080004d10 comp rx=0 tx=0).stop 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 shutdown_connections 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fa074036880 0x7fa074038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 --2- 172.21.15.192:0/2211976962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0880fb940 0x7fa08810d090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 >> 172.21.15.192:0/2211976962 conn(0x7fa0880f70a0 msgr2=0x7fa088103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 shutdown_connections 2024-04-24T22:37:15.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:15.835+0000 7fa090637700 1 -- 172.21.15.192:0/2211976962 wait complete. 2024-04-24T22:37:15.837 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:16 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/2211976962' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:16 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:17.471 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:17.472 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:17.721 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:17.768 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:17.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:17 smithi063 ceph-mon[26912]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 -- 172.21.15.192:0/2724859526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 msgr2=0x7f06cc0fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 --2- 172.21.15.192:0/2724859526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fb290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f06c80048c0 tx=0x7f06c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 -- 172.21.15.192:0/2724859526 shutdown_connections 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 --2- 172.21.15.192:0/2724859526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fb290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 -- 172.21.15.192:0/2724859526 >> 172.21.15.192:0/2724859526 conn(0x7f06cc0f6820 msgr2=0x7f06cc0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 -- 172.21.15.192:0/2724859526 shutdown_connections 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.015+0000 7f06d38ab700 1 -- 172.21.15.192:0/2724859526 wait complete. 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d38ab700 1 Processor -- start 2024-04-24T22:37:19.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d38ab700 1 -- start start 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d38ab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d38ab700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0fddd0 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d1647700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d1647700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:52410/0 (socket says 172.21.15.192:52410) 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d1647700 1 -- 172.21.15.192:0/1648788503 learned_addr learned my addr 172.21.15.192:0/1648788503 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.016+0000 7f06d1647700 1 -- 172.21.15.192:0/1648788503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8005040 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06d1647700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b900 tx=0x7f06c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06c800b5c0 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc0fbea0 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06c800be70 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06c801c590 con 0x7f06cc0fae60 2024-04-24T22:37:19.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.017+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc0fc390 con 0x7f06cc0fae60 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.018+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f06c8013420 con 0x7f06cc0fae60 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.018+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc05f5c0 con 0x7f06cc0fae60 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.018+0000 7f06c1ffb700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 0x7f06b8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.018+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f06c80493b0 con 0x7f06cc0fae60 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.018+0000 7f06d0e46700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 0x7f06b8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.019+0000 7f06d0e46700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 0x7f06b8038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009990 tx=0x7f06bc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:19.024 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.023+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f06c8011550 con 0x7f06cc0fae60 2024-04-24T22:37:19.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.280+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f06cc106420 con 0x7f06cc0fae60 2024-04-24T22:37:19.282 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.281+0000 7f06c1ffb700 1 -- 172.21.15.192:0/1648788503 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f06c8021020 con 0x7f06cc0fae60 2024-04-24T22:37:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:19.282 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:19.283 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.282+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 msgr2=0x7f06b8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.282+0000 7f06d38ab700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 0x7f06b8038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009990 tx=0x7f06bc006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.282+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 msgr2=0x7f06cc0fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.282+0000 7f06d38ab700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b900 tx=0x7f06c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 shutdown_connections 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f06b8036880 0x7f06b8038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 --2- 172.21.15.192:0/1648788503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06cc0fae60 0x7f06cc0fd890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 >> 172.21.15.192:0/1648788503 conn(0x7f06cc0f6820 msgr2=0x7f06cc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 shutdown_connections 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:19.283+0000 7f06d38ab700 1 -- 172.21.15.192:0/1648788503 wait complete. 2024-04-24T22:37:19.284 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:19 smithi063 ceph-mon[26912]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:19 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1648788503' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:20.868 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:20.868 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:21.113 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:21.161 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:21 smithi063 ceph-mon[26912]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:22.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 -- 172.21.15.192:0/2458387441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 msgr2=0x7f392c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:22.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 --2- 172.21.15.192:0/2458387441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c0fbd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f39240048c0 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:22.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 -- 172.21.15.192:0/2458387441 shutdown_connections 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 --2- 172.21.15.192:0/2458387441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c0fbd50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 -- 172.21.15.192:0/2458387441 >> 172.21.15.192:0/2458387441 conn(0x7f392c0f70a0 msgr2=0x7f392c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 -- 172.21.15.192:0/2458387441 shutdown_connections 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.553+0000 7f39339db700 1 -- 172.21.15.192:0/2458387441 wait complete. 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f39339db700 1 Processor -- start 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f39339db700 1 -- start start 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f39339db700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f39339db700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c10c4d0 con 0x7f392c0fb940 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f3931777700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f3931777700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:52428/0 (socket says 172.21.15.192:52428) 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.554+0000 7f3931777700 1 -- 172.21.15.192:0/4135661793 learned_addr learned my addr 172.21.15.192:0/4135661793 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:22.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f3931777700 1 -- 172.21.15.192:0/4135661793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924005040 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f3931777700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39240096b0 tx=0x7f3924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392400bed0 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c10c6d0 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392400b5c0 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392401c5b0 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.555+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c10d240 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.556+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c101e50 con 0x7f392c0fb940 2024-04-24T22:37:22.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.556+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3924013a80 con 0x7f392c0fb940 2024-04-24T22:37:22.559 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.556+0000 7f3929ffb700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 0x7f3914038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:22.559 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.556+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3924049af0 con 0x7f392c0fb940 2024-04-24T22:37:22.559 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.556+0000 7f3930f76700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 0x7f3914038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:22.559 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.557+0000 7f3930f76700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 0x7f3914038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3920009990 tx=0x7f3920006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:22.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.562+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3924011960 con 0x7f392c0fb940 2024-04-24T22:37:22.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.824+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f392c10dd10 con 0x7f392c0fb940 2024-04-24T22:37:22.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.825+0000 7f3929ffb700 1 -- 172.21.15.192:0/4135661793 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3924011370 con 0x7f392c0fb940 2024-04-24T22:37:22.826 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:22.826 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 msgr2=0x7f3914038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 0x7f3914038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3920009990 tx=0x7f3920006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 msgr2=0x7f392c10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39240096b0 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 shutdown_connections 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f3914036880 0x7f3914038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 --2- 172.21.15.192:0/4135661793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392c0fb940 0x7f392c10bf90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:22.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 >> 172.21.15.192:0/4135661793 conn(0x7f392c0f70a0 msgr2=0x7f392c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:22.829 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 shutdown_connections 2024-04-24T22:37:22.829 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:22.827+0000 7f39339db700 1 -- 172.21.15.192:0/4135661793 wait complete. 2024-04-24T22:37:22.829 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:23 smithi063 ceph-mon[26912]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:23 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/4135661793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:24.354 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:24.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:24.602 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:24.649 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:25 smithi063 ceph-mon[26912]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:26.162 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- 172.21.15.192:0/3460653000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 msgr2=0x7f16e00fc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:26.162 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 --2- 172.21.15.192:0/3460653000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fc340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16d00048c0 tx=0x7f16d0009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- 172.21.15.192:0/3460653000 shutdown_connections 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 --2- 172.21.15.192:0/3460653000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fc340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- 172.21.15.192:0/3460653000 >> 172.21.15.192:0/3460653000 conn(0x7f16e00f6820 msgr2=0x7f16e00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- 172.21.15.192:0/3460653000 shutdown_connections 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- 172.21.15.192:0/3460653000 wait complete. 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 Processor -- start 2024-04-24T22:37:26.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.156+0000 7f16e482f700 1 -- start start 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16e482f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16e482f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e010fdd0 con 0x7f16e00fbf30 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16de59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16de59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:52452/0 (socket says 172.21.15.192:52452) 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16de59c700 1 -- 172.21.15.192:0/3299772684 learned_addr learned my addr 172.21.15.192:0/3299772684 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16de59c700 1 -- 172.21.15.192:0/3299772684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0005040 con 0x7f16e00fbf30 2024-04-24T22:37:26.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16de59c700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009680 tx=0x7f16d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:26.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d000bed0 con 0x7f16e00fbf30 2024-04-24T22:37:26.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16d000b5c0 con 0x7f16e00fbf30 2024-04-24T22:37:26.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e010ff10 con 0x7f16e00fbf30 2024-04-24T22:37:26.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.157+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d001c5b0 con 0x7f16e00fbf30 2024-04-24T22:37:26.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.158+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e0110400 con 0x7f16e00fbf30 2024-04-24T22:37:26.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.158+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f16d0013420 con 0x7f16e00fbf30 2024-04-24T22:37:26.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.158+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e01058f0 con 0x7f16e00fbf30 2024-04-24T22:37:26.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.158+0000 7f16ceffd700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 0x7f16c8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:26.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.158+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f16d00499f0 con 0x7f16e00fbf30 2024-04-24T22:37:26.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.159+0000 7f16ddd9b700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 0x7f16c8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:26.167 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.159+0000 7f16ddd9b700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 0x7f16c8038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009990 tx=0x7f16d4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:26.167 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.163+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f16d0013a80 con 0x7f16e00fbf30 2024-04-24T22:37:26.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.425+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f16e005f5c0 con 0x7f16e00fbf30 2024-04-24T22:37:26.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.425+0000 7f16ceffd700 1 -- 172.21.15.192:0/3299772684 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f16d0011370 con 0x7f16e00fbf30 2024-04-24T22:37:26.428 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:26.428 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:26.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.427+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 msgr2=0x7f16c8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.427+0000 7f16e482f700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 0x7f16c8038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009990 tx=0x7f16d4006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.427+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 msgr2=0x7f16e00fb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.427+0000 7f16e482f700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009680 tx=0x7f16d0009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 shutdown_connections 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f16c80368d0 0x7f16c8038d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 --2- 172.21.15.192:0/3299772684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16e00fbf30 0x7f16e00fb730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 >> 172.21.15.192:0/3299772684 conn(0x7f16e00f6820 msgr2=0x7f16e0102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 shutdown_connections 2024-04-24T22:37:26.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:26.428+0000 7f16e482f700 1 -- 172.21.15.192:0/3299772684 wait complete. 2024-04-24T22:37:26.431 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:27 smithi063 ceph-mon[26912]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:27 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3299772684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:28.090 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:28.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:28.338 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:28.386 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:29 smithi063 ceph-mon[26912]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:29.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.827+0000 7fb1eeccb700 1 -- 172.21.15.192:0/2568856534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 msgr2=0x7fb1e80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:29.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.827+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/2568856534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e80fce40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009c60 tx=0x7fb1e0004d10 comp rx=0 tx=0).stop 2024-04-24T22:37:29.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.827+0000 7fb1eeccb700 1 -- 172.21.15.192:0/2568856534 shutdown_connections 2024-04-24T22:37:29.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.827+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/2568856534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e80fce40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.827+0000 7fb1eeccb700 1 -- 172.21.15.192:0/2568856534 >> 172.21.15.192:0/2568856534 conn(0x7fb1e80f8170 msgr2=0x7fb1e80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 -- 172.21.15.192:0/2568856534 shutdown_connections 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 -- 172.21.15.192:0/2568856534 wait complete. 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 Processor -- start 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 -- start start 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:29.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.828+0000 7fb1eeccb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e810d5f0 con 0x7fb1e80fca30 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.829+0000 7fb1eca67700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.829+0000 7fb1eca67700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:36206/0 (socket says 172.21.15.192:36206) 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.829+0000 7fb1eca67700 1 -- 172.21.15.192:0/1576742165 learned_addr learned my addr 172.21.15.192:0/1576742165 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.829+0000 7fb1eca67700 1 -- 172.21.15.192:0/1576742165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0009940 con 0x7fb1e80fca30 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1eca67700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004f80 tx=0x7fb1e0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:29.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1e0007bb0 con 0x7fb1e80fca30 2024-04-24T22:37:29.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e810d7f0 con 0x7fb1e80fca30 2024-04-24T22:37:29.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1e001c8e0 con 0x7fb1e80fca30 2024-04-24T22:37:29.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1e00129c0 con 0x7fb1e80fca30 2024-04-24T22:37:29.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.830+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e810e360 con 0x7fb1e80fca30 2024-04-24T22:37:29.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.831+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e8102f40 con 0x7fb1e80fca30 2024-04-24T22:37:29.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.831+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fb1e0019070 con 0x7fb1e80fca30 2024-04-24T22:37:29.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.832+0000 7fb1e57fa700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 0x7fb1d4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:29.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.832+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb1e004a280 con 0x7fb1e80fca30 2024-04-24T22:37:29.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.832+0000 7fb1e7fff700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 0x7fb1d4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:29.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.832+0000 7fb1e7fff700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 0x7fb1d4038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009990 tx=0x7fb1d0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:29.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:29.837+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb1e001a6f0 con 0x7fb1e80fca30 2024-04-24T22:37:30.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.097+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb1e810ed70 con 0x7fb1e80fca30 2024-04-24T22:37:30.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.098+0000 7fb1e57fa700 1 -- 172.21.15.192:0/1576742165 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb1e002a090 con 0x7fb1e80fca30 2024-04-24T22:37:30.100 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:30.100 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:30.102 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 msgr2=0x7fb1d4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:30.102 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 0x7fb1d4038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009990 tx=0x7fb1d0006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:30.102 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 msgr2=0x7fb1e810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004f80 tx=0x7fb1e0004d10 comp rx=0 tx=0).stop 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 shutdown_connections 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fb1d4036880 0x7fb1d4038d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 --2- 172.21.15.192:0/1576742165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e80fca30 0x7fb1e810d0b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 >> 172.21.15.192:0/1576742165 conn(0x7fb1e80f8170 msgr2=0x7fb1e8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 shutdown_connections 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:30.100+0000 7fb1eeccb700 1 -- 172.21.15.192:0/1576742165 wait complete. 2024-04-24T22:37:30.103 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:30.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:30 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1576742165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:31.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:31 smithi063 ceph-mon[26912]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:31.637 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:31.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:31.885 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:32.408 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- 172.21.15.192:0/782652379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 msgr2=0x7f6e480fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 --2- 172.21.15.192:0/782652379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e480fbd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e380048c0 tx=0x7f6e38009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- 172.21.15.192:0/782652379 shutdown_connections 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 --2- 172.21.15.192:0/782652379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e480fbd50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- 172.21.15.192:0/782652379 >> 172.21.15.192:0/782652379 conn(0x7f6e480f70a0 msgr2=0x7f6e480f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:33.398 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- 172.21.15.192:0/782652379 shutdown_connections 2024-04-24T22:37:33.399 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- 172.21.15.192:0/782652379 wait complete. 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 Processor -- start 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- start start 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.396+0000 7f6e4e163700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4810d5e0 con 0x7f6e480fb940 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.397+0000 7f6e477fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.397+0000 7f6e477fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:36226/0 (socket says 172.21.15.192:36226) 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.397+0000 7f6e477fe700 1 -- 172.21.15.192:0/2578175161 learned_addr learned my addr 172.21.15.192:0/2578175161 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.397+0000 7f6e477fe700 1 -- 172.21.15.192:0/2578175161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38005040 con 0x7f6e480fb940 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.397+0000 7f6e477fe700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e380096b0 tx=0x7f6e38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e3800bed0 con 0x7f6e480fb940 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e3800b5c0 con 0x7f6e480fb940 2024-04-24T22:37:33.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4810d7e0 con 0x7f6e480fb940 2024-04-24T22:37:33.401 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e3801c5b0 con 0x7f6e480fb940 2024-04-24T22:37:33.401 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4810e350 con 0x7f6e480fb940 2024-04-24T22:37:33.401 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f6e3801c820 con 0x7f6e480fb940 2024-04-24T22:37:33.402 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.398+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48101e50 con 0x7f6e480fb940 2024-04-24T22:37:33.402 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.399+0000 7f6e44ff9700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 0x7f6e34038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:33.402 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.399+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6e38049c90 con 0x7f6e480fb940 2024-04-24T22:37:33.402 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.399+0000 7f6e3effd700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 0x7f6e34038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:33.402 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.399+0000 7f6e3effd700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 0x7f6e34038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009990 tx=0x7f6e30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:33.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.404+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6e38023300 con 0x7f6e480fb940 2024-04-24T22:37:33.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:33 smithi063 ceph-mon[26912]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:33.665 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.663+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6e4810e590 con 0x7f6e480fb940 2024-04-24T22:37:33.665 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.663+0000 7f6e44ff9700 1 -- 172.21.15.192:0/2578175161 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6e38011b10 con 0x7f6e480fb940 2024-04-24T22:37:33.665 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:33.666 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:33.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 msgr2=0x7f6e34038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:33.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 0x7f6e34038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009990 tx=0x7f6e30006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:33.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 msgr2=0x7f6e4810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e380096b0 tx=0x7f6e38009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 shutdown_connections 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f6e34036880 0x7f6e34038d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 --2- 172.21.15.192:0/2578175161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e480fb940 0x7f6e4810d0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 >> 172.21.15.192:0/2578175161 conn(0x7f6e480f70a0 msgr2=0x7f6e48065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 shutdown_connections 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:33.666+0000 7f6e4e163700 1 -- 172.21.15.192:0/2578175161 wait complete. 2024-04-24T22:37:33.669 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:34.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:34 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/2578175161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:34.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:34 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:34.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:34 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:34.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:34 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:34 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-24T22:37:35.235 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:35.235 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:35.492 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:35.542 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:35 smithi063 ceph-mon[26912]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:37:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:35 smithi063 ceph-mon[26912]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-24T22:37:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:35 smithi063 ceph-mon[26912]: mgrmap e14: smithi063.upcnvd(active, since 68s) 2024-04-24T22:37:36.905 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.901+0000 7fe7d3751700 1 -- 172.21.15.192:0/3611496769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 msgr2=0x7fe7cc0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:36.905 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.901+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3611496769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc0fe490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0048c0 tx=0x7fe7bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- 172.21.15.192:0/3611496769 shutdown_connections 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3611496769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc0fe490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- 172.21.15.192:0/3611496769 >> 172.21.15.192:0/3611496769 conn(0x7fe7cc0f6820 msgr2=0x7fe7cc0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- 172.21.15.192:0/3611496769 shutdown_connections 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- 172.21.15.192:0/3611496769 wait complete. 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 Processor -- start 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- start start 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d3751700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc10d450 con 0x7fe7cc0fe080 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.902+0000 7fe7d14ed700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d14ed700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:36240/0 (socket says 172.21.15.192:36240) 2024-04-24T22:37:36.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d14ed700 1 -- 172.21.15.192:0/3177740057 learned_addr learned my addr 172.21.15.192:0/3177740057 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d14ed700 1 -- 172.21.15.192:0/3177740057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc005040 con 0x7fe7cc0fe080 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d14ed700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009f80 tx=0x7fe7bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7bc00b5c0 con 0x7fe7cc0fe080 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc10d650 con 0x7fe7cc0fe080 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7bc004e60 con 0x7fe7cc0fe080 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7bc01c7e0 con 0x7fe7cc0fe080 2024-04-24T22:37:36.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.903+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc10e0e0 con 0x7fe7cc0fe080 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7fe7bc013420 con 0x7fe7cc0fe080 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc102f50 con 0x7fe7cc0fe080 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7c1ffb700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 0x7fe7b8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe7bc049d50 con 0x7fe7cc0fe080 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7d0cec700 1 -- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 msgr2=0x7fe7b8038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/201758584 2024-04-24T22:37:36.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.904+0000 7fe7d0cec700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 0x7fe7b8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:37:36.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:36.909+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe7bc025400 con 0x7fe7cc0fe080 2024-04-24T22:37:37.107 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.105+0000 7fe7d0cec700 1 -- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 msgr2=0x7fe7b8038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/201758584 2024-04-24T22:37:37.107 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.105+0000 7fe7d0cec700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 0x7fe7b8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:37:37.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.170+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe7cc05f5c0 con 0x7fe7cc0fe080 2024-04-24T22:37:37.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.171+0000 7fe7c1ffb700 1 -- 172.21.15.192:0/3177740057 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe7bc011370 con 0x7fe7cc0fe080 2024-04-24T22:37:37.173 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:37.173 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 msgr2=0x7fe7b8038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 0x7fe7b8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 msgr2=0x7fe7cc10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009f80 tx=0x7fe7bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 shutdown_connections 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7fe7b80368d0 0x7fe7b8038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 --2- 172.21.15.192:0/3177740057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7cc0fe080 0x7fe7cc10cf10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 >> 172.21.15.192:0/3177740057 conn(0x7fe7cc0f6820 msgr2=0x7fe7cc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 shutdown_connections 2024-04-24T22:37:37.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:37.173+0000 7fe7d3751700 1 -- 172.21.15.192:0/3177740057 wait complete. 2024-04-24T22:37:37.176 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:37.488 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:37 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3177740057' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:38.708 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:38.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:38.958 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:39.018 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:40.366 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.362+0000 7f17a1b43700 1 -- 172.21.15.192:0/2504640259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 msgr2=0x7f179c0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.362+0000 7f17a1b43700 1 --2- 172.21.15.192:0/2504640259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c0fce40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f178c0048c0 tx=0x7f178c009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- 172.21.15.192:0/2504640259 shutdown_connections 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 --2- 172.21.15.192:0/2504640259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c0fce40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- 172.21.15.192:0/2504640259 >> 172.21.15.192:0/2504640259 conn(0x7f179c0f8170 msgr2=0x7f179c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- 172.21.15.192:0/2504640259 shutdown_connections 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- 172.21.15.192:0/2504640259 wait complete. 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 Processor -- start 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- start start 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.363+0000 7f17a1b43700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c072240 con 0x7f179c0fca30 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f179b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f179b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:46706/0 (socket says 172.21.15.192:46706) 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f179b7fe700 1 -- 172.21.15.192:0/1933655040 learned_addr learned my addr 172.21.15.192:0/1933655040 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:40.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f179b7fe700 1 -- 172.21.15.192:0/1933655040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c005040 con 0x7f179c0fca30 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f179b7fe700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f178c0096b0 tx=0x7f178c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f178c00bed0 con 0x7f179c0fca30 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c070310 con 0x7f179c0fca30 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f178c00b5c0 con 0x7f179c0fca30 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f178c01c5b0 con 0x7f179c0fca30 2024-04-24T22:37:40.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.364+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c070800 con 0x7f179c0fca30 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f178c013420 con 0x7f179c0fca30 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c05f5c0 con 0x7f179c0fca30 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f1783fff700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 0x7f1784038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f178c049b70 con 0x7f179c0fca30 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f179affd700 1 -- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 msgr2=0x7f1784038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/201758584 2024-04-24T22:37:40.369 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.365+0000 7f179affd700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 0x7f1784038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:37:40.372 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.370+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f178c025400 con 0x7f179c0fca30 2024-04-24T22:37:40.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.566+0000 7f179affd700 1 -- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 msgr2=0x7f1784038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/201758584 2024-04-24T22:37:40.568 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.566+0000 7f179affd700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 0x7f1784038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:37:40.630 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.628+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f179c106410 con 0x7f179c0fca30 2024-04-24T22:37:40.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.629+0000 7f1783fff700 1 -- 172.21.15.192:0/1933655040 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f178c004d30 con 0x7f179c0fca30 2024-04-24T22:37:40.631 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:40.631 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 msgr2=0x7f1784038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 0x7f1784038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 msgr2=0x7f179c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f178c0096b0 tx=0x7f178c009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 shutdown_connections 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:6800/201758584,v1:172.21.15.63:6801/201758584] conn(0x7f17840368d0 0x7f1784038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 --2- 172.21.15.192:0/1933655040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c0fca30 0x7f179c071d00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 >> 172.21.15.192:0/1933655040 conn(0x7f179c0f8170 msgr2=0x7f179c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 shutdown_connections 2024-04-24T22:37:40.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:40.631+0000 7f17a1b43700 1 -- 172.21.15.192:0/1933655040 wait complete. 2024-04-24T22:37:40.634 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:40.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:40 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1933655040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: mgrmap e15: smithi063.upcnvd(active, starting, since 0.00587097s) 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:37:41.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:37:41.924 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:37:42.256 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:42.256 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:42.498 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:42.547 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: mgrmap e16: smithi063.upcnvd(active, since 1.00833s) 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: [24/Apr/2024:22:37:42] ENGINE Bus STARTING 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: [24/Apr/2024:22:37:42] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: [24/Apr/2024:22:37:42] ENGINE Bus STARTED 2024-04-24T22:37:43.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:43 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:43.891 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 -- 172.21.15.192:0/2932539880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 msgr2=0x7f5c500fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:43.891 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 --2- 172.21.15.192:0/2932539880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c500fce30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c400048c0 tx=0x7f5c40009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 -- 172.21.15.192:0/2932539880 shutdown_connections 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 --2- 172.21.15.192:0/2932539880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c500fce30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 -- 172.21.15.192:0/2932539880 >> 172.21.15.192:0/2932539880 conn(0x7f5c500f81a0 msgr2=0x7f5c500fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 -- 172.21.15.192:0/2932539880 shutdown_connections 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 -- 172.21.15.192:0/2932539880 wait complete. 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.889+0000 7f5c55bce700 1 Processor -- start 2024-04-24T22:37:43.892 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c55bce700 1 -- start start 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c55bce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c55bce700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5010d5c0 con 0x7f5c500fca20 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:46722/0 (socket says 172.21.15.192:46722) 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c4f7fe700 1 -- 172.21.15.192:0/3313791137 learned_addr learned my addr 172.21.15.192:0/3313791137 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:43.893 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c4f7fe700 1 -- 172.21.15.192:0/3313791137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c40005040 con 0x7f5c500fca20 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.890+0000 7f5c4f7fe700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c400096b0 tx=0x7f5c40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.891+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c4000bed0 con 0x7f5c500fca20 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.891+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5010d7c0 con 0x7f5c500fca20 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.891+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c4000b5c0 con 0x7f5c500fca20 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.891+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c4001c5b0 con 0x7f5c500fca20 2024-04-24T22:37:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.891+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5010e330 con 0x7f5c500fca20 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.892+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7f5c4001c820 con 0x7f5c500fca20 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.892+0000 7f5c37fff700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 0x7f5c38038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.892+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c50102f30 con 0x7f5c500fca20 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.892+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5c400498f0 con 0x7f5c500fca20 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.892+0000 7f5c4effd700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 0x7f5c38038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:43.895 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.893+0000 7f5c4effd700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 0x7f5c38038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009990 tx=0x7f5c44006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:43.899 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:43.897+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5c40025400 con 0x7f5c500fca20 2024-04-24T22:37:44.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.173+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c5010ed40 con 0x7f5c500fca20 2024-04-24T22:37:44.176 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.174+0000 7f5c37fff700 1 -- 172.21.15.192:0/3313791137 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5c4002b4b0 con 0x7f5c500fca20 2024-04-24T22:37:44.176 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:44.176 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 msgr2=0x7f5c38038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 0x7f5c38038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009990 tx=0x7f5c44006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 msgr2=0x7f5c5010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c400096b0 tx=0x7f5c40009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 shutdown_connections 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5c38036710 0x7f5c38038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 --2- 172.21.15.192:0/3313791137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c500fca20 0x7f5c5010d080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 >> 172.21.15.192:0/3313791137 conn(0x7f5c500f81a0 msgr2=0x7f5c50103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 shutdown_connections 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:44.176+0000 7f5c55bce700 1 -- 172.21.15.192:0/3313791137 wait complete. 2024-04-24T22:37:44.179 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:44 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:44 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3313791137' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:45 smithi063 ceph-mon[26912]: mgrmap e17: smithi063.upcnvd(active, since 2s) 2024-04-24T22:37:46.562 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:46.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:47.193 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:47.242 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T22:37:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:48.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:37:48.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T22:37:48.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:48.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 -- 172.21.15.192:0/3923280581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 msgr2=0x7fa030102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:48.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 --2- 172.21.15.192:0/3923280581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030102e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0240048c0 tx=0x7fa024009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:48.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 -- 172.21.15.192:0/3923280581 shutdown_connections 2024-04-24T22:37:48.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 --2- 172.21.15.192:0/3923280581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030102e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 -- 172.21.15.192:0/3923280581 >> 172.21.15.192:0/3923280581 conn(0x7fa0300f85b0 msgr2=0x7fa0300fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 -- 172.21.15.192:0/3923280581 shutdown_connections 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.836+0000 7fa03699a700 1 -- 172.21.15.192:0/3923280581 wait complete. 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.837+0000 7fa03699a700 1 Processor -- start 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.837+0000 7fa03699a700 1 -- start start 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.837+0000 7fa03699a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:48.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.837+0000 7fa03699a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030112550 con 0x7fa030102a90 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:58794/0 (socket says 172.21.15.192:58794) 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02ffff700 1 -- 172.21.15.192:0/2531421309 learned_addr learned my addr 172.21.15.192:0/2531421309 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02ffff700 1 -- 172.21.15.192:0/2531421309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024005040 con 0x7fa030102a90 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02ffff700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b820 tx=0x7fa024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:48.841 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0240093c0 con 0x7fa030102a90 2024-04-24T22:37:48.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.838+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301126f0 con 0x7fa030102a90 2024-04-24T22:37:48.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.839+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa024013420 con 0x7fa030102a90 2024-04-24T22:37:48.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.839+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa030112be0 con 0x7fa030102a90 2024-04-24T22:37:48.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.839+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa02401c5b0 con 0x7fa030102a90 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.839+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0300fe190 con 0x7fa030102a90 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.840+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fa024013a90 con 0x7fa030102a90 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.840+0000 7fa02cff9700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 0x7fa018038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.840+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa024049af0 con 0x7fa030102a90 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.840+0000 7fa02f7fe700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 0x7fa018038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:48.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.840+0000 7fa02f7fe700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 0x7fa018038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa020009990 tx=0x7fa020006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:48.847 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:48.845+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa0240098a0 con 0x7fa030102a90 2024-04-24T22:37:49.123 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.121+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa03005f5c0 con 0x7fa030102a90 2024-04-24T22:37:49.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.122+0000 7fa02cff9700 1 -- 172.21.15.192:0/2531421309 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa024023300 con 0x7fa030102a90 2024-04-24T22:37:49.124 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:49.124 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 msgr2=0x7fa018038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 0x7fa018038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa020009990 tx=0x7fa020006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 msgr2=0x7fa030112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b820 tx=0x7fa024009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 shutdown_connections 2024-04-24T22:37:49.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fa018036970 0x7fa018038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:49.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 --2- 172.21.15.192:0/2531421309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa030102a90 0x7fa030112010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:49.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 >> 172.21.15.192:0/2531421309 conn(0x7fa0300f85b0 msgr2=0x7fa0300f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:49.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 shutdown_connections 2024-04-24T22:37:49.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:49.124+0000 7fa03699a700 1 -- 172.21.15.192:0/2531421309 wait complete. 2024-04-24T22:37:49.127 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:49 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:37:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:49 smithi063 ceph-mon[26912]: Deploying daemon crash.smithi192 on smithi192 2024-04-24T22:37:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:49 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/2531421309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:51.553 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:51.553 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:51.818 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:37:53.022 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:53.022 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:53.023 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:53.023 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:53.023 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:52 smithi063 ceph-mon[26912]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.324+0000 7ff494901700 1 -- 172.21.15.192:0/2597339866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 msgr2=0x7ff4900fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.324+0000 7ff494901700 1 --2- 172.21.15.192:0/2597339866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4900fde30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4800048c0 tx=0x7ff480009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.324+0000 7ff494901700 1 -- 172.21.15.192:0/2597339866 shutdown_connections 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.324+0000 7ff494901700 1 --2- 172.21.15.192:0/2597339866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4900fde30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.324+0000 7ff494901700 1 -- 172.21.15.192:0/2597339866 >> 172.21.15.192:0/2597339866 conn(0x7ff4900f74d0 msgr2=0x7ff4900f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:53.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 -- 172.21.15.192:0/2597339866 shutdown_connections 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 -- 172.21.15.192:0/2597339866 wait complete. 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 Processor -- start 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 -- start start 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.325+0000 7ff494901700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490113710 con 0x7ff4900fda60 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff48e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff48e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:58816/0 (socket says 172.21.15.192:58816) 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff48e59c700 1 -- 172.21.15.192:0/1691623958 learned_addr learned my addr 172.21.15.192:0/1691623958 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff48e59c700 1 -- 172.21.15.192:0/1691623958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480005040 con 0x7ff4900fda60 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff48e59c700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b820 tx=0x7ff480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4800093c0 con 0x7ff4900fda60 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.326+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff490113910 con 0x7ff4900fda60 2024-04-24T22:37:53.335 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.327+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff480013420 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.327+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff48001c5b0 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.327+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490113d80 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.327+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4900661c0 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.327+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7ff480009520 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.328+0000 7ff47effd700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 0x7ff478038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.328+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff480049b20 con 0x7ff4900fda60 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.328+0000 7ff48dd9b700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 0x7ff478038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.329+0000 7ff48dd9b700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 0x7ff478038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff484009990 tx=0x7ff484006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:53.336 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.333+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff480013a90 con 0x7ff4900fda60 2024-04-24T22:37:53.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.627+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff49005f5c0 con 0x7ff4900fda60 2024-04-24T22:37:53.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.628+0000 7ff47effd700 1 -- 172.21.15.192:0/1691623958 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff480011370 con 0x7ff4900fda60 2024-04-24T22:37:53.630 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:53.630 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:53.632 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 msgr2=0x7ff478038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:53.632 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 0x7ff478038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff484009990 tx=0x7ff484006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 msgr2=0x7ff4901131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b820 tx=0x7ff480009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 shutdown_connections 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff478036970 0x7ff478038e20 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff484009990 tx=0x7ff484006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 --2- 172.21.15.192:0/1691623958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4900fda60 0x7ff4901131d0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b820 tx=0x7ff480009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 >> 172.21.15.192:0/1691623958 conn(0x7ff4900f74d0 msgr2=0x7ff4901036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 shutdown_connections 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:53.630+0000 7ff494901700 1 -- 172.21.15.192:0/1691623958 wait complete. 2024-04-24T22:37:53.633 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:53 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1691623958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:37:55.188 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:37:55.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:37:55.689 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:37:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:37:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:37:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:37:58.151 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/1419765029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 msgr2=0x7f2cf0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/1419765029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0102e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00048c0 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/1419765029 shutdown_connections 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/1419765029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0102e70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/1419765029 >> 172.21.15.192:0/1419765029 conn(0x7f2cf00f85c0 msgr2=0x7f2cf00fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/1419765029 shutdown_connections 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.148+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/1419765029 wait complete. 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.149+0000 7f2cf5cf3700 1 Processor -- start 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.149+0000 7f2cf5cf3700 1 -- start start 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.149+0000 7f2cf5cf3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.149+0000 7f2cf5cf3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0072260 con 0x7f2cf0102aa0 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.149+0000 7f2cef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:50858/0 (socket says 172.21.15.192:50858) 2024-04-24T22:37:58.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cef7fe700 1 -- 172.21.15.192:0/3436642109 learned_addr learned my addr 172.21.15.192:0/3436642109 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:37:58.153 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cef7fe700 1 -- 172.21.15.192:0/3436642109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0005040 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cef7fe700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b7f0 tx=0x7f2ce0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ce0009370 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf0070800 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ce0013420 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ce001c590 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.150+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf0070cf0 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.151+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf00fe1a0 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.151+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f2ce0013a70 con 0x7f2cf0102aa0 2024-04-24T22:37:58.154 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.151+0000 7f2cd7fff700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 0x7f2cd8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:37:58.155 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.151+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2ce0049420 con 0x7f2cf0102aa0 2024-04-24T22:37:58.155 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.151+0000 7f2ceeffd700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 0x7f2cd8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:37:58.155 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.152+0000 7f2ceeffd700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 0x7f2cd8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce4006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:37:58.158 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.157+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ce0024b50 con 0x7f2cf0102aa0 2024-04-24T22:37:58.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.420+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2cf004bf00 con 0x7f2cf0102aa0 2024-04-24T22:37:58.423 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.421+0000 7f2cd7fff700 1 -- 172.21.15.192:0/3436642109 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2ce0021030 con 0x7f2cf0102aa0 2024-04-24T22:37:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:37:58.423 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 msgr2=0x7f2cd8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 0x7f2cd8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce4006e40 comp rx=0 tx=0).stop 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 msgr2=0x7f2cf0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b7f0 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 shutdown_connections 2024-04-24T22:37:58.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f2cd8036970 0x7f2cd8038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:58.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 --2- 172.21.15.192:0/3436642109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf0102aa0 0x7f2cf0070260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:37:58.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 >> 172.21.15.192:0/3436642109 conn(0x7f2cf00f85c0 msgr2=0x7f2cf00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:37:58.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 shutdown_connections 2024-04-24T22:37:58.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:37:58.423+0000 7f2cf5cf3700 1 -- 172.21.15.192:0/3436642109 wait complete. 2024-04-24T22:37:58.426 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:37:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:58 smithi063 ceph-mon[26912]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:37:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:37:58 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3436642109' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:00.067 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:38:00.068 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:38:00.418 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:01.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:01 smithi063 ceph-mon[26912]: Deploying daemon mon.smithi192 on smithi192 2024-04-24T22:38:02.917 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 -- 172.21.15.192:0/778183807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 msgr2=0x7fd2a80fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 --2- 172.21.15.192:0/778183807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fde40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40048c0 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 -- 172.21.15.192:0/778183807 shutdown_connections 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 --2- 172.21.15.192:0/778183807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fde40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 -- 172.21.15.192:0/778183807 >> 172.21.15.192:0/778183807 conn(0x7fd2a8071800 msgr2=0x7fd2a8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 -- 172.21.15.192:0/778183807 shutdown_connections 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.915+0000 7fd2b0700700 1 -- 172.21.15.192:0/778183807 wait complete. 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.916+0000 7fd2b0700700 1 Processor -- start 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.916+0000 7fd2b0700700 1 -- start start 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.916+0000 7fd2b0700700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.916+0000 7fd2b0700700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80fbe70 con 0x7fd2a80fda70 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.916+0000 7fd2ae49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2ae49c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:50912/0 (socket says 172.21.15.192:50912) 2024-04-24T22:38:02.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2ae49c700 1 -- 172.21.15.192:0/3514825239 learned_addr learned my addr 172.21.15.192:0/3514825239 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2ae49c700 1 -- 172.21.15.192:0/3514825239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4005040 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2ae49c700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b820 tx=0x7fd2a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2a40093c0 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a4013420 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80fc070 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2a401c5b0 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.917+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a80fc4e0 con 0x7fd2a80fda70 2024-04-24T22:38:02.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.918+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fd2a401c7f0 con 0x7fd2a80fda70 2024-04-24T22:38:02.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.918+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8103370 con 0x7fd2a80fda70 2024-04-24T22:38:02.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.918+0000 7fd29effd700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 0x7fd294038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:02.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.918+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd2a4049b50 con 0x7fd2a80fda70 2024-04-24T22:38:02.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.919+0000 7fd2adc9b700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 0x7fd294038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:02.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.919+0000 7fd2adc9b700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 0x7fd294038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd298009990 tx=0x7fd298006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:02.925 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:02.924+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd2a4023300 con 0x7fd2a80fda70 2024-04-24T22:38:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:02 smithi063 ceph-mon[26912]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:03.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.207+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd2a80658b0 con 0x7fd2a80fda70 2024-04-24T22:38:03.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.208+0000 7fd29effd700 1 -- 172.21.15.192:0/3514825239 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd2a4011b60 con 0x7fd2a80fda70 2024-04-24T22:38:03.225 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:38:03.226 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:35:18.856966Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 msgr2=0x7fd294038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 0x7fd294038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd298009990 tx=0x7fd298006e40 comp rx=0 tx=0).stop 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 msgr2=0x7fd2a80fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b820 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 shutdown_connections 2024-04-24T22:38:03.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd294036970 0x7fd294038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 --2- 172.21.15.192:0/3514825239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2a80fda70 0x7fd2a80fb930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:03.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 >> 172.21.15.192:0/3514825239 conn(0x7fd2a8071800 msgr2=0x7fd2a8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:03.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 shutdown_connections 2024-04-24T22:38:03.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:03.210+0000 7fd2b0700700 1 -- 172.21.15.192:0/3514825239 wait complete. 2024-04-24T22:38:03.228 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-24T22:38:04.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:03 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3514825239' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:04.765 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T22:38:04.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T22:38:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:04 smithi063 ceph-mon[26912]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:05.118 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:38:05.241 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:05 smithi192 systemd[1]: Starting Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:38:06.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 podman[33873]: 2024-04-24 22:38:06.229817283 +0000 UTC m=+0.850584072 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17 2024-04-24T22:38:06.454 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 podman[33873]: 2024-04-24 22:38:06.34520873 +0000 UTC m=+0.965975515 container create 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T22:38:06.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 -- 172.21.15.192:0/4263458512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 msgr2=0x7f0544101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:06.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 --2- 172.21.15.192:0/4263458512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544101e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05400048c0 tx=0x7f0540009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:06.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 -- 172.21.15.192:0/4263458512 shutdown_connections 2024-04-24T22:38:06.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 --2- 172.21.15.192:0/4263458512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544101e80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:06.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 -- 172.21.15.192:0/4263458512 >> 172.21.15.192:0/4263458512 conn(0x7f05440f75f0 msgr2=0x7f05440f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 -- 172.21.15.192:0/4263458512 shutdown_connections 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.833+0000 7f054bad5700 1 -- 172.21.15.192:0/4263458512 wait complete. 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f054bad5700 1 Processor -- start 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f054bad5700 1 -- start start 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f054bad5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f054bad5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544112680 con 0x7f0544101ab0 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f0549871700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f0549871700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.192:50944/0 (socket says 172.21.15.192:50944) 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.834+0000 7f0549871700 1 -- 172.21.15.192:0/1149401500 learned_addr learned my addr 172.21.15.192:0/1149401500 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:38:06.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f0549871700 1 -- 172.21.15.192:0/1149401500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540005040 con 0x7f0544101ab0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f0549871700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f054000b820 tx=0x7f0540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05400093c0 con 0x7f0544101ab0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0540013420 con 0x7f0544101ab0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f054001c5b0 con 0x7f0544101ab0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0544112880 con 0x7f0544101ab0 2024-04-24T22:38:06.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.835+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544112cb0 con 0x7f0544101ab0 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.836+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f0540013a90 con 0x7f0544101ab0 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.836+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05440fd1b0 con 0x7f0544101ab0 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.836+0000 7f053a7fc700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 0x7f0530038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.836+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0540049b60 con 0x7f0544101ab0 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.836+0000 7f0549070700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 0x7f0530038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:06.839 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.837+0000 7f0549070700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 0x7f0530038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0534009990 tx=0x7f0534006e40 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:06.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:06.841+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f05400098a0 con 0x7f0544101ab0 2024-04-24T22:38:07.000 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 podman[33873]: 2024-04-24 22:38:06.976453957 +0000 UTC m=+1.597220744 container init 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:38:07.000 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 podman[33873]: 2024-04-24 22:38:06.979656403 +0000 UTC m=+1.600423189 container start 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS) 2024-04-24T22:38:07.032 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:07.031+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f054004e020 con 0x7f0544101ab0 2024-04-24T22:38:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:06 smithi063 ceph-mon[26912]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:07.125+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f054410d790 con 0x7f0544101ab0 2024-04-24T22:38:07.349 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 ceph-mon[33991]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T22:38:07.349 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 ceph-mon[33991]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-24T22:38:07.349 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:06 smithi192 ceph-mon[33991]: pidfile_write: ignore empty --pid-file 2024-04-24T22:38:07.349 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: load: jerasure load: lrc 2024-04-24T22:38:07.349 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: RocksDB version: 6.15.5 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Compile date Apr 24 2024 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: DB SUMMARY 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: DB Session ID: VHE4W4D2QUVEP53JC0OH 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: CURRENT file: CURRENT 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi192/store.db dir, Total Num: 0, files: 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi192/store.db: 000004.log size: 511 ; 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.error_if_exists: 0 2024-04-24T22:38:07.350 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.create_if_missing: 0 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.paranoid_checks: 1 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.env: 0x5635edc5eb00 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.fs: Posix File System 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.info_log: 0x5635eea79620 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.statistics: (nil) 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.use_fsync: 0 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_log_file_size: 0 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T22:38:07.351 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_fallocate: 1 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.db_log_dir: 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi192/store.db 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:38:07.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:38:07.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.write_buffer_manager: 0x5635ef76ca20 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:38:07.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.unordered_write: 0 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.row_cache: None 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.wal_filter: None 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:38:07.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.preserve_deletes: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:38:07.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_open_files: -1 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:38:07.358 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Compression algorithms supported: 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kZSTD supported: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:38:07.359 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000003 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.merge_operator: 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_filter: None 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5635eea4cf58) 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:38:07.360 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_type: 0 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_index_type: 0 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_shortening: 1 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: hash_index_allow_collision: 1 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: checksum: 1 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: no_block_cache: 0 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache: 0x5635eea7f090 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_options: 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: capacity : 536870912 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_shard_bits : 4 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: strict_capacity_limit : 0 2024-04-24T22:38:07.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_compressed: (nil) 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: persistent_cache: (nil) 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size: 4096 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size_deviation: 10 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_restart_interval: 16 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_block_restart_interval: 1 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_block_size: 4096 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: partition_filters: 0 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: use_delta_encoding: 1 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: whole_key_filtering: 1 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: verify_compression: 0 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:38:07.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: format_version: 4 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_index_compression: 1 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_align: 0 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression: NoCompression 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:38:07.363 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:38:07.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:38:07.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.num_levels: 7 2024-04-24T22:38:07.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:38:07.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:38:07.364 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:38:07.365 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:38:07.366 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.arena_block_size: 4194304 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:38:07.367 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:38:07.368 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.ttl: 2592000 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:38:07.369 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998287005381, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998287005692, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1713998287, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "994a5535-2f1d-4d51-8120-148e737b1d3b", "db_session_id": "VHE4W4D2QUVEP53JC0OH"}} 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998287005995, "job": 1, "event": "recovery_finished"} 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi192/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5635eea13180 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: DB pointer 0x5635eeae8000 2024-04-24T22:38:07.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** DB Stats ** 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:38:07.371 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:38:07.372 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-24T22:38:07.373 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:38:07.374 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192 does not exist in monmap, will attempt to join an existing cluster 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: starting mon.smithi192 rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi192 fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(???) e0 preinit fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).mds e1 new map 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).mds e1 print_map 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: e1 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: legacy client fscid: -1 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:38:07.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: No filesystems configured 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T22:38:07.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying cephadm binary to smithi192 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Added host smithi192 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon alertmanager.smithi063 on smithi063 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1483024621' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1483024621' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon grafana.smithi063 on smithi063 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Saving service mon spec with placement smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192;count:2 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/181165690' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/148403256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3902867511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/2103267077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon prometheus.smithi063 on smithi063 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/2211976962' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1648788503' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/4135661793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3299772684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1576742165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.381 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/2578175161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14162 172.21.15.63:0/851132232' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mgrmap e14: smithi063.upcnvd(active, since 68s) 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3177740057' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1933655040' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:38:07.382 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mgrmap e15: smithi063.upcnvd(active, starting, since 0.00587097s) 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:38:07.383 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mgrmap e16: smithi063.upcnvd(active, since 1.00833s) 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: [24/Apr/2024:22:37:42] ENGINE Bus STARTING 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: [24/Apr/2024:22:37:42] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: [24/Apr/2024:22:37:42] ENGINE Bus STARTED 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3313791137' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mgrmap e17: smithi063.upcnvd(active, since 2s) 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:07.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon crash.smithi192 on smithi192 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/2531421309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1691623958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3436642109' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: Deploying daemon mon.smithi192 on smithi192 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3514825239' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 ceph-mon[33991]: mon.smithi192@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 bash[33873]: 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c 2024-04-24T22:38:07.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:07 smithi192 systemd[1]: Started Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:38:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-24T22:38:12.043 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","modified":"2024-04-24T22:38:07.030569Z","created":"2024-04-24T22:35:18.856966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi192","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T22:38:12.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.040+0000 7f053a7fc700 1 -- 172.21.15.192:0/1149401500 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f0540011370 con 0x7f0544101ab0 2024-04-24T22:38:12.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 msgr2=0x7f0530038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 0x7f0530038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0534009990 tx=0x7f0534006e40 comp rx=0 tx=0).stop 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 msgr2=0x7f0544112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f054000b820 tx=0x7f0540009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 shutdown_connections 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0530036970 0x7f0530038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 --2- 172.21.15.192:0/1149401500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0544101ab0 0x7f0544112140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 >> 172.21.15.192:0/1149401500 conn(0x7f05440f75f0 msgr2=0x7f05440f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 shutdown_connections 2024-04-24T22:38:12.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:38:12.044+0000 7f054bad5700 1 -- 172.21.15.192:0/1149401500 wait complete. 2024-04-24T22:38:12.047 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: mon.smithi063 calling monitor election 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1149401500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: mon.smithi192 calling monitor election 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.407 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: fsmap 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: mgrmap e17: smithi063.upcnvd(active, since 30s) 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: overall HEALTH_OK 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:12.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: mon.smithi063 calling monitor election 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1149401500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.459 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: mon.smithi192 calling monitor election 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: fsmap 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: mgrmap e17: smithi063.upcnvd(active, since 30s) 2024-04-24T22:38:12.460 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: overall HEALTH_OK 2024-04-24T22:38:12.461 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:38:12.461 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:12.461 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:12.666 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T22:38:12.666 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T22:38:12.937 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: mgrmap e18: smithi063.upcnvd(active, since 30s), standbys: smithi192.ctmwwc 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:13.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: mgrmap e18: smithi063.upcnvd(active, since 30s), standbys: smithi192.ctmwwc 2024-04-24T22:38:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:38:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:13.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:13.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:38:14.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:14 smithi063 ceph-mon[26912]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:14.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:14.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:14.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:14.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:14 smithi192 ceph-mon[33991]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:38:14.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.540+0000 7f4307f8a700 1 -- 172.21.15.63:0/2321653622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 msgr2=0x7f4300102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:14.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.540+0000 7f4307f8a700 1 --2- 172.21.15.63:0/2321653622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300102e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0048c0 tx=0x7f42fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:14.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.540+0000 7f4307f8a700 1 -- 172.21.15.63:0/2321653622 shutdown_connections 2024-04-24T22:38:14.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.540+0000 7f4307f8a700 1 --2- 172.21.15.63:0/2321653622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300102e70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.541+0000 7f4307f8a700 1 -- 172.21.15.63:0/2321653622 >> 172.21.15.63:0/2321653622 conn(0x7f43000f85c0 msgr2=0x7f43000fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.541+0000 7f4307f8a700 1 -- 172.21.15.63:0/2321653622 shutdown_connections 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.541+0000 7f4307f8a700 1 -- 172.21.15.63:0/2321653622 wait complete. 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.541+0000 7f4307f8a700 1 Processor -- start 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.541+0000 7f4307f8a700 1 -- start start 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4307f8a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4307f8a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4307f8a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300119510 con 0x7f4300102aa0 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4307f8a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300119650 con 0x7f43000707a0 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50128/0 (socket says 172.21.15.63:50128) 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 -- 172.21.15.63:0/1165963262 learned_addr learned my addr 172.21.15.63:0/1165963262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:38:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305525700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 -- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 msgr2=0x7f4300070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305525700 1 -- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 msgr2=0x7f4300070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305525700 1 -- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 msgr2=0x7f4300070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305525700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305525700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 -- 172.21.15.63:0/1165963262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc005040 con 0x7f4300102aa0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f4305d26700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00b5c0 tx=0x7f42fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.542+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42fc00e040 con 0x7f4300102aa0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.543+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43001197f0 con 0x7f4300102aa0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.543+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42fc009950 con 0x7f4300102aa0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.543+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42fc01c3f0 con 0x7f4300102aa0 2024-04-24T22:38:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.543+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4300119db0 con 0x7f4300102aa0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.544+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300101d10 con 0x7f4300102aa0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.544+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f42fc0137d0 con 0x7f4300102aa0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.544+0000 7f42f67fc700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 0x7f42ec06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.544+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f42fc087440 con 0x7f4300102aa0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.545+0000 7f4305525700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 0x7f42ec06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.545+0000 7f4305525700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 0x7f42ec06b810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006f30 tx=0x7f42f0008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.550+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f42fc054850 con 0x7f4300102aa0 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.778+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f43000fe1a0 con 0x7f4300102aa0 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.778+0000 7f42f67fc700 1 -- 172.21.15.63:0/1165963262 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f42fc086bf0 con 0x7f4300102aa0 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stdout:# minimal ceph.conf for c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:38:14.780 INFO:teuthology.orchestra.run.smithi063.stdout: mon_host = [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-04-24T22:38:14.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.780+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 msgr2=0x7f42ec06b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:14.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.780+0000 7f4307f8a700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 0x7f42ec06b810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006f30 tx=0x7f42f0008040 comp rx=0 tx=0).stop 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.780+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 msgr2=0x7f4300070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.780+0000 7f4307f8a700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00b5c0 tx=0x7f42fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 shutdown_connections 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f42ec069360 0x7f42ec06b810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4300102aa0 0x7f4300070260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 --2- 172.21.15.63:0/1165963262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43000707a0 0x7f4300070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 >> 172.21.15.63:0/1165963262 conn(0x7f43000f85c0 msgr2=0x7f43000f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 shutdown_connections 2024-04-24T22:38:14.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:14.781+0000 7f4307f8a700 1 -- 172.21.15.63:0/1165963262 wait complete. 2024-04-24T22:38:15.391 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T22:38:15.391 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:38:15.392 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T22:38:15.418 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:38:15.418 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:38:15.483 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:38:15.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T22:38:15.514 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:38:15.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:38:15.582 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T22:38:15.583 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:38:15.583 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:38:15.599 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:38:15.599 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-24 22:37:33.116359643 +0000 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-24 22:31:16.677146853 +0000 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-24 22:31:16.677146853 +0000 2024-04-24T22:38:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-24 22:31:16.677146853 +0000 2024-04-24T22:38:15.655 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: Reconfiguring daemon mon.smithi063 on smithi063 2024-04-24T22:38:15.695 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:15 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1165963262' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:15.719 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-24T22:38:15.719 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-24T22:38:15.719 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000263145 s, 1.9 MB/s 2024-04-24T22:38:15.720 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:38:15.776 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-04-24T22:38:15.832 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:38:15.832 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:15.832 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T22:38:15.832 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:15.832 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:15.833 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-24 22:37:33.116359643 +0000 2024-04-24T22:38:15.833 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-24 22:31:17.028143942 +0000 2024-04-24T22:38:15.833 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-24 22:31:17.028143942 +0000 2024-04-24T22:38:15.833 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-24 22:31:17.028143942 +0000 2024-04-24T22:38:15.833 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:38:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-24T22:38:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-24T22:38:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000185233 s, 2.8 MB/s 2024-04-24T22:38:15.897 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: Reconfiguring daemon mon.smithi063 on smithi063 2024-04-24T22:38:15.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:15 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1165963262' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:15.953 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.009 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-24 22:37:33.116359643 +0000 2024-04-24T22:38:16.010 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-24 22:31:17.394140906 +0000 2024-04-24T22:38:16.010 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-24 22:31:17.394140906 +0000 2024-04-24T22:38:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-24 22:31:17.394140906 +0000 2024-04-24T22:38:16.010 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:38:16.072 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-24T22:38:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-24T22:38:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000195712 s, 2.6 MB/s 2024-04-24T22:38:16.074 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:38:16.130 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-04-24T22:38:16.187 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:38:16.187 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-24 22:37:33.117359634 +0000 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-24 22:31:17.740138036 +0000 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-24 22:31:17.740138036 +0000 2024-04-24T22:38:16.188 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-24 22:31:17.740138036 +0000 2024-04-24T22:38:16.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:38:16.250 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-24T22:38:16.250 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-24T22:38:16.250 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000242818 s, 2.1 MB/s 2024-04-24T22:38:16.251 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:38:16.308 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:38:16.308 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:38:16.327 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:38:16.327 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-04-24T22:38:16.384 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:38:16.384 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.384 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-24 22:38:05.005286954 +0000 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-24 22:31:12.038360004 +0000 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-24 22:31:12.038360004 +0000 2024-04-24T22:38:16.385 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-04-24 22:31:12.038360004 +0000 2024-04-24T22:38:16.385 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:38:16.450 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-24T22:38:16.450 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-24T22:38:16.450 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000230993 s, 2.2 MB/s 2024-04-24T22:38:16.451 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:38:16.509 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-04-24T22:38:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:38:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.566 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T22:38:16.566 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.566 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.567 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-24 22:38:05.006286936 +0000 2024-04-24T22:38:16.567 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-24 22:31:12.388353650 +0000 2024-04-24T22:38:16.567 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-24 22:31:12.388353650 +0000 2024-04-24T22:38:16.567 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-04-24 22:31:12.388353650 +0000 2024-04-24T22:38:16.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:38:16.629 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-24T22:38:16.630 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-24T22:38:16.630 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000222503 s, 2.3 MB/s 2024-04-24T22:38:16.631 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:38:16.687 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-24 22:38:05.006286936 +0000 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-24 22:31:12.707347860 +0000 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-24 22:31:12.707347860 +0000 2024-04-24T22:38:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-04-24 22:31:12.707347860 +0000 2024-04-24T22:38:16.744 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:38:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:16 smithi063 ceph-mon[26912]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:16 smithi192 ceph-mon[33991]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:16.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:16.805 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-24T22:38:16.806 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-24T22:38:16.806 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000171381 s, 3.0 MB/s 2024-04-24T22:38:16.807 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:38:16.862 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-04-24T22:38:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-24 22:38:05.006286936 +0000 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-24 22:31:13.040341815 +0000 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-24 22:31:13.040341815 +0000 2024-04-24T22:38:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-04-24 22:31:13.040341815 +0000 2024-04-24T22:38:16.918 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:38:16.980 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-24T22:38:16.980 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-24T22:38:16.980 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000250071 s, 2.0 MB/s 2024-04-24T22:38:16.981 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:38:17.038 INFO:tasks.cephadm:Deploying osd.0 on smithi063 with /dev/vg_nvme/lv_4... 2024-04-24T22:38:17.038 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T22:38:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:17 smithi192 ceph-mon[33991]: Reconfiguring mgr.smithi063.upcnvd (unknown last config time)... 2024-04-24T22:38:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:17 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:38:17.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:17 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:38:17.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:17 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:17.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:17 smithi192 ceph-mon[33991]: Reconfiguring daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:38:17.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:17 smithi063 ceph-mon[26912]: Reconfiguring mgr.smithi063.upcnvd (unknown last config time)... 2024-04-24T22:38:17.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:17 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:38:17.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:17 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:38:17.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:17 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:17.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:17 smithi063 ceph-mon[26912]: Reconfiguring daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:38:18.067 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:38:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:18 smithi192 ceph-mon[33991]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:38:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:18 smithi063 ceph-mon[26912]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:38:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:19.741 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:19 smithi063 ceph-mon[26912]: Reconfiguring crash.smithi063 (monmap changed)... 2024-04-24T22:38:19.741 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:19 smithi063 ceph-mon[26912]: Reconfiguring daemon crash.smithi063 on smithi063 2024-04-24T22:38:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:19 smithi192 ceph-mon[33991]: Reconfiguring crash.smithi063 (monmap changed)... 2024-04-24T22:38:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:19 smithi192 ceph-mon[33991]: Reconfiguring daemon crash.smithi063 on smithi063 2024-04-24T22:38:20.637 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112477 s, 932 MB/s 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 2024-04-24T22:38:20.638 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-04-24T22:38:20.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:20 smithi192 ceph-mon[33991]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:21.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:20 smithi063 ceph-mon[26912]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:21.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-04-24T22:38:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:21 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:21 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:21 smithi063 ceph-mon[26912]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-04-24T22:38:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:21 smithi063 ceph-mon[26912]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-04-24T22:38:22.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:21 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:22.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:21 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:22.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:21 smithi192 ceph-mon[33991]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-04-24T22:38:22.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:21 smithi192 ceph-mon[33991]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-04-24T22:38:22.883 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:22 smithi063 ceph-mon[26912]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:22 smithi192 ceph-mon[33991]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:23.454 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:38:24.954 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:24 smithi063 ceph-mon[26912]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:24 smithi192 ceph-mon[33991]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:25.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 -- 172.21.15.63:0/2490437793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 msgr2=0x7f06640fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:25.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 --2- 172.21.15.63:0/2490437793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f06640fd270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f064c004940 tx=0x7f064c009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:25.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 -- 172.21.15.63:0/2490437793 shutdown_connections 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 --2- 172.21.15.63:0/2490437793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 0x7f06640fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 --2- 172.21.15.63:0/2490437793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f06640fd270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 -- 172.21.15.63:0/2490437793 >> 172.21.15.63:0/2490437793 conn(0x7f06640f8540 msgr2=0x7f06640fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 -- 172.21.15.63:0/2490437793 shutdown_connections 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.456+0000 7f066b0d4700 1 -- 172.21.15.63:0/2490437793 wait complete. 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 Processor -- start 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 -- start start 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 0x7f066410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066410e460 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.457+0000 7f066b0d4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066410e5d0 con 0x7f06640fdfd0 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49656/0 (socket says 172.21.15.63:49656) 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 -- 172.21.15.63:0/3692891269 learned_addr learned my addr 172.21.15.63:0/3692891269 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0663fff700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 0x7f066410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 -- 172.21.15.63:0/3692891269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 msgr2=0x7f066410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 0x7f066410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 -- 172.21.15.63:0/3692891269 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c005040 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f0668e70700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f064c009150 tx=0x7f064c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064c00e040 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f064c00bbf0 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0664112fd0 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.458+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064c01c630 con 0x7f06640fce60 2024-04-24T22:38:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.459+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0664113500 con 0x7f06640fce60 2024-04-24T22:38:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.460+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066404c710 con 0x7f06640fce60 2024-04-24T22:38:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.460+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f064c00bd60 con 0x7f06640fce60 2024-04-24T22:38:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.460+0000 7f06617fa700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 0x7f065006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.460+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f064c0873c0 con 0x7f06640fce60 2024-04-24T22:38:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.460+0000 7f0663fff700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 0x7f065006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.461+0000 7f0663fff700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 0x7f065006b6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0654004550 tx=0x7f0654009040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.465+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f064c0583b0 con 0x7f06640fce60 2024-04-24T22:38:25.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:25.673+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f06641023b0 con 0x7f0650069210 2024-04-24T22:38:25.978 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:25 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:38:25.979 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:25 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:38:25.979 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:25 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:25 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:38:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:25 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:38:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:25 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:27.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:26 smithi063 ceph-mon[26912]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:26 smithi063 ceph-mon[26912]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:26 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:27.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:26 smithi192 ceph-mon[33991]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:27.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:26 smithi192 ceph-mon[33991]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:38:27.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:26 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:29.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:28 smithi063 ceph-mon[26912]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:29.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:29.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:28 smithi192 ceph-mon[33991]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3824534130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61754976-3983-4832-86d4-497489f4e01f"}]: dispatch 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3824534130' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61754976-3983-4832-86d4-497489f4e01f"}]': finished 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T22:38:30.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-04-24T22:38:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-04-24T22:38:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3824534130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61754976-3983-4832-86d4-497489f4e01f"}]: dispatch 2024-04-24T22:38:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3824534130' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61754976-3983-4832-86d4-497489f4e01f"}]': finished 2024-04-24T22:38:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T22:38:30.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:30 smithi063 ceph-mon[26912]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:30 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4117665541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:38:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:30 smithi192 ceph-mon[33991]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:30 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4117665541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:38:33.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:32 smithi063 ceph-mon[26912]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:32 smithi192 ceph-mon[33991]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:34.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:34 smithi063 ceph-mon[26912]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:34 smithi192 ceph-mon[33991]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:36.998 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:36 smithi192 ceph-mon[33991]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:36 smithi063 ceph-mon[26912]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:38.890 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:38 smithi063 ceph-mon[26912]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:38.890 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:38 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:38.890 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:38 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:39.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:38 smithi192 ceph-mon[33991]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:39.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:38 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:39.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:38 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:39.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:39 smithi063 ceph-mon[26912]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:38:39.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:39 smithi063 ceph-mon[26912]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:38:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:39 smithi192 ceph-mon[33991]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:38:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:39 smithi192 ceph-mon[33991]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:38:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:40 smithi063 ceph-mon[26912]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:38:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:40 smithi192 ceph-mon[33991]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:38:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:41 smithi063 ceph-mon[26912]: Deploying daemon osd.0 on smithi063 2024-04-24T22:38:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:41 smithi192 ceph-mon[33991]: Deploying daemon osd.0 on smithi063 2024-04-24T22:38:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:43.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:42 smithi063 ceph-mon[26912]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:42 smithi192 ceph-mon[33991]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:44.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:44 smithi063 ceph-mon[26912]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:44.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:44 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:44.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:44 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:44.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:44 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:38:44.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:44 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:44.915 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:44 smithi192 ceph-mon[33991]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:44.916 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:44 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:44.916 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:44 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:44.916 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:44 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:38:44.916 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:44 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:45.920 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-24T22:38:45.920 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-24T22:38:45.920 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:45.921 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:45.921 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:38:45.921 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:38:45.921 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:46.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-24T22:38:46.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-24T22:38:46.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:46.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:46.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:38:46.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:38:46.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:46.904 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:46 smithi063 ceph-mon[26912]: Reconfiguring mgr.smithi192.ctmwwc (monmap changed)... 2024-04-24T22:38:46.904 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:46 smithi063 ceph-mon[26912]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:46.904 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:46 smithi063 ceph-mon[26912]: Reconfiguring daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:38:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:46 smithi192 ceph-mon[33991]: Reconfiguring mgr.smithi192.ctmwwc (monmap changed)... 2024-04-24T22:38:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:46 smithi192 ceph-mon[33991]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:46 smithi192 ceph-mon[33991]: Reconfiguring daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-24T22:38:48.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:47 smithi063 ceph-mon[26912]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-24T22:38:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:47 smithi192 ceph-mon[33991]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:49.387 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:38:49.388 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:38:49.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:38:49.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:38:49.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.389 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:49 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:49.424 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.424 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.424 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:38:49.424 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:49.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:49 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:38:50.383 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.383 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.383 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:50 smithi063 ceph-mon[26912]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:50.383 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.383 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:50 smithi192 ceph-mon[33991]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:52.707 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:52 smithi063 ceph-mon[26912]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:52.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.705+0000 7f06617fa700 1 -- 172.21.15.63:0/3692891269 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f06641023b0 con 0x7f0650069210 2024-04-24T22:38:52.708 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 0 on host 'smithi063' 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 msgr2=0x7f065006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 0x7f065006b6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0654004550 tx=0x7f0654009040 comp rx=0 tx=0).stop 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 msgr2=0x7f066410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f064c009150 tx=0x7f064c009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 shutdown_connections 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0650069210 0x7f065006b6c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640fce60 0x7f066410d920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:52.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 --2- 172.21.15.63:0/3692891269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06640fdfd0 0x7f066410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:52.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 >> 172.21.15.63:0/3692891269 conn(0x7f06640f8540 msgr2=0x7f06640f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:52.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 shutdown_connections 2024-04-24T22:38:52.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:52.707+0000 7f066b0d4700 1 -- 172.21.15.63:0/3692891269 wait complete. 2024-04-24T22:38:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:52 smithi192 ceph-mon[33991]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:53.278 DEBUG:teuthology.orchestra.run.smithi063:osd.0> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T22:38:53.281 INFO:tasks.cephadm:Deploying osd.1 on smithi063 with /dev/vg_nvme/lv_3... 2024-04-24T22:38:53.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T22:38:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:53 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:53 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:53 smithi192 ceph-mon[33991]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:38:54.012 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:38:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:53 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:53 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:53 smithi063 ceph-mon[26912]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:38:54.949 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-mon[26912]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:54.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-mon[26912]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:38:54.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-mon[26912]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T22:38:54.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:54.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-mon[26912]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:38:54.950 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[45208]: 2024-04-24T22:38:54.690+0000 7fd69bb35700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T22:38:54.950 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:38:54 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[45208]: 2024-04-24T22:38:54.695+0000 7fd69392a700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:38:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:54 smithi192 ceph-mon[33991]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:54 smithi192 ceph-mon[33991]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:38:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:54 smithi192 ceph-mon[33991]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T22:38:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:54 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:54 smithi192 ceph-mon[33991]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:38:56.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:55 smithi063 ceph-mon[26912]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:38:56.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:55 smithi063 ceph-mon[26912]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T22:38:56.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:55 smithi192 ceph-mon[33991]: from='osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:38:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:55 smithi192 ceph-mon[33991]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T22:38:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T22:38:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T22:38:56.467 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-04-24T22:38:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-04-24T22:38:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125269 s, 837 MB/s 2024-04-24T22:38:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] boot 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: osd.0 [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] boot 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T22:38:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:38:57.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:38:57.454 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-04-24T22:38:57.731 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:38:57.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:57 smithi063 ceph-mon[26912]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T22:38:57.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:57.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:57 smithi192 ceph-mon[33991]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T22:38:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:57 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:57 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:38:58.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:38:58 smithi063 ceph-mon[26912]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:38:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:38:58 smithi192 ceph-mon[33991]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- 172.21.15.63:0/3274668502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040662c0 msgr2=0x7f92040ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 --2- 172.21.15.63:0/3274668502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040662c0 0x7f92040ff9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9200004940 tx=0x7f9200009e30 comp rx=0 tx=0).stop 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- 172.21.15.63:0/3274668502 shutdown_connections 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 --2- 172.21.15.63:0/3274668502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040ffee0 0x7f9204102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 --2- 172.21.15.63:0/3274668502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040662c0 0x7f92040ff9a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- 172.21.15.63:0/3274668502 >> 172.21.15.63:0/3274668502 conn(0x7f92040f7430 msgr2=0x7f92040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- 172.21.15.63:0/3274668502 shutdown_connections 2024-04-24T22:38:59.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- 172.21.15.63:0/3274668502 wait complete. 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 Processor -- start 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.837+0000 7f920c27b700 1 -- start start 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040662c0 0x7f920410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204110760 con 0x7f92040ffee0 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204110fb0 con 0x7f92040662c0 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47198/0 (socket says 172.21.15.63:47198) 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 -- 172.21.15.63:0/2580323973 learned_addr learned my addr 172.21.15.63:0/2580323973 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920a017700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040662c0 0x7f920410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 -- 172.21.15.63:0/2580323973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040662c0 msgr2=0x7f920410fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040662c0 0x7f920410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 -- 172.21.15.63:0/2580323973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200005040 con 0x7f92040ffee0 2024-04-24T22:38:59.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f9209816700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c8c0 tx=0x7f91f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:59.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f400a980 con 0x7f92040ffee0 2024-04-24T22:38:59.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204111290 con 0x7f92040ffee0 2024-04-24T22:38:59.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91f400ce60 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f40115e0 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.838+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041117d0 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.839+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920405f5c0 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.840+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f91f4012400 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.840+0000 7f91faffd700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 0x7f91f006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.840+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f91f40862e0 con 0x7f92040ffee0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.840+0000 7f920a017700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 0x7f91f006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:38:59.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.841+0000 7f920a017700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 0x7f91f006b6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f920000b970 tx=0x7f9200009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:38:59.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:38:59.845+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f91f4056fc0 con 0x7f92040ffee0 2024-04-24T22:39:00.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:00.053+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9204111b40 con 0x7f91f0069210 2024-04-24T22:39:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:00 smithi063 ceph-mon[26912]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:00 smithi192 ceph-mon[33991]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:01.974 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:01 smithi063 ceph-mon[26912]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:01 smithi192 ceph-mon[33991]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:03.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:03.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:03.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:03.981 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi063 2024-04-24T22:39:03.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi063 to 12864M 2024-04-24T22:39:03.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4231096218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0"}]: dispatch 2024-04-24T22:39:03.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4231096218' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0"}]': finished 2024-04-24T22:39:03.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T22:39:03.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:03 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi063 2024-04-24T22:39:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi063 to 12864M 2024-04-24T22:39:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4231096218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0"}]: dispatch 2024-04-24T22:39:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4231096218' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0"}]': finished 2024-04-24T22:39:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T22:39:04.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:03 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:04 smithi063 ceph-mon[26912]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:04 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1241277346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:04 smithi192 ceph-mon[33991]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:04 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1241277346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:06.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:07.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:07.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:07.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:08 smithi063 ceph-mon[26912]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:08 smithi192 ceph-mon[33991]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:11.007 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:10 smithi063 ceph-mon[26912]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:10 smithi192 ceph-mon[33991]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:39:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:12.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:39:12.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:12.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:12.739 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:12 smithi063 ceph-mon[26912]: Deploying daemon osd.1 on smithi063 2024-04-24T22:39:13.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:12 smithi063 ceph-mon[26912]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:12 smithi192 ceph-mon[33991]: Deploying daemon osd.1 on smithi063 2024-04-24T22:39:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:12 smithi192 ceph-mon[33991]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:15.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:14 smithi063 ceph-mon[26912]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:14 smithi192 ceph-mon[33991]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:16 smithi192 ceph-mon[33991]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:16.965 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:16 smithi063 ceph-mon[26912]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:16.965 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:16.965 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:16.965 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:18.792 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:18 smithi063 ceph-mon[26912]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:18.793 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:18.793 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:18 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:18 smithi192 ceph-mon[33991]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:18 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:20.058 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 1 on host 'smithi063' 2024-04-24T22:39:20.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.055+0000 7f91faffd700 1 -- 172.21.15.63:0/2580323973 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9204111b40 con 0x7f91f0069210 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 msgr2=0x7f91f006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 0x7f91f006b6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f920000b970 tx=0x7f9200009e30 comp rx=0 tx=0).stop 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 msgr2=0x7f9204110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c8c0 tx=0x7f91f40044c0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 shutdown_connections 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f91f0069210 0x7f91f006b6c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92040662c0 0x7f920410fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 --2- 172.21.15.63:0/2580323973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92040ffee0 0x7f9204110160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 >> 172.21.15.63:0/2580323973 conn(0x7f92040f7430 msgr2=0x7f92040fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 shutdown_connections 2024-04-24T22:39:20.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:20.058+0000 7f920c27b700 1 -- 172.21.15.63:0/2580323973 wait complete. 2024-04-24T22:39:20.989 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:20 smithi063 ceph-mon[26912]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:20.989 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:20 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:20.989 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:20 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:20.989 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:20 smithi063 ceph-mon[26912]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:39:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:20 smithi192 ceph-mon[33991]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:20 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:21.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:20 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:21.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:20 smithi192 ceph-mon[33991]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:39:21.317 DEBUG:teuthology.orchestra.run.smithi063:osd.1> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T22:39:21.320 INFO:tasks.cephadm:Deploying osd.2 on smithi063 with /dev/vg_nvme/lv_2... 2024-04-24T22:39:21.320 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T22:39:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-mon[26912]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:39:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-mon[26912]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T22:39:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-mon[26912]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:39:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-mon[26912]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:22.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[52914]: 2024-04-24T22:39:22.057+0000 7f0ff5faf700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T22:39:22.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:22 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[52914]: 2024-04-24T22:39:22.062+0000 7f0ff05a9700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:22 smithi192 ceph-mon[33991]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:39:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:22 smithi192 ceph-mon[33991]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T22:39:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:22 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:22 smithi192 ceph-mon[33991]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:39:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:22 smithi192 ceph-mon[33991]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:22.708 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: from='osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:39:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T22:39:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] boot 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:24.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:24 smithi063 ceph-mon[26912]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:24.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:39:24.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:39:24.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:24.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: osd.1 [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] boot 2024-04-24T22:39:24.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T22:39:24.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:39:24.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:24 smithi192 ceph-mon[33991]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113796 s, 921 MB/s 2024-04-24T22:39:24.652 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-04-24T22:39:25.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:25 smithi192 ceph-mon[33991]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T22:39:25.487 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:25 smithi063 ceph-mon[26912]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T22:39:25.919 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-04-24T22:39:26.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:26 smithi063 ceph-mon[26912]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:26.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:26 smithi192 ceph-mon[33991]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:39:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 -- 172.21.15.63:0/2401596089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 msgr2=0x7fe6740ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2401596089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe6740ffa30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe664009ce0 tx=0x7fe664004d10 comp rx=0 tx=0).stop 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 -- 172.21.15.63:0/2401596089 shutdown_connections 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2401596089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 0x7fe674102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2401596089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe6740ffa30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 -- 172.21.15.63:0/2401596089 >> 172.21.15.63:0/2401596089 conn(0x7fe6740f74c0 msgr2=0x7fe6740f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:28.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 -- 172.21.15.63:0/2401596089 shutdown_connections 2024-04-24T22:39:28.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.643+0000 7fe678ee6700 1 -- 172.21.15.63:0/2401596089 wait complete. 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 Processor -- start 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 -- start start 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 0x7fe67410e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67410e800 con 0x7fe674066b50 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe678ee6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67410ef50 con 0x7fe6740fff70 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55382/0 (socket says 172.21.15.63:55382) 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 -- 172.21.15.63:0/2838843340 learned_addr learned my addr 172.21.15.63:0/2838843340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe671d9b700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 0x7fe67410e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 -- 172.21.15.63:0/2838843340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 msgr2=0x7fe67410e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 0x7fe67410e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 -- 172.21.15.63:0/2838843340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009940 con 0x7fe674066b50 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.644+0000 7fe67259c700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe664009380 tx=0x7fe664004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.645+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66400e040 con 0x7fe674066b50 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.645+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6640128e0 con 0x7fe674066b50 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.645+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67410f1d0 con 0x7fe674066b50 2024-04-24T22:39:28.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.645+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66401c9f0 con 0x7fe674066b50 2024-04-24T22:39:28.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.645+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67410f790 con 0x7fe674066b50 2024-04-24T22:39:28.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.646+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67405f5c0 con 0x7fe674066b50 2024-04-24T22:39:28.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.646+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fe664012a50 con 0x7fe674066b50 2024-04-24T22:39:28.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.646+0000 7fe66affd700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 0x7fe66006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:28.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.647+0000 7fe671d9b700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 0x7fe66006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:28.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.647+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fe664088150 con 0x7fe674066b50 2024-04-24T22:39:28.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.647+0000 7fe671d9b700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 0x7fe66006b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:28.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.651+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe664058b60 con 0x7fe674066b50 2024-04-24T22:39:28.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:28.864+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe67410fa70 con 0x7fe660069210 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi063 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi063 to 6432M 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.004 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi063 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi063 to 6432M 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:29.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:29.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:29 smithi063 ceph-mon[26912]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:29.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:29.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:29 smithi192 ceph-mon[33991]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:30.935 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:30 smithi063 ceph-mon[26912]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:30 smithi192 ceph-mon[33991]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:32.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:32 smithi063 ceph-mon[26912]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:33.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:32 smithi192 ceph-mon[33991]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:33 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3435692126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8955baa8-680f-44bd-86b8-7ac429ed953b"}]: dispatch 2024-04-24T22:39:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:33 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3435692126' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8955baa8-680f-44bd-86b8-7ac429ed953b"}]': finished 2024-04-24T22:39:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:33 smithi063 ceph-mon[26912]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T22:39:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:33 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:33 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/918578068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:33 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3435692126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8955baa8-680f-44bd-86b8-7ac429ed953b"}]: dispatch 2024-04-24T22:39:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:33 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3435692126' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8955baa8-680f-44bd-86b8-7ac429ed953b"}]': finished 2024-04-24T22:39:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:33 smithi192 ceph-mon[33991]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T22:39:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:33 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:33 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/918578068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:35.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:35.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:35.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:35.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:35.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:35.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:35.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:35.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:36 smithi192 ceph-mon[33991]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:36 smithi063 ceph-mon[26912]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:39.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:38 smithi063 ceph-mon[26912]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:39.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:38 smithi192 ceph-mon[33991]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:40 smithi063 ceph-mon[26912]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:39:41.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:40 smithi192 ceph-mon[33991]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:39:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:42.014 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:41 smithi063 ceph-mon[26912]: Deploying daemon osd.2 on smithi063 2024-04-24T22:39:42.014 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:41 smithi192 ceph-mon[33991]: Deploying daemon osd.2 on smithi063 2024-04-24T22:39:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:42.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:42 smithi063 ceph-mon[26912]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:42 smithi192 ceph-mon[33991]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:45.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:44 smithi063 ceph-mon[26912]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:44 smithi192 ceph-mon[33991]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:46 smithi192 ceph-mon[33991]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:47.005 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:47.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:47.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:46 smithi063 ceph-mon[26912]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:47.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:48.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.622+0000 7fe66affd700 1 -- 172.21.15.63:0/2838843340 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe67410fa70 con 0x7fe660069210 2024-04-24T22:39:48.625 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 2 on host 'smithi063' 2024-04-24T22:39:48.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 msgr2=0x7fe66006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 0x7fe66006b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-04-24T22:39:48.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 msgr2=0x7fe67410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe664009380 tx=0x7fe664004d10 comp rx=0 tx=0).stop 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 shutdown_connections 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.624+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe660069210 0x7fe66006b6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.625+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674066b50 0x7fe67410dc30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.625+0000 7fe678ee6700 1 --2- 172.21.15.63:0/2838843340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6740fff70 0x7fe67410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.625+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 >> 172.21.15.63:0/2838843340 conn(0x7fe6740f74c0 msgr2=0x7fe6740f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.625+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 shutdown_connections 2024-04-24T22:39:48.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:39:48.625+0000 7fe678ee6700 1 -- 172.21.15.63:0/2838843340 wait complete. 2024-04-24T22:39:48.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:48 smithi192 ceph-mon[33991]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:48.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:48 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:48.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:48 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:48.994 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:48 smithi063 ceph-mon[26912]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:48.994 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:48.994 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:49.346 DEBUG:teuthology.orchestra.run.smithi063:osd.2> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T22:39:49.349 INFO:tasks.cephadm:Deploying osd.3 on smithi192 with /dev/vg_nvme/lv_4... 2024-04-24T22:39:49.349 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T22:39:49.608 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:39:49.643 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:49 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:39:49.510+0000 7efd2b902540 -1 osd.2 0 log_to_monitors true 2024-04-24T22:39:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:49 smithi192 ceph-mon[33991]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:39:49.938 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:49 smithi063 ceph-mon[26912]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:39:50.911 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:50 smithi192 ceph-mon[33991]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:50.911 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:50 smithi192 ceph-mon[33991]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:39:50.911 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:50 smithi192 ceph-mon[33991]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T22:39:50.912 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:50.912 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:50 smithi192 ceph-mon[33991]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:39:51.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-mon[26912]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:51.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-mon[26912]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:39:51.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-mon[26912]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T22:39:51.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:51.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-mon[26912]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:39:51.045 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:39:50.648+0000 7efd238d7700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T22:39:51.045 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:50 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:39:50.653+0000 7efd1cecf700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:51.516 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T22:39:51.516 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T22:39:51.517 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-24T22:39:51.517 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-24T22:39:51.518 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011376 s, 922 MB/s 2024-04-24T22:39:51.518 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-24T22:39:51.837 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:51 smithi192 ceph-mon[33991]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:39:51.838 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:51 smithi192 ceph-mon[33991]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T22:39:51.838 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:51 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:51.838 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:51 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:51.940 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:51 smithi063 ceph-mon[26912]: from='osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-04-24T22:39:51.941 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:51 smithi063 ceph-mon[26912]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T22:39:51.941 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:51 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:51.941 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:51 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:52.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-04-24T22:39:52.445 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:39:52.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:39:52.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] boot 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: osd.2 [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] boot 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:39:52.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:52.978 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 -- 172.21.15.192:0/4135583816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fbcb0 msgr2=0x7fee3c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 --2- 172.21.15.192:0/4135583816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0fc0c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fee2c004940 tx=0x7fee2c009e30 comp rx=0 tx=0).stop 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 -- 172.21.15.192:0/4135583816 shutdown_connections 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 --2- 172.21.15.192:0/4135583816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 --2- 172.21.15.192:0/4135583816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0fc0c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 -- 172.21.15.192:0/4135583816 >> 172.21.15.192:0/4135583816 conn(0x7fee3c0f7430 msgr2=0x7fee3c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 -- 172.21.15.192:0/4135583816 shutdown_connections 2024-04-24T22:39:53.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.822+0000 7fee40de4700 1 -- 172.21.15.192:0/4135583816 wait complete. 2024-04-24T22:39:53.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 Processor -- start 2024-04-24T22:39:53.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 -- start start 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0fe850 con 0x7fee3c0fbcb0 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee40de4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0fe9c0 con 0x7fee3c0fcf60 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44344/0 (socket says 172.21.15.192:44344) 2024-04-24T22:39:53.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 -- 172.21.15.192:0/1300790560 learned_addr learned my addr 172.21.15.192:0/1300790560 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:39:53.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee3a59c700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:53.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 -- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fbcb0 msgr2=0x7fee3c0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:53.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:53.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 -- 172.21.15.192:0/1300790560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c005040 con 0x7fee3c0fcf60 2024-04-24T22:39:53.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.823+0000 7fee39d9b700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fee24002a50 tx=0x7fee240046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:53.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee24007cc0 con 0x7fee3c0fcf60 2024-04-24T22:39:53.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2400f040 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c0feca0 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2400da30 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c115a20 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.824+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c101af0 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.825+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fee2400db90 con 0x7fee3c0fcf60 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.825+0000 7fee32ffd700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 0x7fee2806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:53.834 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.826+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2842+0+0 (secure 0 0 0) 0x7fee24086310 con 0x7fee3c0fcf60 2024-04-24T22:39:53.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.826+0000 7fee3a59c700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 0x7fee2806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:53.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.826+0000 7fee3a59c700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 0x7fee2806b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:53.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:53.830+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fee24053160 con 0x7fee3c0fcf60 2024-04-24T22:39:54.025 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:54.024+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fee3c05e3a0 con 0x7fee28069210 2024-04-24T22:39:54.667 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T22:39:54.668 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: pgmap v74: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:54.668 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-24T22:39:54.668 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:54.668 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:54.668 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:54 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T22:39:54.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: pgmap v74: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:54.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-24T22:39:54.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:39:54.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:39:54.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:54 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:55.438 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:55 smithi063 ceph-mon[26912]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:55.438 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-24T22:39:55.438 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:55 smithi063 ceph-mon[26912]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T22:39:55.438 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:39:55.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:55 smithi192 ceph-mon[33991]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:39:55.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-24T22:39:55.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:55 smithi192 ceph-mon[33991]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T22:39:55.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T22:39:56.533 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:39:56.533 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T22:39:56.533 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi063 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi063 to 4288M 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:56.534 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:56.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:56.645 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T22:39:56.645 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi063 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi063 to 4288M 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:39:56.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:39:56.647 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:56.647 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:56.647 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:39:56.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:56 smithi063 sudo[64877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:39:56.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:56 smithi063 sudo[64877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:56.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:56 smithi063 sudo[64877]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:56.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:56 smithi063 sudo[64877]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:57.276 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:39:57.276 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:57.276 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:57.276 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64963]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:57.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:39:57.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:57.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64990]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:57.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[64990]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/4204629327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]: dispatch 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]: dispatch 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]': finished 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1475789608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:57.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:57 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/4204629327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]: dispatch 2024-04-24T22:39:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]: dispatch 2024-04-24T22:39:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36de3a3c-de49-41f6-9ef3-d999b18c99f7"}]': finished 2024-04-24T22:39:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T22:39:57.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:39:57.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:39:57.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1475789608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:39:57.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:57 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:39:57.979 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[65064]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:39:57.979 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[65064]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:57.980 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[65064]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:57.980 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:39:57 smithi063 sudo[65064]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:58.374 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:39:58.373+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fee24053530 con 0x7fee3c0fcf60 2024-04-24T22:39:58.656 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:58 smithi063 ceph-mon[26912]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:58.656 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:39:58.656 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:58.656 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65113]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:58.656 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65113]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:58.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:58 smithi192 ceph-mon[33991]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:39:59.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65145]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:39:59.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:59.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65145]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:59.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:39:58 smithi063 sudo[65145]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:39:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:39:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:39:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65219]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:39:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65219]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:59 smithi192 ceph-mon[33991]: mgrmap e19: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:39:59 smithi192 ceph-mon[33991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38519]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38519]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38519]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:40:00.031 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38519]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 sudo[65263]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 ceph-mon[26912]: mgrmap e19: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:40:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:39:59 smithi063 ceph-mon[26912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:40:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T22:40:00.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T22:40:00.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T22:40:00.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 sudo[38618]: pam_unix(sudo:session): session closed for user root 2024-04-24T22:40:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:40:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:40:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: overall HEALTH_OK 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: overall HEALTH_OK 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:01.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:01.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:02 smithi063 ceph-mon[26912]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:03.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:02 smithi192 ceph-mon[33991]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:03 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:40:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:03 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:03 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:40:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:03 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:04.977 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:04 smithi192 ceph-mon[33991]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:04.977 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:04 smithi192 ceph-mon[33991]: Deploying daemon osd.3 on smithi192 2024-04-24T22:40:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:04 smithi063 ceph-mon[26912]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:04 smithi063 ceph-mon[26912]: Deploying daemon osd.3 on smithi192 2024-04-24T22:40:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:06 smithi192 ceph-mon[33991]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T22:40:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:06 smithi063 ceph-mon[26912]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T22:40:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:08 smithi192 ceph-mon[33991]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-24T22:40:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:08 smithi063 ceph-mon[26912]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-24T22:40:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:11.002 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:10 smithi192 ceph-mon[33991]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:40:11.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:10 smithi063 ceph-mon[26912]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:40:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:12.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.135 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:12.135 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.135 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:12.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:12.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:12.485 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 3 on host 'smithi192' 2024-04-24T22:40:12.486 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.483+0000 7fee32ffd700 1 -- 172.21.15.192:0/1300790560 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fee3c05e3a0 con 0x7fee28069210 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 msgr2=0x7fee2806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 0x7fee2806b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c009e30 comp rx=0 tx=0).stop 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 msgr2=0x7fee3c0fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fee24002a50 tx=0x7fee240046b0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 shutdown_connections 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fee28069210 0x7fee2806b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee3c0fbcb0 0x7fee3c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 --2- 172.21.15.192:0/1300790560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee3c0fcf60 0x7fee3c0fe2a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.485+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 >> 172.21.15.192:0/1300790560 conn(0x7fee3c0f7430 msgr2=0x7fee3c065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.486+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 shutdown_connections 2024-04-24T22:40:12.488 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:12.486+0000 7fee40de4700 1 -- 172.21.15.192:0/1300790560 wait complete. 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: from='osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:13.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:12 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: from='osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:13.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:12 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:13.105 DEBUG:teuthology.orchestra.run.smithi192:osd.3> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T22:40:13.111 INFO:tasks.cephadm:Deploying osd.4 on smithi192 with /dev/vg_nvme/lv_3... 2024-04-24T22:40:13.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T22:40:14.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-mon[33991]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:40:14.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-mon[33991]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T22:40:14.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:14.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-mon[33991]: from='osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:14.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-mon[33991]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:14.179 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[39433]: 2024-04-24T22:40:13.810+0000 7fb97ad02700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T22:40:14.179 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:40:13 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[39433]: 2024-04-24T22:40:13.816+0000 7fb9732f8700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:40:14.207 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:13 smithi063 ceph-mon[26912]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:40:14.208 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:13 smithi063 ceph-mon[26912]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T22:40:14.208 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:13 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:14.208 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:13 smithi063 ceph-mon[26912]: from='osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:14.208 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:13 smithi063 ceph-mon[26912]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:14.756 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:40:15.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:14 smithi192 ceph-mon[33991]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:40:15.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:14 smithi192 ceph-mon[33991]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:40:15.136 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:14 smithi192 ceph-mon[33991]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T22:40:15.137 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:15.137 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:14 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:15.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:14 smithi063 ceph-mon[26912]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T22:40:15.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:14 smithi063 ceph-mon[26912]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:40:15.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:14 smithi063 ceph-mon[26912]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T22:40:15.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:15.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:14 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:16.076 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:40:16.076 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:40:16.076 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:16.077 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] boot 2024-04-24T22:40:16.077 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T22:40:16.077 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:15 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:40:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:40:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: osd.3 [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] boot 2024-04-24T22:40:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T22:40:16.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:15 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:16 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T22:40:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T22:40:17.231 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-24T22:40:17.231 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-24T22:40:17.231 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012473 s, 841 MB/s 2024-04-24T22:40:17.231 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-24T22:40:17.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T22:40:17.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T22:40:17.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:17.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:17.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:17.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:17.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:17.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:17.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:16 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:17.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-04-24T22:40:18.084 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:40:18.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:17 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi192 2024-04-24T22:40:18.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:17 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi192 to 15936M 2024-04-24T22:40:18.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:17 smithi192 ceph-mon[33991]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T22:40:18.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:17 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi192 2024-04-24T22:40:18.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:17 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi192 to 15936M 2024-04-24T22:40:18.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:17 smithi063 ceph-mon[26912]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T22:40:19.140 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:18 smithi192 ceph-mon[33991]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:19.140 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:18 smithi192 ceph-mon[33991]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T22:40:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:18 smithi063 ceph-mon[26912]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:18 smithi063 ceph-mon[26912]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 -- 172.21.15.192:0/421379484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fdfd0 msgr2=0x7efd780fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 --2- 172.21.15.192:0/421379484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fdfd0 0x7efd780fe440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd68004940 tx=0x7efd68009e30 comp rx=0 tx=0).stop 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 -- 172.21.15.192:0/421379484 shutdown_connections 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 --2- 172.21.15.192:0/421379484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fdfd0 0x7efd780fe440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 --2- 172.21.15.192:0/421379484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fce60 0x7efd780fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 -- 172.21.15.192:0/421379484 >> 172.21.15.192:0/421379484 conn(0x7efd780f8540 msgr2=0x7efd780fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 -- 172.21.15.192:0/421379484 shutdown_connections 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.624+0000 7efd7d52b700 1 -- 172.21.15.192:0/421379484 wait complete. 2024-04-24T22:40:19.628 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 Processor -- start 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 -- start start 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 0x7efd780702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd78072200 con 0x7efd780fdfd0 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd7d52b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780707f0 con 0x7efd780fce60 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54436/0 (socket says 172.21.15.192:54436) 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 -- 172.21.15.192:0/2567794173 learned_addr learned my addr 172.21.15.192:0/2567794173 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd767fc700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 0x7efd780702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 -- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 msgr2=0x7efd780702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 0x7efd780702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd76ffd700 1 -- 172.21.15.192:0/2567794173 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68005040 con 0x7efd780fce60 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.625+0000 7efd767fc700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 0x7efd780702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd76ffd700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c830 tx=0x7efd60004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6000a760 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd6000ad90 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd78070aa0 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd600113a0 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.626+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd78071060 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.627+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7804c710 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.627+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efd60011610 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.627+0000 7efd6f7fe700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 0x7efd6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.627+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7efd60086ba0 con 0x7efd780fce60 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.628+0000 7efd767fc700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 0x7efd6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.631 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.628+0000 7efd767fc700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 0x7efd6406b750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd6800b970 tx=0x7efd68009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.634 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.632+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efd60057000 con 0x7efd780fce60 2024-04-24T22:40:19.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:19.829+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efd781023b0 con 0x7efd640692a0 2024-04-24T22:40:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:19 smithi192 ceph-mon[33991]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T22:40:20.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:19 smithi063 ceph-mon[26912]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T22:40:21.141 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:20 smithi192 ceph-mon[33991]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T22:40:21.142 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:20 smithi192 ceph-mon[33991]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:40:21.142 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:20 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:40:21.142 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:20 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:40:21.142 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:20 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:20 smithi063 ceph-mon[26912]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T22:40:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:20 smithi063 ceph-mon[26912]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:40:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:20 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:40:21.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:20 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:40:21.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:20 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:22.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T22:40:22.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/1754359182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]: dispatch 2024-04-24T22:40:22.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]: dispatch 2024-04-24T22:40:22.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]': finished 2024-04-24T22:40:22.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T22:40:22.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:22 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:23.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T22:40:23.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/1754359182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]: dispatch 2024-04-24T22:40:23.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]: dispatch 2024-04-24T22:40:23.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33041276-b462-4198-b31a-6c0360b1159d"}]': finished 2024-04-24T22:40:23.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T22:40:23.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:22 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:23 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3385944352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:40:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:23 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3385944352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:40:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:24 smithi192 ceph-mon[33991]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T22:40:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:24 smithi063 ceph-mon[26912]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T22:40:27.164 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:26 smithi192 ceph-mon[33991]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T22:40:27.164 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:26 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:26 smithi063 ceph-mon[26912]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T22:40:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:26 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:28 smithi192 ceph-mon[33991]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T22:40:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:28 smithi063 ceph-mon[26912]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T22:40:30.162 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:40:30.162 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:30.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:40:30.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:30 smithi192 ceph-mon[33991]: Deploying daemon osd.4 on smithi192 2024-04-24T22:40:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:30 smithi192 ceph-mon[33991]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:30 smithi063 ceph-mon[26912]: Deploying daemon osd.4 on smithi192 2024-04-24T22:40:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:30 smithi063 ceph-mon[26912]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:32 smithi192 ceph-mon[33991]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:33.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:32 smithi063 ceph-mon[26912]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:35.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:34 smithi192 ceph-mon[33991]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:35.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:35.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:35.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:34 smithi063 ceph-mon[26912]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:36 smithi192 ceph-mon[33991]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:37.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:36 smithi063 ceph-mon[26912]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.348+0000 7efd6f7fe700 1 -- 172.21.15.192:0/2567794173 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd781023b0 con 0x7efd640692a0 2024-04-24T22:40:38.351 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 4 on host 'smithi192' 2024-04-24T22:40:38.353 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 msgr2=0x7efd6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:38.353 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 0x7efd6406b750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd6800b970 tx=0x7efd68009e30 comp rx=0 tx=0).stop 2024-04-24T22:40:38.353 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 msgr2=0x7efd78071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:38.353 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c830 tx=0x7efd60004470 comp rx=0 tx=0).stop 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 shutdown_connections 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7efd640692a0 0x7efd6406b750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd780fce60 0x7efd78071c00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 --2- 172.21.15.192:0/2567794173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd780fdfd0 0x7efd780702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 >> 172.21.15.192:0/2567794173 conn(0x7efd780f8540 msgr2=0x7efd780f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 shutdown_connections 2024-04-24T22:40:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:38.351+0000 7efd7d52b700 1 -- 172.21.15.192:0/2567794173 wait complete. 2024-04-24T22:40:38.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:38 smithi192 ceph-mon[33991]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:38.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:38 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:38.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:38 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:38.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:38 smithi192 ceph-mon[33991]: from='osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:40:38.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:38 smithi192 ceph-mon[33991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:40:39.012 DEBUG:teuthology.orchestra.run.smithi192:osd.4> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T22:40:39.014 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_2... 2024-04-24T22:40:39.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T22:40:39.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:38 smithi063 ceph-mon[26912]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:39.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:38 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:39.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:38 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:39.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:38 smithi063 ceph-mon[26912]: from='osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:40:39.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:38 smithi063 ceph-mon[26912]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:40:39.320 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:40:40.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:40.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:40.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:40:40.356 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T22:40:40.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:40.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:40.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:40.357 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-mon[33991]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:40:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:40 smithi063 ceph-mon[26912]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:40.618 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[45845]: 2024-04-24T22:40:40.353+0000 7f3769302700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T22:40:40.618 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:40:40 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[45845]: 2024-04-24T22:40:40.360+0000 7f37630fb700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:40:41.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:41 smithi192 ceph-mon[33991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:40:41.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:41 smithi192 ceph-mon[33991]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T22:40:41.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:41.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0184268 s, 569 MB/s 2024-04-24T22:40:41.695 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-24T22:40:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:41 smithi063 ceph-mon[26912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:40:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:41 smithi063 ceph-mon[26912]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T22:40:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:42.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:40:42.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:40:42.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] boot 2024-04-24T22:40:42.361 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T22:40:42.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-04-24T22:40:42.616 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:42.616 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:42.616 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:42 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: osd.4 [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] boot 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T22:40:42.705 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:42 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:42.892 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:40:43.553 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:43 smithi192 ceph-mon[33991]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T22:40:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:43 smithi063 ceph-mon[26912]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T22:40:44.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:44 smithi192 ceph-mon[33991]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:44 smithi063 ceph-mon[26912]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:45.081 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 -- 172.21.15.192:0/3716226381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 msgr2=0x7fd8b80fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.081 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/3716226381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80fe4f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8004940 tx=0x7fd8a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:40:45.081 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 -- 172.21.15.192:0/3716226381 shutdown_connections 2024-04-24T22:40:45.081 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/3716226381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80fe4f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.081 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/3716226381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 0x7fd8b80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 -- 172.21.15.192:0/3716226381 >> 172.21.15.192:0/3716226381 conn(0x7fd8b80f8590 msgr2=0x7fd8b80fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 -- 172.21.15.192:0/3716226381 shutdown_connections 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.077+0000 7fd8c00e3700 1 -- 172.21.15.192:0/3716226381 wait complete. 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 Processor -- start 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 -- start start 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 0x7fd8b8071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8070840 con 0x7fd8b80fce80 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8c00e3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8070980 con 0x7fd8b80fe080 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:35344/0 (socket says 172.21.15.192:35344) 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 -- 172.21.15.192:0/2024574619 learned_addr learned my addr 172.21.15.192:0/2024574619 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bde7f700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 0x7fd8b8071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.082 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 -- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 msgr2=0x7fd8b8071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 0x7fd8b8071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 -- 172.21.15.192:0/2024574619 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8005040 con 0x7fd8b80fe080 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8bd67e700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800b3b0 tx=0x7fd8a8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a800e040 con 0x7fd8b80fe080 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.079+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b8070c00 con 0x7fd8b80fe080 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.078+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a800bcc0 con 0x7fd8b80fe080 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.079+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a801c630 con 0x7fd8b80fe080 2024-04-24T22:40:45.083 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.079+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8115a90 con 0x7fd8b80fe080 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.079+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b805f5c0 con 0x7fd8b80fe080 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.080+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd8a8013980 con 0x7fd8b80fe080 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.080+0000 7fd8ae7fc700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 0x7fd8a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.080+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fd8a8087ad0 con 0x7fd8b80fe080 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.081+0000 7fd8bde7f700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 0x7fd8a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:45.084 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.081+0000 7fd8bde7f700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 0x7fd8a406b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40041d0 tx=0x7fd8b4009040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:45.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.085+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8a8054270 con 0x7fd8b80fe080 2024-04-24T22:40:45.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:40:45.276+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b81023f0 con 0x7fd8a40692a0 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi192 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi192 to 7968M 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:45.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:40:45.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:45 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi192 2024-04-24T22:40:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi192 to 7968M 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:40:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:46.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:40:46.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:40:46.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T22:40:46.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T22:40:46.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:45 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:46.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:46 smithi192 ceph-mon[33991]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:40:46.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:46 smithi192 ceph-mon[33991]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:46 smithi063 ceph-mon[26912]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:40:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:46 smithi063 ceph-mon[26912]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/4035905071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]: dispatch 2024-04-24T22:40:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]: dispatch 2024-04-24T22:40:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]': finished 2024-04-24T22:40:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T22:40:49.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:40:49.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:48 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/3599357481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:40:49.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/4035905071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]: dispatch 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]: dispatch 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e17e5ed-ac3f-489f-a2fd-b399694502b3"}]': finished 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:40:49.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:48 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/3599357481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T22:40:51.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:50 smithi063 ceph-mon[26912]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:50 smithi192 ceph-mon[33991]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:52 smithi192 ceph-mon[33991]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:52 smithi063 ceph-mon[26912]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:54 smithi063 ceph-mon[26912]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:54 smithi192 ceph-mon[33991]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:55.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:40:55.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:55 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:55.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:55 smithi192 ceph-mon[33991]: Deploying daemon osd.5 on smithi192 2024-04-24T22:40:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:40:56.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:55 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:40:56.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:55 smithi063 ceph-mon[26912]: Deploying daemon osd.5 on smithi192 2024-04-24T22:40:56.733 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:56 smithi192 ceph-mon[33991]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:56 smithi063 ceph-mon[26912]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:40:59.019 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:40:58 smithi192 ceph-mon[33991]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:40:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:40:58 smithi063 ceph-mon[26912]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:01.087 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:00 smithi192 ceph-mon[33991]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:01.087 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:01.087 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:01.087 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:00 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:00 smithi063 ceph-mon[26912]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:00 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:02.792 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.790+0000 7fd8ae7fc700 1 -- 172.21.15.192:0/2024574619 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd8b81023f0 con 0x7fd8a40692a0 2024-04-24T22:41:02.793 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-04-24T22:41:02.794 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 msgr2=0x7fd8a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:02.794 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 0x7fd8a406b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40041d0 tx=0x7fd8b4009040 comp rx=0 tx=0).stop 2024-04-24T22:41:02.794 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 msgr2=0x7fd8b80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:02.794 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800b3b0 tx=0x7fd8a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 shutdown_connections 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd8a40692a0 0x7fd8a406b750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b80fce80 0x7fd8b8071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 --2- 172.21.15.192:0/2024574619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8b80fe080 0x7fd8b80702b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 >> 172.21.15.192:0/2024574619 conn(0x7fd8b80f8590 msgr2=0x7fd8b80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 shutdown_connections 2024-04-24T22:41:02.795 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:02.793+0000 7fd8c00e3700 1 -- 172.21.15.192:0/2024574619 wait complete. 2024-04-24T22:41:03.171 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:02 smithi192 ceph-mon[33991]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:03.171 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:03.171 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:02 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:02 smithi063 ceph-mon[26912]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:02 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:03.999 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:03 smithi192 ceph-mon[33991]: from='osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:41:03.999 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:03 smithi192 ceph-mon[33991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:41:04.026 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T22:41:04.028 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-24T22:41:04.028 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T22:41:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:03 smithi063 ceph-mon[26912]: from='osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:41:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:03 smithi063 ceph-mon[26912]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:41:04.310 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:05.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:05.153 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:05.154 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:05.155 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[52335]: 2024-04-24T22:41:04.840+0000 7f4f2bcca700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T22:41:05.155 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:41:04 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[52335]: 2024-04-24T22:41:04.849+0000 7f4f252c2700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:41:06.044 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:05 smithi063 ceph-mon[26912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:41:06.044 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:05 smithi063 ceph-mon[26912]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T22:41:06.044 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:05 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:06.044 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:05 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 -- 172.21.15.63:0/3165970235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 msgr2=0x7f365c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 --2- 172.21.15.63:0/3165970235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c0fd3d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3648004940 tx=0x7f3648009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 -- 172.21.15.63:0/3165970235 shutdown_connections 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 --2- 172.21.15.63:0/3165970235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c0fd3d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 --2- 172.21.15.63:0/3165970235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 0x7f365c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 -- 172.21.15.63:0/3165970235 >> 172.21.15.63:0/3165970235 conn(0x7f365c0f7430 msgr2=0x7f365c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 -- 172.21.15.63:0/3165970235 shutdown_connections 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.058+0000 7f3662f9b700 1 -- 172.21.15.63:0/3165970235 wait complete. 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 Processor -- start 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 -- start start 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 0x7f365c066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c112c50 con 0x7f365c0fcf60 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3662f9b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c112d90 con 0x7f365c0fbcb0 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f365bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f365bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36252/0 (socket says 172.21.15.63:36252) 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f365bfff700 1 -- 172.21.15.63:0/4221629481 learned_addr learned my addr 172.21.15.63:0/4221629481 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.059+0000 7f3660d37700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 0x7f365c066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f365bfff700 1 -- 172.21.15.63:0/4221629481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 msgr2=0x7f365c066c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f365bfff700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 0x7f365c066c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f365bfff700 1 -- 172.21.15.63:0/4221629481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648005040 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f365bfff700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f364800b3b0 tx=0x7f3648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364800e040 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3648004290 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c112f90 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364801c3f0 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.060+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c113520 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.061+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c04c710 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.062+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f364800bbf0 con 0x7f365c0fcf60 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.062+0000 7f36597fa700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 0x7f364c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:06.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.062+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f36480880c0 con 0x7f365c0fcf60 2024-04-24T22:41:06.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.062+0000 7f3660d37700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 0x7f364c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:06.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.063+0000 7f3660d37700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 0x7f364c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3650007fd0 tx=0x7f3650009040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.067+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3648058260 con 0x7f365c0fcf60 2024-04-24T22:41:06.170 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:05 smithi192 ceph-mon[33991]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-24T22:41:06.170 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:05 smithi192 ceph-mon[33991]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T22:41:06.170 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:05 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:06.170 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:05 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:06.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.263+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f365c1047a0 con 0x7f365c0fcf60 2024-04-24T22:41:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.263+0000 7f36597fa700 1 -- 172.21.15.63:0/4221629481 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f3648057df0 con 0x7f365c0fcf60 2024-04-24T22:41:06.266 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:06.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.265+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 msgr2=0x7f364c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.265+0000 7f3662f9b700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 0x7f364c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3650007fd0 tx=0x7f3650009040 comp rx=0 tx=0).stop 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 msgr2=0x7f365c112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f364800b3b0 tx=0x7f3648009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 shutdown_connections 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f364c0692a0 0x7f364c06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f365c0fbcb0 0x7f365c066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 --2- 172.21.15.63:0/4221629481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c0fcf60 0x7f365c112710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 >> 172.21.15.63:0/4221629481 conn(0x7f365c0f7430 msgr2=0x7f365c0f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 shutdown_connections 2024-04-24T22:41:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:06.266+0000 7f3662f9b700 1 -- 172.21.15.63:0/4221629481 wait complete. 2024-04-24T22:41:06.965 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1713998465,"num_in_osds":6,"osd_in_since":1713998447,"num_remapped_pgs":0} 2024-04-24T22:41:06.966 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:41:07.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: purged_snaps scrub starts 2024-04-24T22:41:07.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: purged_snaps scrub ok 2024-04-24T22:41:07.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:07.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:07.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] boot 2024-04-24T22:41:07.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T22:41:07.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:07.148 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:06 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4221629481' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:41:07.250 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:07.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: purged_snaps scrub starts 2024-04-24T22:41:07.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: purged_snaps scrub ok 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: osd.5 [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] boot 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:41:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:06 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4221629481' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T22:41:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T22:41:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:08.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:07 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T22:41:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:41:08.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:08.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:08.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:07 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:08.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.933+0000 7f9f5e258700 1 -- 172.21.15.63:0/3022854617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58102df0 msgr2=0x7f9f581031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.933+0000 7f9f5e258700 1 --2- 172.21.15.63:0/3022854617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58102df0 0x7f9f581031c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004940 tx=0x7f9f48009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 -- 172.21.15.63:0/3022854617 shutdown_connections 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 --2- 172.21.15.63:0/3022854617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58103790 0x7f9f580fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 --2- 172.21.15.63:0/3022854617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58102df0 0x7f9f581031c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 -- 172.21.15.63:0/3022854617 >> 172.21.15.63:0/3022854617 conn(0x7f9f580f8980 msgr2=0x7f9f580fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 -- 172.21.15.63:0/3022854617 shutdown_connections 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 -- 172.21.15.63:0/3022854617 wait complete. 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 Processor -- start 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.934+0000 7f9f5e258700 1 -- start start 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58102df0 0x7f9f58113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58114710 con 0x7f9f58103790 2024-04-24T22:41:08.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5810db20 con 0x7f9f58102df0 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55378/0 (socket says 172.21.15.63:55378) 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 -- 172.21.15.63:0/1237462952 learned_addr learned my addr 172.21.15.63:0/1237462952 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f577fe700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58102df0 0x7f9f58113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 -- 172.21.15.63:0/1237462952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58102df0 msgr2=0x7f9f58113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58102df0 0x7f9f58113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 -- 172.21.15.63:0/1237462952 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48005040 con 0x7f9f58103790 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f56ffd700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c002a50 tx=0x7f9f4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c007cc0 con 0x7f9f58103790 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4c00f040 con 0x7f9f58103790 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5810de00 con 0x7f9f58103790 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c00d9c0 con 0x7f9f58103790 2024-04-24T22:41:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.935+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5810e3c0 con 0x7f9f58103790 2024-04-24T22:41:08.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.936+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58100100 con 0x7f9f58103790 2024-04-24T22:41:08.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.937+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9f4c009730 con 0x7f9f58103790 2024-04-24T22:41:08.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.937+0000 7f9f3ffff700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 0x7f9f4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:08.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.937+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9f4c086c00 con 0x7f9f58103790 2024-04-24T22:41:08.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.937+0000 7f9f577fe700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 0x7f9f4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:08.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.938+0000 7f9f577fe700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 0x7f9f4006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004f90 tx=0x7f9f48009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:08.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:08.942+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9f4c08c020 con 0x7f9f58103790 2024-04-24T22:41:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: Detected new or changed devices on smithi192 2024-04-24T22:41:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi192 to 5312M 2024-04-24T22:41:09.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:09.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:09.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:09.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:08 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:09.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.137+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f5810f1d0 con 0x7f9f58103790 2024-04-24T22:41:09.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.138+0000 7f9f3ffff700 1 -- 172.21.15.63:0/1237462952 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f9f4c056c80 con 0x7f9f58103790 2024-04-24T22:41:09.140 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:09.140 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":40,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:41:06.842527+0000","last_up_change":"2024-04-24T22:41:05.837531+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6803","nonce":758545231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6805","nonce":758545231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6809","nonce":758545231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6807","nonce":758545231}]},"public_addr":"172.21.15.63:6803/758545231","cluster_addr":"172.21.15.63:6805/758545231","heartbeat_back_addr":"172.21.15.63:6809/758545231","heartbeat_front_addr":"172.21.15.63:6807/758545231","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6811","nonce":1039875891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6813","nonce":1039875891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6817","nonce":1039875891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6815","nonce":1039875891}]},"public_addr":"172.21.15.63:6811/1039875891","cluster_addr":"172.21.15.63:6813/1039875891","heartbeat_back_addr":"172.21.15.63:6817/1039875891","heartbeat_front_addr":"172.21.15.63:6815/1039875891","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6819","nonce":2431699450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6821","nonce":2431699450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6825","nonce":2431699450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6823","nonce":2431699450}]},"public_addr":"172.21.15.63:6819/2431699450","cluster_addr":"172.21.15.63:6821/2431699450","heartbeat_back_addr":"172.21.15.63:6825/2431699450","heartbeat_front_addr":"172.21.15.63:6823/2431699450","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6801","nonce":2771944170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6803","nonce":2771944170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6807","nonce":2771944170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6805","nonce":2771944170}]},"public_addr":"172.21.15.192:6801/2771944170","cluster_addr":"172.21.15.192:6803/2771944170","heartbeat_back_addr":"172.21.15.192:6807/2771944170","heartbeat_front_addr":"172.21.15.192:6805/2771944170","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6809","nonce":1725358058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6811","nonce":1725358058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6815","nonce":1725358058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6813","nonce":1725358058}]},"public_addr":"172.21.15.192:6809/1725358058","cluster_addr":"172.21.15.192:6811/1725358058","heartbeat_back_addr":"172.21.15.192:6815/1725358058","heartbeat_front_addr":"172.21.15.192:6813/1725358058","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6817","nonce":2882470257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6819","nonce":2882470257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6823","nonce":2882470257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6821","nonce":2882470257}]},"public_addr":"172.21.15.192:6817/2882470257","cluster_addr":"172.21.15.192:6819/2882470257","heartbeat_back_addr":"172.21.15.192:6823/2882470257","heartbeat_front_addr":"172.21.15.192:6821/2882470257","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 msgr2=0x7f9f4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 0x7f9f4006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004f90 tx=0x7f9f48009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 msgr2=0x7f9f581140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c002a50 tx=0x7f9f4c004600 comp rx=0 tx=0).stop 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 shutdown_connections 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9f400692a0 0x7f9f4006b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f58102df0 0x7f9f58113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 --2- 172.21.15.63:0/1237462952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f58103790 0x7f9f581140c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:09.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 >> 172.21.15.63:0/1237462952 conn(0x7f9f580f8980 msgr2=0x7f9f581073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:09.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 shutdown_connections 2024-04-24T22:41:09.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:09.140+0000 7f9f5e258700 1 -- 172.21.15.63:0/1237462952 wait complete. 2024-04-24T22:41:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi192 2024-04-24T22:41:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi192 to 5312M 2024-04-24T22:41:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:41:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:41:09.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:41:09.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:08 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:41:09.815 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:54.076844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-24T22:41:09.815 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T22:41:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:10.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:09 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1237462952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:10.207 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:09 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1237462952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:10 smithi192 ceph-mon[33991]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:11.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:10 smithi063 ceph-mon[26912]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 -- 172.21.15.63:0/1776815988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fde00 msgr2=0x7f4ad40fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/1776815988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fe1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4004940 tx=0x7f4ac4009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 -- 172.21.15.63:0/1776815988 shutdown_connections 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/1776815988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/1776815988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fe1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 -- 172.21.15.63:0/1776815988 >> 172.21.15.63:0/1776815988 conn(0x7f4ad40f7860 msgr2=0x7f4ad40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 -- 172.21.15.63:0/1776815988 shutdown_connections 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.718+0000 7f4ad9faf700 1 -- 172.21.15.63:0/1776815988 wait complete. 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 Processor -- start 2024-04-24T22:41:11.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 -- start start 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad40fdca0 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad9faf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad40fc740 con 0x7f4ad40fde00 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.719+0000 7f4ad2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55392/0 (socket says 172.21.15.63:55392) 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 -- 172.21.15.63:0/3350921377 learned_addr learned my addr 172.21.15.63:0/3350921377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad37fe700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 -- 172.21.15.63:0/3350921377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fde00 msgr2=0x7f4ad40fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 -- 172.21.15.63:0/3350921377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4005040 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad2ffd700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b790 tx=0x7f4ac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac800f770 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad40fca20 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac800fda0 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac800d310 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.720+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad411a210 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.721+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad4066ad0 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.722+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4ac8010420 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.722+0000 7f4abbfff700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 0x7f4abc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.722+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4ac8086d90 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.722+0000 7f4ad37fe700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 0x7f4abc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.723+0000 7f4ad37fe700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 0x7f4abc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009fd0 tx=0x7f4ac4009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.727+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4ac80534a0 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.920+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ad4109f40 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.920+0000 7f4abbfff700 1 -- 172.21.15.63:0/3350921377 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ac8056ac0 con 0x7f4ad40fe7a0 2024-04-24T22:41:11.922 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 msgr2=0x7f4abc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 0x7f4abc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009fd0 tx=0x7f4ac4009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 msgr2=0x7f4ad40fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b790 tx=0x7f4ac8009250 comp rx=0 tx=0).stop 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 shutdown_connections 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4abc0692a0 0x7f4abc06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ad40fde00 0x7f4ad40fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 --2- 172.21.15.63:0/3350921377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ad40fe7a0 0x7f4ad40fc200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 >> 172.21.15.63:0/3350921377 conn(0x7f4ad40f7860 msgr2=0x7f4ad40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 shutdown_connections 2024-04-24T22:41:11.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:11.922+0000 7f4ad9faf700 1 -- 172.21.15.63:0/3350921377 wait complete. 2024-04-24T22:41:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:12.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:12.709 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T22:41:12.709 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T22:41:12.976 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:13.058 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:12 smithi063 ceph-mon[26912]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:13.059 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:12 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3350921377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:13.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:12 smithi192 ceph-mon[33991]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:13.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:12 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3350921377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- 172.21.15.63:0/1676566186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 msgr2=0x7f8f100fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1676566186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f100fd410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f04004940 tx=0x7f8f04009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- 172.21.15.63:0/1676566186 shutdown_connections 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1676566186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f100fd410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1676566186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f100fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- 172.21.15.63:0/1676566186 >> 172.21.15.63:0/1676566186 conn(0x7f8f100f7430 msgr2=0x7f8f100f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- 172.21.15.63:0/1676566186 shutdown_connections 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- 172.21.15.63:0/1676566186 wait complete. 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 Processor -- start 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.513+0000 7f8f160ea700 1 -- start start 2024-04-24T22:41:14.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f1010d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1010e570 con 0x7f8f100fcfa0 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f10112f10 con 0x7f8f100fbcf0 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55410/0 (socket says 172.21.15.63:55410) 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 -- 172.21.15.63:0/1331860931 learned_addr learned my addr 172.21.15.63:0/1331860931 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:14.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 -- 172.21.15.63:0/1331860931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 msgr2=0x7f8f1010d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:14.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0f7fe700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f1010d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:14.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f1010d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 -- 172.21.15.63:0/1331860931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04005040 con 0x7f8f100fcfa0 2024-04-24T22:41:14.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0f7fe700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f1010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:14.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f0effd700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f04008d50 tx=0x7f8f04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:14.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0400e070 con 0x7f8f100fcfa0 2024-04-24T22:41:14.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f0400bbf0 con 0x7f8f100fcfa0 2024-04-24T22:41:14.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f10113110 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0401c400 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.514+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f10113620 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.516+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1004c710 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.516+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8f0400bd60 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.516+0000 7f8ef7fff700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 0x7f8ef806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.516+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8f040887c0 con 0x7f8f100fcfa0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.517+0000 7f8f0f7fe700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 0x7f8ef806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.517+0000 7f8f0f7fe700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 0x7f8ef806b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006fd0 tx=0x7f8f00008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:14.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.521+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8f040588c0 con 0x7f8f100fcfa0 2024-04-24T22:41:14.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.796+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8f10113a70 con 0x7f8f100fcfa0 2024-04-24T22:41:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.800+0000 7f8ef7fff700 1 -- 172.21.15.63:0/1331860931 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8f04058450 con 0x7f8f100fcfa0 2024-04-24T22:41:14.802 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-04-24T22:41:14.802 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQCKiilmx3eNLxAArJlNLY0xTrRWp+9EAQpWmQ== 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 msgr2=0x7f8ef806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 0x7f8ef806b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006fd0 tx=0x7f8f00008040 comp rx=0 tx=0).stop 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 msgr2=0x7f8f1010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f04008d50 tx=0x7f8f04009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 shutdown_connections 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8ef80692b0 0x7f8ef806b760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f100fbcf0 0x7f8f1010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 --2- 172.21.15.63:0/1331860931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f100fcfa0 0x7f8f1010dee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 >> 172.21.15.63:0/1331860931 conn(0x7f8f100f7430 msgr2=0x7f8f100f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 shutdown_connections 2024-04-24T22:41:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:14.802+0000 7f8f160ea700 1 -- 172.21.15.63:0/1331860931 wait complete. 2024-04-24T22:41:15.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:14 smithi063 ceph-mon[26912]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:15.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:14 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1331860931' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:15.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:14 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1331860931' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:41:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:14 smithi192 ceph-mon[33991]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:15.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:14 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1331860931' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:15.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:14 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1331860931' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:41:15.374 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:41:15.374 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T22:41:15.374 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T22:41:15.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T22:41:15.675 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-24T22:41:16.899 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:16 smithi192 ceph-mon[33991]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:17.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 -- 172.21.15.192:0/455638103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fcf70 msgr2=0x7fd7080fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 --2- 172.21.15.192:0/455638103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fcf70 0x7fd7080fd3e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4004940 tx=0x7fd6f4009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 -- 172.21.15.192:0/455638103 shutdown_connections 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 --2- 172.21.15.192:0/455638103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fcf70 0x7fd7080fd3e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 --2- 172.21.15.192:0/455638103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fbcc0 0x7fd7080fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 -- 172.21.15.192:0/455638103 >> 172.21.15.192:0/455638103 conn(0x7fd7080f7440 msgr2=0x7fd7080f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 -- 172.21.15.192:0/455638103 shutdown_connections 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.222+0000 7fd70f228700 1 -- 172.21.15.192:0/455638103 wait complete. 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 Processor -- start 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 -- start start 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fcf70 0x7fd708112700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708112c40 con 0x7fd7080fcf70 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70f228700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708112db0 con 0x7fd7080fbcc0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.223+0000 7fd70cfc4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54690/0 (socket says 172.21.15.192:54690) 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 -- 172.21.15.192:0/2288348942 learned_addr learned my addr 172.21.15.192:0/2288348942 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd707fff700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fcf70 0x7fd708112700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 -- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fcf70 msgr2=0x7fd708112700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fcf70 0x7fd708112700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 -- 172.21.15.192:0/2288348942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4005040 con 0x7fd7080fbcc0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70cfc4700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b790 tx=0x7fd6fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6fc00f770 con 0x7fd7080fbcc0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708113090 con 0x7fd7080fbcc0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6fc00fda0 con 0x7fd7080fbcc0 2024-04-24T22:41:17.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6fc00d310 con 0x7fd7080fbcc0 2024-04-24T22:41:17.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.224+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708113650 con 0x7fd7080fbcc0 2024-04-24T22:41:17.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.225+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70804c710 con 0x7fd7080fbcc0 2024-04-24T22:41:17.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.226+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd6fc00f8d0 con 0x7fd7080fbcc0 2024-04-24T22:41:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.226+0000 7fd7057fa700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 0x7fd6f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.226+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd6fc086ea0 con 0x7fd7080fbcc0 2024-04-24T22:41:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.226+0000 7fd707fff700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 0x7fd6f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.227+0000 7fd707fff700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 0x7fd6f806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400b3b0 tx=0x7fd6f4009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:17.232 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.230+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd6fc056fa0 con 0x7fd7080fbcc0 2024-04-24T22:41:17.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:16 smithi063 ceph-mon[26912]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:17.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.490+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd708104830 con 0x7fd7080fbcc0 2024-04-24T22:41:17.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.495+0000 7fd7057fa700 1 -- 172.21.15.192:0/2288348942 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd6fc020250 con 0x7fd7080fbcc0 2024-04-24T22:41:17.497 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-04-24T22:41:17.497 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCNiilmf3ZaHRAA91lU+0WbGjGvCYokH2G+qA== 2024-04-24T22:41:17.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 msgr2=0x7fd6f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 0x7fd6f806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400b3b0 tx=0x7fd6f4009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:17.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 msgr2=0x7fd708066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b790 tx=0x7fd6fc009250 comp rx=0 tx=0).stop 2024-04-24T22:41:17.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 shutdown_connections 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd6f80692a0 0x7fd6f806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7080fbcc0 0x7fd708066cd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 --2- 172.21.15.192:0/2288348942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7080fcf70 0x7fd708112700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 >> 172.21.15.192:0/2288348942 conn(0x7fd7080f7440 msgr2=0x7fd7080f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 shutdown_connections 2024-04-24T22:41:17.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:41:17.497+0000 7fd70f228700 1 -- 172.21.15.192:0/2288348942 wait complete. 2024-04-24T22:41:18.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:17 smithi192 ceph-mon[33991]: from='client.? 172.21.15.192:0/2288348942' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:18.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:17 smithi192 ceph-mon[33991]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:18.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:17 smithi192 ceph-mon[33991]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:41:18.073 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:41:18.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-24T22:41:18.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T22:41:18.109 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T22:41:18.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T22:41:18.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T22:41:18.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:17 smithi063 ceph-mon[26912]: from='client.? 172.21.15.192:0/2288348942' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:18.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:17 smithi063 ceph-mon[26912]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T22:41:18.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:17 smithi063 ceph-mon[26912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T22:41:18.386 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:18 smithi063 ceph-mon[26912]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:18 smithi192 ceph-mon[33991]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 -- 172.21.15.63:0/1468609475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 msgr2=0x7fb714103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/1468609475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb714103330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb708004940 tx=0x7fb708009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 -- 172.21.15.63:0/1468609475 shutdown_connections 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/1468609475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 0x7fb7140fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/1468609475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb714103330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 -- 172.21.15.63:0/1468609475 >> 172.21.15.63:0/1468609475 conn(0x7fb7140714d0 msgr2=0x7fb7140718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 -- 172.21.15.63:0/1468609475 shutdown_connections 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.023+0000 7fb71c5e0700 1 -- 172.21.15.63:0/1468609475 wait complete. 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 Processor -- start 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 -- start start 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 0x7fb7140fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140fe6e0 con 0x7fb714102f60 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71c5e0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140fd180 con 0x7fb7140fb520 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb719b7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb719b7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46224/0 (socket says 172.21.15.63:46224) 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb719b7b700 1 -- 172.21.15.63:0/530034068 learned_addr learned my addr 172.21.15.63:0/530034068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.024+0000 7fb71a37c700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 0x7fb7140fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb719b7b700 1 -- 172.21.15.63:0/530034068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 msgr2=0x7fb7140fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb719b7b700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 0x7fb7140fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb719b7b700 1 -- 172.21.15.63:0/530034068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708005040 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb719b7b700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb71000bbb0 tx=0x7fb710009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7100092e0 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb71000cbd0 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7140fd460 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb710007400 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.025+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71411a200 con 0x7fb714102f60 2024-04-24T22:41:20.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.026+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb714100250 con 0x7fb714102f60 2024-04-24T22:41:20.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.026+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb710012070 con 0x7fb714102f60 2024-04-24T22:41:20.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.027+0000 7fb706ffd700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 0x7fb70006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.027+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb7100869f0 con 0x7fb714102f60 2024-04-24T22:41:20.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.027+0000 7fb71a37c700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 0x7fb70006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.027+0000 7fb71a37c700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 0x7fb70006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb708004f90 tx=0x7fb708009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.031+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb710088e60 con 0x7fb714102f60 2024-04-24T22:41:20.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.284+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb7140fdbe0 con 0x7fb714102f60 2024-04-24T22:41:20.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.287+0000 7fb706ffd700 1 -- 172.21.15.63:0/530034068 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167176 (secure 0 0 0) 0x7fb710053060 con 0x7fb714102f60 2024-04-24T22:41:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 msgr2=0x7fb70006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 0x7fb70006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb708004f90 tx=0x7fb708009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 msgr2=0x7fb7140fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb71000bbb0 tx=0x7fb710009840 comp rx=0 tx=0).stop 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 shutdown_connections 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb7000692a0 0x7fb70006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7140fb520 0x7fb7140fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 --2- 172.21.15.63:0/530034068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb714102f60 0x7fb7140fcc40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 >> 172.21.15.63:0/530034068 conn(0x7fb7140714d0 msgr2=0x7fb7140fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 shutdown_connections 2024-04-24T22:41:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:20.289+0000 7fb71c5e0700 1 -- 172.21.15.63:0/530034068 wait complete. 2024-04-24T22:41:20.990 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"flags":0,"active_gid":14216,"active_name":"smithi063.upcnvd","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":622073725},{"type":"v1","addr":"172.21.15.63:6801","nonce":622073725}]},"active_addr":"172.21.15.63:6801/622073725","active_change":"2024-04-24T22:37:41.558719+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14236,"name":"smithi192.ctmwwc","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.63:8443/","prometheus":"http://172.21.15.63:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2342719260}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3056400683}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":4263661376}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1700011071}]}]} 2024-04-24T22:41:20.993 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T22:41:20.994 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T22:41:20.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:41:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:20 smithi192 ceph-mon[33991]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:21.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:20 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/530034068' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T22:41:21.298 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:20 smithi063 ceph-mon[26912]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:20 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/530034068' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T22:41:22.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 -- 172.21.15.63:0/4159881240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28381037c0 msgr2=0x7f28380fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 --2- 172.21.15.63:0/4159881240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28381037c0 0x7f28380fb3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2824004940 tx=0x7f2824009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 -- 172.21.15.63:0/4159881240 shutdown_connections 2024-04-24T22:41:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 --2- 172.21.15.63:0/4159881240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28381037c0 0x7f28380fb3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 --2- 172.21.15.63:0/4159881240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2838102e20 0x7f28381031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 -- 172.21.15.63:0/4159881240 >> 172.21.15.63:0/4159881240 conn(0x7f2838072380 msgr2=0x7f2838072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:22.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 -- 172.21.15.63:0/4159881240 shutdown_connections 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.810+0000 7f283eacf700 1 -- 172.21.15.63:0/4159881240 wait complete. 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 Processor -- start 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 -- start start 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28381037c0 0x7f2838113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28381146d0 con 0x7f2838102e20 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283eacf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283810dc20 con 0x7f28381037c0 2024-04-24T22:41:22.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283c86b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.811+0000 7f283c86b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46234/0 (socket says 172.21.15.63:46234) 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283c86b700 1 -- 172.21.15.63:0/2955921869 learned_addr learned my addr 172.21.15.63:0/2955921869 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283c86b700 1 -- 172.21.15.63:0/2955921869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28381037c0 msgr2=0x7f2838113ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283c86b700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28381037c0 0x7f2838113ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283c86b700 1 -- 172.21.15.63:0/2955921869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2824005040 con 0x7f2838102e20 2024-04-24T22:41:22.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283c86b700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f282c00b760 tx=0x7f282c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282c00f6f0 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f282c00fd20 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283810dea0 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282c011e40 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.812+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283810e3e0 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.813+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28380fe6e0 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.814+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f282c011670 con 0x7f2838102e20 2024-04-24T22:41:22.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.814+0000 7f28357fa700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 0x7f282806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:22.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.814+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f282c086ac0 con 0x7f2838102e20 2024-04-24T22:41:22.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.814+0000 7f2837fff700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 0x7f282806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:22.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.815+0000 7f2837fff700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 0x7f282806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f282400b3b0 tx=0x7f2824009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:22.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:22.819+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f282c056d10 con 0x7f2838102e20 2024-04-24T22:41:23.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.013+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2838109fc0 con 0x7f2838102e20 2024-04-24T22:41:23.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.014+0000 7f28357fa700 1 -- 172.21.15.63:0/2955921869 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f282c011400 con 0x7f2838102e20 2024-04-24T22:41:23.016 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:23.016 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":40,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:41:06.842527+0000","last_up_change":"2024-04-24T22:41:05.837531+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6803","nonce":758545231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6805","nonce":758545231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6809","nonce":758545231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6807","nonce":758545231}]},"public_addr":"172.21.15.63:6803/758545231","cluster_addr":"172.21.15.63:6805/758545231","heartbeat_back_addr":"172.21.15.63:6809/758545231","heartbeat_front_addr":"172.21.15.63:6807/758545231","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6811","nonce":1039875891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6813","nonce":1039875891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6817","nonce":1039875891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6815","nonce":1039875891}]},"public_addr":"172.21.15.63:6811/1039875891","cluster_addr":"172.21.15.63:6813/1039875891","heartbeat_back_addr":"172.21.15.63:6817/1039875891","heartbeat_front_addr":"172.21.15.63:6815/1039875891","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6819","nonce":2431699450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6821","nonce":2431699450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6825","nonce":2431699450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6823","nonce":2431699450}]},"public_addr":"172.21.15.63:6819/2431699450","cluster_addr":"172.21.15.63:6821/2431699450","heartbeat_back_addr":"172.21.15.63:6825/2431699450","heartbeat_front_addr":"172.21.15.63:6823/2431699450","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6801","nonce":2771944170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6803","nonce":2771944170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6807","nonce":2771944170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6805","nonce":2771944170}]},"public_addr":"172.21.15.192:6801/2771944170","cluster_addr":"172.21.15.192:6803/2771944170","heartbeat_back_addr":"172.21.15.192:6807/2771944170","heartbeat_front_addr":"172.21.15.192:6805/2771944170","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6809","nonce":1725358058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6811","nonce":1725358058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6815","nonce":1725358058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6813","nonce":1725358058}]},"public_addr":"172.21.15.192:6809/1725358058","cluster_addr":"172.21.15.192:6811/1725358058","heartbeat_back_addr":"172.21.15.192:6815/1725358058","heartbeat_front_addr":"172.21.15.192:6813/1725358058","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6817","nonce":2882470257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6819","nonce":2882470257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6823","nonce":2882470257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6821","nonce":2882470257}]},"public_addr":"172.21.15.192:6817/2882470257","cluster_addr":"172.21.15.192:6819/2882470257","heartbeat_back_addr":"172.21.15.192:6823/2882470257","heartbeat_front_addr":"172.21.15.192:6821/2882470257","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 msgr2=0x7f282806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 0x7f282806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f282400b3b0 tx=0x7f2824009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 msgr2=0x7f2838113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f282c00b760 tx=0x7f282c0091a0 comp rx=0 tx=0).stop 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 shutdown_connections 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f28280692a0 0x7f282806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2838102e20 0x7f2838113ab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:23.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 --2- 172.21.15.63:0/2955921869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28381037c0 0x7f2838113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:23.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.016+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 >> 172.21.15.63:0/2955921869 conn(0x7f2838072380 msgr2=0x7f28380f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:23.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.017+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 shutdown_connections 2024-04-24T22:41:23.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:23.017+0000 7f283eacf700 1 -- 172.21.15.63:0/2955921869 wait complete. 2024-04-24T22:41:23.029 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:22 smithi063 ceph-mon[26912]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:22 smithi192 ceph-mon[33991]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:23.684 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T22:41:23.684 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T22:41:23.948 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:23 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2955921869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:23 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2955921869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:25.160 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:24 smithi063 ceph-mon[26912]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:24 smithi192 ceph-mon[33991]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- 172.21.15.63:0/845389107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38103780 msgr2=0x7f9b380fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 --2- 172.21.15.63:0/845389107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38103780 0x7f9b380fcdf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b28004940 tx=0x7f9b28009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- 172.21.15.63:0/845389107 shutdown_connections 2024-04-24T22:41:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 --2- 172.21.15.63:0/845389107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38103780 0x7f9b380fcdf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 --2- 172.21.15.63:0/845389107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38102de0 0x7f9b381031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- 172.21.15.63:0/845389107 >> 172.21.15.63:0/845389107 conn(0x7f9b380f8990 msgr2=0x7f9b380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- 172.21.15.63:0/845389107 shutdown_connections 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- 172.21.15.63:0/845389107 wait complete. 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 Processor -- start 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.511+0000 7f9b3f663700 1 -- start start 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3f663700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3f663700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 0x7f9b38113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3f663700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381145a0 con 0x7f9b38102de0 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3f663700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3810db60 con 0x7f9b38103780 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46252/0 (socket says 172.21.15.63:46252) 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 -- 172.21.15.63:0/88636299 learned_addr learned my addr 172.21.15.63:0/88636299 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3cbfe700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 0x7f9b38113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 -- 172.21.15.63:0/88636299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 msgr2=0x7f9b38113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:25.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 0x7f9b38113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 -- 172.21.15.63:0/88636299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b28005040 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3cbfe700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 0x7f9b38113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3d3ff700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b790 tx=0x7f9b34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3400f770 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3810de40 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b3400fda0 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.512+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3400d310 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.513+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3810e400 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.514+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b381000f0 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.514+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9b34010420 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.514+0000 7f9b2dffb700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 0x7f9b2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.515+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9b34086e80 con 0x7f9b38102de0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.515+0000 7f9b3cbfe700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 0x7f9b2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.515+0000 7f9b3cbfe700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 0x7f9b2406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b3b0 tx=0x7f9b28009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:25.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.519+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9b34056f80 con 0x7f9b38102de0 2024-04-24T22:41:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.715+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b3810efd0 con 0x7f9b38102de0 2024-04-24T22:41:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.716+0000 7f9b2dffb700 1 -- 172.21.15.63:0/88636299 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f9b34056b10 con 0x7f9b38102de0 2024-04-24T22:41:25.718 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:25.718 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":40,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:41:06.842527+0000","last_up_change":"2024-04-24T22:41:05.837531+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6803","nonce":758545231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6805","nonce":758545231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6809","nonce":758545231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6807","nonce":758545231}]},"public_addr":"172.21.15.63:6803/758545231","cluster_addr":"172.21.15.63:6805/758545231","heartbeat_back_addr":"172.21.15.63:6809/758545231","heartbeat_front_addr":"172.21.15.63:6807/758545231","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6811","nonce":1039875891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6813","nonce":1039875891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6817","nonce":1039875891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6815","nonce":1039875891}]},"public_addr":"172.21.15.63:6811/1039875891","cluster_addr":"172.21.15.63:6813/1039875891","heartbeat_back_addr":"172.21.15.63:6817/1039875891","heartbeat_front_addr":"172.21.15.63:6815/1039875891","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6819","nonce":2431699450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6821","nonce":2431699450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6825","nonce":2431699450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6823","nonce":2431699450}]},"public_addr":"172.21.15.63:6819/2431699450","cluster_addr":"172.21.15.63:6821/2431699450","heartbeat_back_addr":"172.21.15.63:6825/2431699450","heartbeat_front_addr":"172.21.15.63:6823/2431699450","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6801","nonce":2771944170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6803","nonce":2771944170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6807","nonce":2771944170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6805","nonce":2771944170}]},"public_addr":"172.21.15.192:6801/2771944170","cluster_addr":"172.21.15.192:6803/2771944170","heartbeat_back_addr":"172.21.15.192:6807/2771944170","heartbeat_front_addr":"172.21.15.192:6805/2771944170","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6809","nonce":1725358058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6811","nonce":1725358058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6815","nonce":1725358058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6813","nonce":1725358058}]},"public_addr":"172.21.15.192:6809/1725358058","cluster_addr":"172.21.15.192:6811/1725358058","heartbeat_back_addr":"172.21.15.192:6815/1725358058","heartbeat_front_addr":"172.21.15.192:6813/1725358058","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6817","nonce":2882470257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6819","nonce":2882470257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6823","nonce":2882470257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6821","nonce":2882470257}]},"public_addr":"172.21.15.192:6817/2882470257","cluster_addr":"172.21.15.192:6819/2882470257","heartbeat_back_addr":"172.21.15.192:6823/2882470257","heartbeat_front_addr":"172.21.15.192:6821/2882470257","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:41:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 msgr2=0x7f9b2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 0x7f9b2406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b3b0 tx=0x7f9b28009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 msgr2=0x7f9b38113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b790 tx=0x7f9b34009250 comp rx=0 tx=0).stop 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 shutdown_connections 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f9b240692a0 0x7f9b2406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b38102de0 0x7f9b38113980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 --2- 172.21.15.63:0/88636299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b38103780 0x7f9b38113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 >> 172.21.15.63:0/88636299 conn(0x7f9b380f8990 msgr2=0x7f9b38107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 shutdown_connections 2024-04-24T22:41:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:25.718+0000 7f9b3f663700 1 -- 172.21.15.63:0/88636299 wait complete. 2024-04-24T22:41:25.981 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:25 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/88636299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:25 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/88636299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:41:26.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T22:41:26.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T22:41:26.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T22:41:26.282 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T22:41:26.282 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T22:41:26.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T22:41:26.597 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:26.612 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:26 smithi192 ceph-mon[33991]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:26 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:26.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:26 smithi063 ceph-mon[26912]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:26.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:26 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:28.389 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:28.389 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:28.761 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:29.022 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:28 smithi063 ceph-mon[26912]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:28 smithi192 ceph-mon[33991]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 -- 172.21.15.63:0/3727212399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c102e50 msgr2=0x7eff5c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 --2- 172.21.15.63:0/3727212399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c102e50 0x7eff5c103220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009ce0 tx=0x7eff4c004d10 comp rx=0 tx=0).stop 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 -- 172.21.15.63:0/3727212399 shutdown_connections 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 --2- 172.21.15.63:0/3727212399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff5c065640 0x7eff5c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 --2- 172.21.15.63:0/3727212399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c102e50 0x7eff5c103220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 -- 172.21.15.63:0/3727212399 >> 172.21.15.63:0/3727212399 conn(0x7eff5c071d00 msgr2=0x7eff5c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 -- 172.21.15.63:0/3727212399 shutdown_connections 2024-04-24T22:41:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.150+0000 7eff639be700 1 -- 172.21.15.63:0/3727212399 wait complete. 2024-04-24T22:41:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 Processor -- start 2024-04-24T22:41:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 -- start start 2024-04-24T22:41:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff5c102e50 0x7eff5c114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c114880 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.151+0000 7eff639be700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c10dc50 con 0x7eff5c102e50 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43286/0 (socket says 172.21.15.63:43286) 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 -- 172.21.15.63:0/2843395478 learned_addr learned my addr 172.21.15.63:0/2843395478 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 -- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff5c102e50 msgr2=0x7eff5c114230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff5c102e50 0x7eff5c114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c009940 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff6175a700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00be90 tx=0x7eff4c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff4c00e040 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c10ded0 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff4c008ea0 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff4c01c810 con 0x7eff5c065640 2024-04-24T22:41:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.152+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c10e490 con 0x7eff5c065640 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.153+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff5c10e9d0 con 0x7eff5c065640 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7eff4c012c00 con 0x7eff5c065640 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 0x7eff4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7eff4c088ad0 con 0x7eff5c065640 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff60f59700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 0x7eff4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 0x7eff44071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff44071a30 con 0x7eff4406ef70 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.154+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff4c08d220 con 0x7eff5c065640 2024-04-24T22:41:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.155+0000 7eff61f5b700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 0x7eff44071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.155+0000 7eff60f59700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 0x7eff4406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff5c10f590 tx=0x7eff50007cc0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.155+0000 7eff61f5b700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 0x7eff44071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.156+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== osd.3 v2:172.21.15.192:6800/2771944170 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7eff44071a30 con 0x7eff4406ef70 2024-04-24T22:41:30.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.172+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 --> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff5c05e660 con 0x7eff4406ef70 2024-04-24T22:41:30.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff59ffb700 1 -- 172.21.15.63:0/2843395478 <== osd.3 v2:172.21.15.192:6800/2771944170 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7eff5c05e660 con 0x7eff4406ef70 2024-04-24T22:41:30.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 msgr2=0x7eff44071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 0x7eff44071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 msgr2=0x7eff4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 0x7eff4406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff5c10f590 tx=0x7eff50007cc0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 msgr2=0x7eff5c113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.173+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00be90 tx=0x7eff4c004d10 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 shutdown_connections 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:6800/2771944170,v1:172.21.15.192:6801/2771944170] conn(0x7eff4406ef70 0x7eff44071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7eff440692a0 0x7eff4406b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff5c065640 0x7eff5c113cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 --2- 172.21.15.63:0/2843395478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff5c102e50 0x7eff5c114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 >> 172.21.15.63:0/2843395478 conn(0x7eff5c071d00 msgr2=0x7eff5c0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:30.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 shutdown_connections 2024-04-24T22:41:30.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:30.174+0000 7eff639be700 1 -- 172.21.15.63:0/2843395478 wait complete. 2024-04-24T22:41:31.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:30 smithi192 ceph-mon[33991]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:31.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:30 smithi063 ceph-mon[26912]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:31.503 INFO:teuthology.orchestra.run.smithi063.stdout:111669149714 2024-04-24T22:41:31.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T22:41:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.922+0000 7f2524d31700 1 -- 172.21.15.63:0/2401844291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25201026b0 msgr2=0x7f25200fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.922+0000 7f2524d31700 1 --2- 172.21.15.63:0/2401844291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25201026b0 0x7f25200fbd20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2514004940 tx=0x7f2514009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- 172.21.15.63:0/2401844291 shutdown_connections 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 --2- 172.21.15.63:0/2401844291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25201026b0 0x7f25200fbd20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 --2- 172.21.15.63:0/2401844291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2520101d10 0x7f25201020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- 172.21.15.63:0/2401844291 >> 172.21.15.63:0/2401844291 conn(0x7f25200f78a0 msgr2=0x7f25200f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- 172.21.15.63:0/2401844291 shutdown_connections 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- 172.21.15.63:0/2401844291 wait complete. 2024-04-24T22:41:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 Processor -- start 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- start start 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25201026b0 0x7f25201030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25201035f0 con 0x7f2520101d10 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.923+0000 7f2524d31700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520103760 con 0x7f25201026b0 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43304/0 (socket says 172.21.15.63:43304) 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 -- 172.21.15.63:0/111019759 learned_addr learned my addr 172.21.15.63:0/111019759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251dd9b700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25201026b0 0x7f25201030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 -- 172.21.15.63:0/111019759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25201026b0 msgr2=0x7f25201030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25201026b0 0x7f25201030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514005040 con 0x7f2520101d10 2024-04-24T22:41:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f251e59c700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f251000b790 tx=0x7f2510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f251000f770 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f251000fda0 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2520119e90 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f251000d310 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.924+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252011a1e0 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.925+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25200fd460 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2510010420 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 0x7f250806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2510087810 con 0x7f2520101d10 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f251dd9b700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 0x7f250806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 0x7f2508071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2508071a30 con 0x7f250806ef70 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.926+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2510087c10 con 0x7f2520101d10 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.927+0000 7f251ed9d700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 0x7f2508071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.927+0000 7f251dd9b700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 0x7f250806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f251400b020 tx=0x7f2514009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.927+0000 7f251ed9d700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 0x7f2508071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.928+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== osd.2 v2:172.21.15.63:6818/2431699450 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2508071a30 con 0x7f250806ef70 2024-04-24T22:41:31.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 --> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2520066be0 con 0x7f250806ef70 2024-04-24T22:41:31.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f250effd700 1 -- 172.21.15.63:0/111019759 <== osd.2 v2:172.21.15.63:6818/2431699450 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2520066be0 con 0x7f250806ef70 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 msgr2=0x7f2508071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 0x7f2508071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 msgr2=0x7f250806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 0x7f250806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f251400b020 tx=0x7f2514009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 msgr2=0x7f2520102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.948+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f251000b790 tx=0x7f2510009250 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 shutdown_connections 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6818/2431699450,v1:172.21.15.63:6819/2431699450] conn(0x7f250806ef70 0x7f2508071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f25080692a0 0x7f250806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2520101d10 0x7f2520102b70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 --2- 172.21.15.63:0/111019759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25201026b0 0x7f25201030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:31.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 >> 172.21.15.63:0/111019759 conn(0x7f25200f78a0 msgr2=0x7f2520066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 shutdown_connections 2024-04-24T22:41:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:31.949+0000 7f2524d31700 1 -- 172.21.15.63:0/111019759 wait complete. 2024-04-24T22:41:33.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:32 smithi192 ceph-mon[33991]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:33.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:32 smithi063 ceph-mon[26912]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:33.691 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:34.597 INFO:teuthology.orchestra.run.smithi063.stdout:81604378647 2024-04-24T22:41:34.597 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T22:41:35.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:34 smithi063 ceph-mon[26912]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:34 smithi192 ceph-mon[33991]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- 172.21.15.63:0/2344389620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fe7d0 msgr2=0x7f52b0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 --2- 172.21.15.63:0/2344389620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fe7d0 0x7f52b0106e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f529c004970 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- 172.21.15.63:0/2344389620 shutdown_connections 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 --2- 172.21.15.63:0/2344389620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fe7d0 0x7f52b0106e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 --2- 172.21.15.63:0/2344389620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fde30 0x7f52b00fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- 172.21.15.63:0/2344389620 >> 172.21.15.63:0/2344389620 conn(0x7f52b00f7860 msgr2=0x7f52b00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- 172.21.15.63:0/2344389620 shutdown_connections 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- 172.21.15.63:0/2344389620 wait complete. 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 Processor -- start 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.731+0000 7f52b70a1700 1 -- start start 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fe7d0 0x7f52b0113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01144f0 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b010daf0 con 0x7f52b00fe7d0 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43326/0 (socket says 172.21.15.63:43326) 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 -- 172.21.15.63:0/3879265711 learned_addr learned my addr 172.21.15.63:0/3879265711 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52affff700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fe7d0 0x7f52b0113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 -- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fe7d0 msgr2=0x7f52b0113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fe7d0 0x7f52b0113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c005040 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b4e3d700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52a4002a50 tx=0x7f52a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a4007cc0 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52a400f040 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b010ddd0 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a400da30 con 0x7f52b00fde30 2024-04-24T22:41:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.732+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b010e2e0 con 0x7f52b00fde30 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.733+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52b010e890 con 0x7f52b00fde30 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.734+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f52a400dbb0 con 0x7f52b00fde30 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.734+0000 7f52ad7fa700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 0x7f52a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.734+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f52a4086c10 con 0x7f52b00fde30 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.734+0000 7f52affff700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 0x7f52a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52ad7fa700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 0x7f52a0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52a0071a30 con 0x7f52a006ef70 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a408c210 con 0x7f52b00fde30 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52affff700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 0x7f52a006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f529c004f50 tx=0x7f529c009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52b563e700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 0x7f52a0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.735+0000 7f52b563e700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 0x7f52a0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.736+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== osd.4 v2:172.21.15.192:6808/1725358058 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f52a0071a30 con 0x7f52a006ef70 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.751+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 --> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52b010df60 con 0x7f52a006ef70 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52ad7fa700 1 -- 172.21.15.63:0/3879265711 <== osd.4 v2:172.21.15.192:6808/1725358058 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f52b010df60 con 0x7f52a006ef70 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 msgr2=0x7f52a0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 0x7f52a0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 msgr2=0x7f52a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 0x7f52a006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f529c004f50 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 msgr2=0x7f52b0113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52a4002a50 tx=0x7f52a40046b0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 shutdown_connections 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:6808/1725358058,v1:172.21.15.192:6809/1725358058] conn(0x7f52a006ef70 0x7f52a0071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f52a00692a0 0x7f52a006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52b00fde30 0x7f52b0113960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 --2- 172.21.15.63:0/3879265711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52b00fe7d0 0x7f52b0113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 >> 172.21.15.63:0/3879265711 conn(0x7f52b00f7860 msgr2=0x7f52b01036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 shutdown_connections 2024-04-24T22:41:35.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:35.752+0000 7f52b70a1700 1 -- 172.21.15.63:0/3879265711 wait complete. 2024-04-24T22:41:37.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:36 smithi063 ceph-mon[26912]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:37.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 -- 172.21.15.63:0/3684632824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc103790 msgr2=0x7f82dc0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:37.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 --2- 172.21.15.63:0/3684632824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc103790 0x7f82dc0fce00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004940 tx=0x7f82cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:37.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 -- 172.21.15.63:0/3684632824 shutdown_connections 2024-04-24T22:41:37.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 --2- 172.21.15.63:0/3684632824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc103790 0x7f82dc0fce00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 --2- 172.21.15.63:0/3684632824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc102df0 0x7f82dc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 -- 172.21.15.63:0/3684632824 >> 172.21.15.63:0/3684632824 conn(0x7f82dc0f8980 msgr2=0x7f82dc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:37.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 -- 172.21.15.63:0/3684632824 shutdown_connections 2024-04-24T22:41:37.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.026+0000 7f82e3879700 1 -- 172.21.15.63:0/3684632824 wait complete. 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 Processor -- start 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 -- start start 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 0x7f82dc114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc1146b0 con 0x7f82dc102df0 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e3879700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc10daa0 con 0x7f82dc103790 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e1615700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e1615700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43348/0 (socket says 172.21.15.63:43348) 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e1615700 1 -- 172.21.15.63:0/4041777244 learned_addr learned my addr 172.21.15.63:0/4041777244 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.027+0000 7f82e0e14700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 0x7f82dc114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e0e14700 1 -- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 msgr2=0x7f82dc113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e0e14700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e0e14700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc005040 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e1615700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e0e14700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 0x7f82dc114060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc00e070 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82cc013c00 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc10dd20 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc00b650 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.028+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc10e200 con 0x7f82dc103790 2024-04-24T22:41:37.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.029+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82dc10e6e0 con 0x7f82dc103790 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f82cc01c400 con 0x7f82dc103790 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 0x7f82c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f82cc0883f0 con 0x7f82dc103790 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82e1615700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 0x7f82c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 0x7f82c8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:37.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82c8071a30 con 0x7f82c806ef70 2024-04-24T22:41:37.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.030+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82cc08a9a0 con 0x7f82dc103790 2024-04-24T22:41:37.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.031+0000 7f82e1e16700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 0x7f82c8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:37.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.031+0000 7f82e1615700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 0x7f82c806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82d80041d0 tx=0x7f82d800b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:37.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.031+0000 7f82e1e16700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 0x7f82c8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:37.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.032+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== osd.0 v2:172.21.15.63:6802/758545231 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f82c8071a30 con 0x7f82c806ef70 2024-04-24T22:41:37.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 --> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82dc101770 con 0x7f82c806ef70 2024-04-24T22:41:37.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82d1ffb700 1 -- 172.21.15.63:0/4041777244 <== osd.0 v2:172.21.15.63:6802/758545231 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82dc101770 con 0x7f82c806ef70 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 msgr2=0x7f82c8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 0x7f82c8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 msgr2=0x7f82c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 0x7f82c806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82d80041d0 tx=0x7f82d800b040 comp rx=0 tx=0).stop 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 msgr2=0x7f82dc114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 0x7f82dc114060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 shutdown_connections 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.047+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6802/758545231,v1:172.21.15.63:6803/758545231] conn(0x7f82c806ef70 0x7f82c8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82c80692a0 0x7f82c806b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82dc102df0 0x7f82dc113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 --2- 172.21.15.63:0/4041777244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82dc103790 0x7f82dc114060 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 >> 172.21.15.63:0/4041777244 conn(0x7f82dc0f8980 msgr2=0x7f82dc100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 shutdown_connections 2024-04-24T22:41:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:37.048+0000 7f82e3879700 1 -- 172.21.15.63:0/4041777244 wait complete. 2024-04-24T22:41:37.171 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:36 smithi192 ceph-mon[33991]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:37.458 INFO:teuthology.orchestra.run.smithi063.stdout:146028888078 2024-04-24T22:41:37.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T22:41:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.719+0000 7fd9fdb47700 1 -- 172.21.15.63:0/1978475076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8101ce0 msgr2=0x7fd9f81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.719+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/1978475076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8101ce0 0x7fd9f81020b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8004940 tx=0x7fd9e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 -- 172.21.15.63:0/1978475076 shutdown_connections 2024-04-24T22:41:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/1978475076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8102680 0x7fd9f80fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/1978475076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8101ce0 0x7fd9f81020b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 -- 172.21.15.63:0/1978475076 >> 172.21.15.63:0/1978475076 conn(0x7fd9f80f7850 msgr2=0x7fd9f80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 -- 172.21.15.63:0/1978475076 shutdown_connections 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 -- 172.21.15.63:0/1978475076 wait complete. 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 Processor -- start 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.720+0000 7fd9fdb47700 1 -- start start 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8101ce0 0x7fd9f8112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8113630 con 0x7fd9f8102680 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f810ca20 con 0x7fd9f8101ce0 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44354/0 (socket says 172.21.15.63:44354) 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 -- 172.21.15.63:0/4126911709 learned_addr learned my addr 172.21.15.63:0/4126911709 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f77fe700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8101ce0 0x7fd9f8112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 -- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8101ce0 msgr2=0x7fd9f8112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8101ce0 0x7fd9f8112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8005040 con 0x7fd9f8102680 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9f6ffd700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00fce0 tx=0x7fd9ec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9ec007ea0 con 0x7fd9f8102680 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f810cca0 con 0x7fd9f8102680 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9ec00dcf0 con 0x7fd9f8102680 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9ec012480 con 0x7fd9f8102680 2024-04-24T22:41:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.721+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f810d260 con 0x7fd9f8102680 2024-04-24T22:41:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.722+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9f810d740 con 0x7fd9f8102680 2024-04-24T22:41:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.723+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd9ec009080 con 0x7fd9f8102680 2024-04-24T22:41:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9dffff700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 0x7fd9e006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd9ec089990 con 0x7fd9f8102680 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9f77fe700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 0x7fd9e006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9dffff700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 0x7fd9e0071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9e0071a40 con 0x7fd9e006ef80 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9ec089d70 con 0x7fd9f8102680 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9f77fe700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 0x7fd9e006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009fd0 tx=0x7fd9e8009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.724+0000 7fd9f7fff700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 0x7fd9e0071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.725+0000 7fd9f7fff700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 0x7fd9e0071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:38.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.725+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== osd.5 v2:172.21.15.192:6816/2882470257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd9e0071a40 con 0x7fd9e006ef80 2024-04-24T22:41:38.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.740+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 --> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9f80f91e0 con 0x7fd9e006ef80 2024-04-24T22:41:38.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9dffff700 1 -- 172.21.15.63:0/4126911709 <== osd.5 v2:172.21.15.192:6816/2882470257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd9f80f91e0 con 0x7fd9e006ef80 2024-04-24T22:41:38.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 msgr2=0x7fd9e0071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:38.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 0x7fd9e0071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 msgr2=0x7fd9e006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:38.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 0x7fd9e006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009fd0 tx=0x7fd9e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:38.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 msgr2=0x7fd9f8112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:38.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00fce0 tx=0x7fd9ec004600 comp rx=0 tx=0).stop 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 shutdown_connections 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:6816/2882470257,v1:172.21.15.192:6817/2882470257] conn(0x7fd9e006ef80 0x7fd9e0071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fd9e00692b0 0x7fd9e006b760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f8101ce0 0x7fd9f8112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 --2- 172.21.15.63:0/4126911709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9f8102680 0x7fd9f8112fe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 >> 172.21.15.63:0/4126911709 conn(0x7fd9f80f7850 msgr2=0x7fd9f80f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 shutdown_connections 2024-04-24T22:41:38.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:38.741+0000 7fd9fdb47700 1 -- 172.21.15.63:0/4126911709 wait complete. 2024-04-24T22:41:39.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:38 smithi063 ceph-mon[26912]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:39.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:38 smithi192 ceph-mon[33991]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:39.540 INFO:teuthology.orchestra.run.smithi063.stdout:167503724554 2024-04-24T22:41:39.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T22:41:40.018 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:39 smithi063 ceph-mon[26912]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:39 smithi192 ceph-mon[33991]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:40.511 INFO:teuthology.orchestra.run.smithi063.stdout:38654705699 2024-04-24T22:41:40.512 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T22:41:40.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 -- 172.21.15.63:0/3884927638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 msgr2=0x7f15a8065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 --2- 172.21.15.63:0/3884927638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8065ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1598004940 tx=0x7f1598009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:40.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 -- 172.21.15.63:0/3884927638 shutdown_connections 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 --2- 172.21.15.63:0/3884927638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8065ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 --2- 172.21.15.63:0/3884927638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 0x7f15a8103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 -- 172.21.15.63:0/3884927638 >> 172.21.15.63:0/3884927638 conn(0x7f15a8071d00 msgr2=0x7f15a8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 -- 172.21.15.63:0/3884927638 shutdown_connections 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.691+0000 7f15b0246700 1 -- 172.21.15.63:0/3884927638 wait complete. 2024-04-24T22:41:40.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 Processor -- start 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 -- start start 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 0x7f15a81140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8114700 con 0x7f15a8065640 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15b0246700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810daf0 con 0x7f15a8102e50 2024-04-24T22:41:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15adfe2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15adfe2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44372/0 (socket says 172.21.15.63:44372) 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15adfe2700 1 -- 172.21.15.63:0/748534303 learned_addr learned my addr 172.21.15.63:0/748534303 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.692+0000 7f15ad7e1700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 0x7f15a81140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15adfe2700 1 -- 172.21.15.63:0/748534303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 msgr2=0x7f15a81140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15adfe2700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 0x7f15a81140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15adfe2700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598005040 con 0x7f15a8065640 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15adfe2700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f15a4002a50 tx=0x7f15a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a4007cc0 con 0x7f15a8065640 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15a400f040 con 0x7f15a8065640 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a810ddd0 con 0x7f15a8065640 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a400da30 con 0x7f15a8065640 2024-04-24T22:41:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.693+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a810e360 con 0x7f15a8065640 2024-04-24T22:41:40.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.694+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15a810e8d0 con 0x7f15a8065640 2024-04-24T22:41:40.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.695+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f15a400db90 con 0x7f15a8065640 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.695+0000 7f159e7fc700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 0x7f159406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f15a4086cb0 con 0x7f15a8065640 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f15ad7e1700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 0x7f159406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f159e7fc700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 0x7f1594071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1594071a30 con 0x7f159406ef70 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15a4089260 con 0x7f15a8065640 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f15ae7e3700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 0x7f1594071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f15ad7e1700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 0x7f159406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15a810edd0 tx=0x7f1598009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.696+0000 7f15ae7e3700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 0x7f1594071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.697+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== osd.1 v2:172.21.15.63:6810/1039875891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1594071a30 con 0x7f159406ef70 2024-04-24T22:41:40.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.713+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 --> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15a805e660 con 0x7f159406ef70 2024-04-24T22:41:40.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.713+0000 7f159e7fc700 1 -- 172.21.15.63:0/748534303 <== osd.1 v2:172.21.15.63:6810/1039875891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f15a805e660 con 0x7f159406ef70 2024-04-24T22:41:40.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.713+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 msgr2=0x7f1594071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.713+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 0x7f1594071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.713+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 msgr2=0x7f159406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 0x7f159406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15a810edd0 tx=0x7f1598009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:40.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 msgr2=0x7f15a8113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f15a4002a50 tx=0x7f15a40046b0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 shutdown_connections 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6810/1039875891,v1:172.21.15.63:6811/1039875891] conn(0x7f159406ef70 0x7f1594071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15940692a0 0x7f159406b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a8065640 0x7f15a8113b70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 --2- 172.21.15.63:0/748534303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15a8102e50 0x7f15a81140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 >> 172.21.15.63:0/748534303 conn(0x7f15a8071d00 msgr2=0x7f15a80fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 shutdown_connections 2024-04-24T22:41:40.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.714+0000 7f15b0246700 1 -- 172.21.15.63:0/748534303 wait complete. 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 -- 172.21.15.63:0/778600090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc103790 msgr2=0x7fafbc0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 --2- 172.21.15.63:0/778600090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc103790 0x7fafbc0fce00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004940 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 -- 172.21.15.63:0/778600090 shutdown_connections 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 --2- 172.21.15.63:0/778600090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc103790 0x7fafbc0fce00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 --2- 172.21.15.63:0/778600090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc102df0 0x7fafbc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 -- 172.21.15.63:0/778600090 >> 172.21.15.63:0/778600090 conn(0x7fafbc0f8980 msgr2=0x7fafbc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 -- 172.21.15.63:0/778600090 shutdown_connections 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.887+0000 7fafc1095700 1 -- 172.21.15.63:0/778600090 wait complete. 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 Processor -- start 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 -- start start 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc103790 0x7fafbc1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc114710 con 0x7fafbc102df0 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafc1095700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc10db00 con 0x7fafbc103790 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44388/0 (socket says 172.21.15.63:44388) 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 -- 172.21.15.63:0/664600347 learned_addr learned my addr 172.21.15.63:0/664600347 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafba59c700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc103790 0x7fafbc1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 -- 172.21.15.63:0/664600347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc103790 msgr2=0x7fafbc1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc103790 0x7fafbc1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:40.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 -- 172.21.15.63:0/664600347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0005040 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.888+0000 7fafbad9d700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c830 tx=0x7fafac004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.889+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafac00a760 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.889+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc10dde0 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.889+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafac00ad90 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.889+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafac0113a0 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.889+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc10e3a0 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.890+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc0fe540 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.890+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fafac012400 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.890+0000 7fafa37fe700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 0x7fafa406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.891+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fafac086f20 con 0x7fafbc102df0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.891+0000 7fafba59c700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 0x7fafa406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.891+0000 7fafba59c700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 0x7fafa406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafbc10ecc0 tx=0x7fafb0009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:40.895+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fafac057020 con 0x7fafbc102df0 2024-04-24T22:41:41.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.107+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fafbc109ed0 con 0x7fafbc102df0 2024-04-24T22:41:41.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.107+0000 7fafa37fe700 1 -- 172.21.15.63:0/664600347 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafac051e60 con 0x7fafbc102df0 2024-04-24T22:41:41.109 INFO:teuthology.orchestra.run.smithi063.stdout:111669149716 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 msgr2=0x7fafa406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 0x7fafa406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafbc10ecc0 tx=0x7fafb0009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 msgr2=0x7fafbc113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c830 tx=0x7fafac004470 comp rx=0 tx=0).stop 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 shutdown_connections 2024-04-24T22:41:41.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fafa40692a0 0x7fafa406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafbc102df0 0x7fafbc113b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 --2- 172.21.15.63:0/664600347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc103790 0x7fafbc1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:41.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 >> 172.21.15.63:0/664600347 conn(0x7fafbc0f8980 msgr2=0x7fafbc0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:41.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 shutdown_connections 2024-04-24T22:41:41.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:41.109+0000 7fafc1095700 1 -- 172.21.15.63:0/664600347 wait complete. 2024-04-24T22:41:41.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:41 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/664600347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T22:41:41.493 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:41 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/664600347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T22:41:41.522 INFO:teuthology.orchestra.run.smithi063.stdout:60129542174 2024-04-24T22:41:41.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T22:41:42.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149716 for osd.3 2024-04-24T22:41:42.139 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:42.204 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:42.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:42 smithi063 ceph-mon[26912]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:42.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:42 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:42 smithi192 ceph-mon[33991]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:42 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:42.758 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:44 smithi192 ceph-mon[33991]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:45.020 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:44 smithi063 ceph-mon[26912]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- 172.21.15.63:0/158870473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fe810 msgr2=0x7fb074106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 --2- 172.21.15.63:0/158870473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fe810 0x7fb074106e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb060004940 tx=0x7fb060009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- 172.21.15.63:0/158870473 shutdown_connections 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 --2- 172.21.15.63:0/158870473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fe810 0x7fb074106e50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 --2- 172.21.15.63:0/158870473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fde70 0x7fb0740fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- 172.21.15.63:0/158870473 >> 172.21.15.63:0/158870473 conn(0x7fb0740f78e0 msgr2=0x7fb0740f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- 172.21.15.63:0/158870473 shutdown_connections 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- 172.21.15.63:0/158870473 wait complete. 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 Processor -- start 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.145+0000 7fb07acaa700 1 -- start start 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb07acaa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb07acaa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fe810 0x7fb0740fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb07acaa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740fdbf0 con 0x7fb0740fde70 2024-04-24T22:41:46.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb07acaa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740fc720 con 0x7fb0740fe810 2024-04-24T22:41:46.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44414/0 (socket says 172.21.15.63:44414) 2024-04-24T22:41:46.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 -- 172.21.15.63:0/344490140 learned_addr learned my addr 172.21.15.63:0/344490140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb073fff700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fe810 0x7fb0740fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 -- 172.21.15.63:0/344490140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fe810 msgr2=0x7fb0740fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fe810 0x7fb0740fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.146+0000 7fb078a46700 1 -- 172.21.15.63:0/344490140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060005040 con 0x7fb0740fde70 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb078a46700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb068002a50 tx=0x7fb0680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb068007cc0 con 0x7fb0740fde70 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0740fca00 con 0x7fb0740fde70 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb06800f040 con 0x7fb0740fde70 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06800da30 con 0x7fb0740fde70 2024-04-24T22:41:46.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.147+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07411a050 con 0x7fb0740fde70 2024-04-24T22:41:46.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.148+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074066ad0 con 0x7fb0740fde70 2024-04-24T22:41:46.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.149+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb06800db90 con 0x7fb0740fde70 2024-04-24T22:41:46.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.149+0000 7fb0717fa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 0x7fb06406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:46.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.149+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb068086c40 con 0x7fb0740fde70 2024-04-24T22:41:46.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.149+0000 7fb073fff700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 0x7fb06406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:46.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.150+0000 7fb073fff700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 0x7fb06406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0740fd3d0 tx=0x7fb060009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:46.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.154+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb0680532a0 con 0x7fb0740fde70 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.353+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb07404c710 con 0x7fb0740fde70 2024-04-24T22:41:46.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.354+0000 7fb0717fa700 1 -- 172.21.15.63:0/344490140 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0680568c0 con 0x7fb0740fde70 2024-04-24T22:41:46.356 INFO:teuthology.orchestra.run.smithi063.stdout:81604378650 2024-04-24T22:41:46.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.355+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 msgr2=0x7fb06406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.355+0000 7fb07acaa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 0x7fb06406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0740fd3d0 tx=0x7fb060009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:46.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.355+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 msgr2=0x7fb0740fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.355+0000 7fb07acaa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb068002a50 tx=0x7fb0680046b0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 shutdown_connections 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0640692a0 0x7fb06406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0740fde70 0x7fb0740fbca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 --2- 172.21.15.63:0/344490140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740fe810 0x7fb0740fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 >> 172.21.15.63:0/344490140 conn(0x7fb0740f78e0 msgr2=0x7fb0741036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 shutdown_connections 2024-04-24T22:41:46.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:46.356+0000 7fb07acaa700 1 -- 172.21.15.63:0/344490140 wait complete. 2024-04-24T22:41:46.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:46 smithi063 ceph-mon[26912]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:46.922 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:46 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/344490140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T22:41:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:46 smithi192 ceph-mon[33991]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:46 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/344490140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T22:41:47.209 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:47.799 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378647 got 81604378650 for osd.2 2024-04-24T22:41:47.800 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:48.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 -- 172.21.15.63:0/1408065276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 msgr2=0x7f09a80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 --2- 172.21.15.63:0/1408065276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a80fc1b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0994004940 tx=0x7f0994009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:48.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 -- 172.21.15.63:0/1408065276 shutdown_connections 2024-04-24T22:41:48.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 --2- 172.21.15.63:0/1408065276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a80fc1b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 --2- 172.21.15.63:0/1408065276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 0x7f09a8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 -- 172.21.15.63:0/1408065276 >> 172.21.15.63:0/1408065276 conn(0x7f09a8071d00 msgr2=0x7f09a8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 -- 172.21.15.63:0/1408065276 shutdown_connections 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.057+0000 7f09af020700 1 -- 172.21.15.63:0/1408065276 wait complete. 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 Processor -- start 2024-04-24T22:41:48.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 -- start start 2024-04-24T22:41:48.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 0x7f09a8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8103610 con 0x7f09a80fbd40 2024-04-24T22:41:48.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09af020700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8103780 con 0x7f09a8101d90 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44446/0 (socket says 172.21.15.63:44446) 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 -- 172.21.15.63:0/1644095824 learned_addr learned my addr 172.21.15.63:0/1644095824 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09a7fff700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 0x7f09a8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 -- 172.21.15.63:0/1644095824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 msgr2=0x7f09a8103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 0x7f09a8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.058+0000 7f09acdbc700 1 -- 172.21.15.63:0/1644095824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994005040 con 0x7f09a80fbd40 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09acdbc700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c830 tx=0x7f099c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099c00a760 con 0x7f09a80fbd40 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a8119eb0 con 0x7f09a80fbd40 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f099c00ad90 con 0x7f09a80fbd40 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099c0113a0 con 0x7f09a80fbd40 2024-04-24T22:41:48.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.059+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a811a310 con 0x7f09a80fbd40 2024-04-24T22:41:48.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.060+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a80fd430 con 0x7f09a80fbd40 2024-04-24T22:41:48.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.061+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f099c012400 con 0x7f09a80fbd40 2024-04-24T22:41:48.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.061+0000 7f09a57fa700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 0x7f099806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.061+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f099c086df0 con 0x7f09a80fbd40 2024-04-24T22:41:48.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.061+0000 7f09a7fff700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 0x7f099806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.062+0000 7f09a7fff700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 0x7f099806b760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f099400b3b0 tx=0x7f0994009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.066+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f099c053500 con 0x7f09a80fbd40 2024-04-24T22:41:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.271+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f09a805f5c0 con 0x7f09a80fbd40 2024-04-24T22:41:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.271+0000 7f09a57fa700 1 -- 172.21.15.63:0/1644095824 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f099c056b20 con 0x7f09a80fbd40 2024-04-24T22:41:48.273 INFO:teuthology.orchestra.run.smithi063.stdout:60129542175 2024-04-24T22:41:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 msgr2=0x7f099806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 0x7f099806b760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f099400b3b0 tx=0x7f0994009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 msgr2=0x7f09a8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c830 tx=0x7f099c004470 comp rx=0 tx=0).stop 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 shutdown_connections 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f09980692b0 0x7f099806b760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a80fbd40 0x7f09a8102b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 --2- 172.21.15.63:0/1644095824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8101d90 0x7f09a8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 >> 172.21.15.63:0/1644095824 conn(0x7f09a8071d00 msgr2=0x7f09a8065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.273+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 shutdown_connections 2024-04-24T22:41:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.274+0000 7f09af020700 1 -- 172.21.15.63:0/1644095824 wait complete. 2024-04-24T22:41:48.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- 172.21.15.63:0/1026026856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844101ce0 msgr2=0x7f68441020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 --2- 172.21.15.63:0/1026026856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844101ce0 0x7f68441020b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6834004940 tx=0x7f6834009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- 172.21.15.63:0/1026026856 shutdown_connections 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 --2- 172.21.15.63:0/1026026856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844102680 0x7f68440fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 --2- 172.21.15.63:0/1026026856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844101ce0 0x7f68441020b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- 172.21.15.63:0/1026026856 >> 172.21.15.63:0/1026026856 conn(0x7f68440f7850 msgr2=0x7f68440f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- 172.21.15.63:0/1026026856 shutdown_connections 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- 172.21.15.63:0/1026026856 wait complete. 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 Processor -- start 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.969+0000 7f684b566700 1 -- start start 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f684b566700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844101ce0 0x7f6844113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f684b566700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f684b566700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68441146d0 con 0x7f6844102680 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f684b566700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684410dac0 con 0x7f6844101ce0 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44424/0 (socket says 172.21.15.63:44424) 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 -- 172.21.15.63:0/3334148205 learned_addr learned my addr 172.21.15.63:0/3334148205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 -- 172.21.15.63:0/3334148205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844101ce0 msgr2=0x7f6844113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6849302700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844101ce0 0x7f6844113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844101ce0 0x7f6844113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 -- 172.21.15.63:0/3334148205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834005040 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6848b01700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f684000c800 tx=0x7f6840004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684000a760 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f684000ad90 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.970+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684410dda0 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.971+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68400113a0 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.971+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684410e360 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.972+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68440fd430 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.972+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6840011500 con 0x7f6844102680 2024-04-24T22:41:48.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.972+0000 7f6839ffb700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 0x7f683006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.973+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6840086d80 con 0x7f6844102680 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.973+0000 7f6849302700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 0x7f683006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stdout:146028888081 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.973+0000 7f6849302700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 0x7f683006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6834004f90 tx=0x7f6834009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:47.977+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6840056f90 con 0x7f6844102680 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.179+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6844108de0 con 0x7f6844102680 2024-04-24T22:41:48.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.180+0000 7f6839ffb700 1 -- 172.21.15.63:0/3334148205 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6840088020 con 0x7f6844102680 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 msgr2=0x7f683006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 0x7f683006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6834004f90 tx=0x7f6834009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 msgr2=0x7f6844114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f684000c800 tx=0x7f6840004470 comp rx=0 tx=0).stop 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 shutdown_connections 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f68300692a0 0x7f683006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6844101ce0 0x7f6844113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 --2- 172.21.15.63:0/3334148205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6844102680 0x7f6844114080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 >> 172.21.15.63:0/3334148205 conn(0x7f68440f7850 msgr2=0x7f68440f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 shutdown_connections 2024-04-24T22:41:48.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:48.182+0000 7f684b566700 1 -- 172.21.15.63:0/3334148205 wait complete. 2024-04-24T22:41:48.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:48 smithi063 ceph-mon[26912]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:48.916 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:48 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3334148205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T22:41:48.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:48 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1644095824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T22:41:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:48 smithi192 ceph-mon[33991]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:48 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3334148205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T22:41:48.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:48 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1644095824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T22:41:49.536 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888081 for osd.4 2024-04-24T22:41:49.537 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:50.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542175 for osd.1 2024-04-24T22:41:50.057 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:50.901 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:50 smithi063 ceph-mon[26912]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:50.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:50 smithi192 ceph-mon[33991]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 -- 172.21.15.63:0/3950218798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e40fbd40 msgr2=0x7f15e40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 --2- 172.21.15.63:0/3950218798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e40fbd40 0x7f15e40fc1b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15d8004940 tx=0x7f15d8009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 -- 172.21.15.63:0/3950218798 shutdown_connections 2024-04-24T22:41:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 --2- 172.21.15.63:0/3950218798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e40fbd40 0x7f15e40fc1b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 --2- 172.21.15.63:0/3950218798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e4101d90 0x7f15e4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 -- 172.21.15.63:0/3950218798 >> 172.21.15.63:0/3950218798 conn(0x7f15e4071d00 msgr2=0x7f15e4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 -- 172.21.15.63:0/3950218798 shutdown_connections 2024-04-24T22:41:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 -- 172.21.15.63:0/3950218798 wait complete. 2024-04-24T22:41:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.687+0000 7f15ea219700 1 Processor -- start 2024-04-24T22:41:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15ea219700 1 -- start start 2024-04-24T22:41:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15ea219700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e40fbd40 0x7f15e4113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15ea219700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15ea219700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e4114740 con 0x7f15e4101d90 2024-04-24T22:41:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15ea219700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e410db30 con 0x7f15e40fbd40 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59522/0 (socket says 172.21.15.63:59522) 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 -- 172.21.15.63:0/675232476 learned_addr learned my addr 172.21.15.63:0/675232476 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e37fe700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e40fbd40 0x7f15e4113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 -- 172.21.15.63:0/675232476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e40fbd40 msgr2=0x7f15e4113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e40fbd40 0x7f15e4113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 -- 172.21.15.63:0/675232476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8005040 con 0x7f15e4101d90 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.688+0000 7f15e2ffd700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15d800b8b0 tx=0x7f15d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.689+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d800e040 con 0x7f15e4101d90 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.689+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15d8013420 con 0x7f15e4101d90 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.689+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e410ddb0 con 0x7f15e4101d90 2024-04-24T22:41:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.689+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d801c630 con 0x7f15e4101d90 2024-04-24T22:41:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.689+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e410e370 con 0x7f15e4101d90 2024-04-24T22:41:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.690+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e40fd430 con 0x7f15e4101d90 2024-04-24T22:41:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.690+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f15d8013a00 con 0x7f15e4101d90 2024-04-24T22:41:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.691+0000 7f15cbfff700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 0x7f15cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.691+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f15d8088120 con 0x7f15e4101d90 2024-04-24T22:41:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.691+0000 7f15e37fe700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 0x7f15cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.691+0000 7f15e37fe700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 0x7f15cc06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15d4007fd0 tx=0x7f15d4009040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.696+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f15d8058220 con 0x7f15e4101d90 2024-04-24T22:41:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.899+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f15e410f260 con 0x7f15e4101d90 2024-04-24T22:41:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.899+0000 7f15cbfff700 1 -- 172.21.15.63:0/675232476 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15d8057db0 con 0x7f15e4101d90 2024-04-24T22:41:51.233 INFO:teuthology.orchestra.run.smithi063.stdout:38654705701 2024-04-24T22:41:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 msgr2=0x7f15cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 0x7f15cc06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15d4007fd0 tx=0x7f15d4009040 comp rx=0 tx=0).stop 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 msgr2=0x7f15e41140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15d800b8b0 tx=0x7f15d8009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 shutdown_connections 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f15cc0692a0 0x7f15cc06b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e40fbd40 0x7f15e4113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 --2- 172.21.15.63:0/675232476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e4101d90 0x7f15e41140f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 >> 172.21.15.63:0/675232476 conn(0x7f15e4071d00 msgr2=0x7f15e4065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 shutdown_connections 2024-04-24T22:41:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:50.901+0000 7f15ea219700 1 -- 172.21.15.63:0/675232476 wait complete. 2024-04-24T22:41:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:51 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/675232476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T22:41:51.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-04-24T22:41:51.920 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:51 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/675232476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T22:41:52.886 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:52 smithi063 ceph-mon[26912]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 -- 172.21.15.63:0/2895506698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 msgr2=0x7f466c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 --2- 172.21.15.63:0/2895506698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c102160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4664009ce0 tx=0x7f4664004d10 comp rx=0 tx=0).stop 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 -- 172.21.15.63:0/2895506698 shutdown_connections 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 --2- 172.21.15.63:0/2895506698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 --2- 172.21.15.63:0/2895506698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c102160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 -- 172.21.15.63:0/2895506698 >> 172.21.15.63:0/2895506698 conn(0x7f466c071d00 msgr2=0x7f466c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 -- 172.21.15.63:0/2895506698 shutdown_connections 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.906+0000 7f467433f700 1 -- 172.21.15.63:0/2895506698 wait complete. 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 Processor -- start 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 -- start start 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c10ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:52.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c10f920 con 0x7f466c101d90 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f467433f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c10fa60 con 0x7f466c0fbd40 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59542/0 (socket says 172.21.15.63:59542) 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 -- 172.21.15.63:0/4089052713 learned_addr learned my addr 172.21.15.63:0/4089052713 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46720db700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c10ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 -- 172.21.15.63:0/4089052713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 msgr2=0x7f466c10ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c10ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 -- 172.21.15.63:0/4089052713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664009940 con 0x7f466c101d90 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46720db700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.907+0000 7f46718da700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f466000b790 tx=0x7f4660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.908+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466000f770 con 0x7f466c101d90 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.908+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c117d80 con 0x7f466c101d90 2024-04-24T22:41:52.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.908+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f466000fda0 con 0x7f466c101d90 2024-04-24T22:41:52.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.908+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466000d310 con 0x7f466c101d90 2024-04-24T22:41:52.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.908+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c118310 con 0x7f466c101d90 2024-04-24T22:41:52.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.909+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c0fd430 con 0x7f466c101d90 2024-04-24T22:41:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.909+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4660010420 con 0x7f466c101d90 2024-04-24T22:41:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.910+0000 7f466affd700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 0x7f465406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.910+0000 7f46720db700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 0x7f465406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.910+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4660086e60 con 0x7f466c101d90 2024-04-24T22:41:52.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.910+0000 7f46720db700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 0x7f465406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46640048c0 tx=0x7f4664004d10 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:52.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:52.914+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f46600534d0 con 0x7f466c101d90 2024-04-24T22:41:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:52 smithi192 ceph-mon[33991]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.113+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f466c108ed0 con 0x7f466c101d90 2024-04-24T22:41:53.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.113+0000 7f466affd700 1 -- 172.21.15.63:0/4089052713 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4660056af0 con 0x7f466c101d90 2024-04-24T22:41:53.116 INFO:teuthology.orchestra.run.smithi063.stdout:167503724556 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 msgr2=0x7f465406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 0x7f465406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46640048c0 tx=0x7f4664004d10 comp rx=0 tx=0).stop 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 msgr2=0x7f466c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f466000b790 tx=0x7f4660009250 comp rx=0 tx=0).stop 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 shutdown_connections 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46540692a0 0x7f465406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f466c0fbd40 0x7f466c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 --2- 172.21.15.63:0/4089052713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f466c101d90 0x7f466c10f350 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 >> 172.21.15.63:0/4089052713 conn(0x7f466c071d00 msgr2=0x7f466c0fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 shutdown_connections 2024-04-24T22:41:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:53.116+0000 7f467433f700 1 -- 172.21.15.63:0/4089052713 wait complete. 2024-04-24T22:41:53.699 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:53 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4089052713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T22:41:53.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724556 for osd.5 2024-04-24T22:41:53.781 DEBUG:teuthology.parallel:result is None 2024-04-24T22:41:53.781 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T22:41:53.781 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T22:41:54.047 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:53 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4089052713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T22:41:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:54 smithi063 ceph-mon[26912]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:54 smithi192 ceph-mon[33991]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 -- 172.21.15.63:0/3803389409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 msgr2=0x7f7168102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 --2- 172.21.15.63:0/3803389409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f7168102160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7164004940 tx=0x7f7164009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 -- 172.21.15.63:0/3803389409 shutdown_connections 2024-04-24T22:41:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 --2- 172.21.15.63:0/3803389409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 0x7f71680fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 --2- 172.21.15.63:0/3803389409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f7168102160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 -- 172.21.15.63:0/3803389409 >> 172.21.15.63:0/3803389409 conn(0x7f7168071d00 msgr2=0x7f7168072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 -- 172.21.15.63:0/3803389409 shutdown_connections 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.786+0000 7f716fad2700 1 -- 172.21.15.63:0/3803389409 wait complete. 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 Processor -- start 2024-04-24T22:41:55.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 -- start start 2024-04-24T22:41:55.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 0x7f7168115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7168116980 con 0x7f7168101d90 2024-04-24T22:41:55.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716fad2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716810fcc0 con 0x7f71680fbd40 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59564/0 (socket says 172.21.15.63:59564) 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 -- 172.21.15.63:0/1527627918 learned_addr learned my addr 172.21.15.63:0/1527627918 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d86e700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 0x7f7168115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 -- 172.21.15.63:0/1527627918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 msgr2=0x7f7168115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 0x7f7168115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 -- 172.21.15.63:0/1527627918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164005040 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.787+0000 7f716d06d700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f715c00b790 tx=0x7f715c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.788+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f715c00f6f0 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.788+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f715c00fd20 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.788+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716810ff40 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.788+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f715c0103a0 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.788+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7168110500 con 0x7f7168101d90 2024-04-24T22:41:55.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.789+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71681061c0 con 0x7f7168101d90 2024-04-24T22:41:55.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.789+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f715c011400 con 0x7f7168101d90 2024-04-24T22:41:55.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.789+0000 7f715a7fc700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:55.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.790+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f715c086b60 con 0x7f7168101d90 2024-04-24T22:41:55.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.790+0000 7f716d86e700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:55.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.790+0000 7f716d86e700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 0x7f715406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7164004560 tx=0x7f7164009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:55.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.794+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f715c056d00 con 0x7f7168101d90 2024-04-24T22:41:55.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.984+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7168108e70 con 0x7f71540692a0 2024-04-24T22:41:55.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.985+0000 7f715a7fc700 1 -- 172.21.15.63:0/1527627918 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28034 (secure 0 0 0) 0x7f7168108e70 con 0x7f71540692a0 2024-04-24T22:41:55.988 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 msgr2=0x7f715406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 0x7f715406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7164004560 tx=0x7f7164009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 msgr2=0x7f71681162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f715c00b790 tx=0x7f715c009250 comp rx=0 tx=0).stop 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 shutdown_connections 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680fbd40 0x7f7168115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 --2- 172.21.15.63:0/1527627918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7168101d90 0x7f71681162a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 >> 172.21.15.63:0/1527627918 conn(0x7f7168071d00 msgr2=0x7f7168065e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 shutdown_connections 2024-04-24T22:41:55.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:55.988+0000 7f716fad2700 1 -- 172.21.15.63:0/1527627918 wait complete. 2024-04-24T22:41:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-04-24T22:41:56.694 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-24T22:41:55.608051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2603946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002621"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":188,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-24T22:41:05.844294+0000","last_change":"2024-04-24T22:40:41.359242+0000","last_active":"2024-04-24T22:41:05.844294+0000","last_peered":"2024-04-24T22:41:05.844294+0000","last_clean":"2024-04-24T22:41:05.844294+0000","last_became_active":"2024-04-24T22:40:41.358961+0000","last_became_peered":"2024-04-24T22:40:41.358961+0000","last_unstale":"2024-04-24T22:41:05.844294+0000","last_undegraded":"2024-04-24T22:41:05.844294+0000","last_fullsized":"2024-04-24T22:41:05.844294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:54.337667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:54.337667+0000","last_clean_scrub_stamp":"2024-04-24T22:39:54.337667+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:20.685146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":6.8879999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed Apr 24 22:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.31},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Apr 24 22:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.308},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.29799999999999999}]},{"osd":3,"last update":"Wed Apr 24 22:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Apr 24 22:41:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 22:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.26700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.22},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.249,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.27500000000000002}]},{"osd":2,"last update":"Wed Apr 24 22:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.24199999999999999}]},{"osd":3,"last update":"Wed Apr 24 22:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Wed Apr 24 22:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:27 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Apr 24 22:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.48899999999999999}]},{"osd":3,"last update":"Wed Apr 24 22:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Apr 24 22:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:41:56.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T22:41:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:56 smithi192 ceph-mon[33991]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:57.003 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:56 smithi063 ceph-mon[26912]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:41:57.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:57 smithi063 ceph-mon[26912]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:57 smithi192 ceph-mon[33991]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 -- 172.21.15.63:0/1443499287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 msgr2=0x7f8374109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 --2- 172.21.15.63:0/1443499287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f8374109180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8368004940 tx=0x7f8368009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 -- 172.21.15.63:0/1443499287 shutdown_connections 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 --2- 172.21.15.63:0/1443499287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f8374109180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 --2- 172.21.15.63:0/1443499287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 0x7f8374070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 -- 172.21.15.63:0/1443499287 >> 172.21.15.63:0/1443499287 conn(0x7f83740f8980 msgr2=0x7f83740fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 -- 172.21.15.63:0/1443499287 shutdown_connections 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.624+0000 7f8378e82700 1 -- 172.21.15.63:0/1443499287 wait complete. 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 Processor -- start 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 -- start start 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 0x7f8374115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8374116930 con 0x7f8374070b90 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8378e82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837410fd40 con 0x7f8374070280 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8371d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8371d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42332/0 (socket says 172.21.15.63:42332) 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f8371d9b700 1 -- 172.21.15.63:0/40945117 learned_addr learned my addr 172.21.15.63:0/40945117 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.625+0000 7f837259c700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 0x7f8374115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8371d9b700 1 -- 172.21.15.63:0/40945117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 msgr2=0x7f8374115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8371d9b700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 0x7f8374115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8371d9b700 1 -- 172.21.15.63:0/40945117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8368005040 con 0x7f8374070b90 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8371d9b700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8368009150 tx=0x7f8368009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f836800e040 con 0x7f8374070b90 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8368013420 con 0x7f8374070b90 2024-04-24T22:41:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f836801c610 con 0x7f8374070b90 2024-04-24T22:41:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837410ffc0 con 0x7f8374070b90 2024-04-24T22:41:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.626+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8374110500 con 0x7f8374070b90 2024-04-24T22:41:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.627+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83740719c0 con 0x7f8374070b90 2024-04-24T22:41:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.627+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f836800baf0 con 0x7f8374070b90 2024-04-24T22:41:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.628+0000 7f8362ffd700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 0x7f835c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.628+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8368087fd0 con 0x7f8374070b90 2024-04-24T22:41:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.628+0000 7f837259c700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 0x7f835c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.628+0000 7f837259c700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 0x7f835c06b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f836c001450 tx=0x7f836c00b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.632+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f83680546e0 con 0x7f8374070b90 2024-04-24T22:41:58.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.823+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f837410c0f0 con 0x7f835c0692b0 2024-04-24T22:41:58.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.825+0000 7f8362ffd700 1 -- 172.21.15.63:0/40945117 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27994 (secure 0 0 0) 0x7f837410c0f0 con 0x7f835c0692b0 2024-04-24T22:41:58.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 msgr2=0x7f835c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 0x7f835c06b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f836c001450 tx=0x7f836c00b040 comp rx=0 tx=0).stop 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 msgr2=0x7f83741162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8368009150 tx=0x7f8368009e30 comp rx=0 tx=0).stop 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 shutdown_connections 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f835c0692b0 0x7f835c06b760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8374070280 0x7f8374115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 --2- 172.21.15.63:0/40945117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8374070b90 0x7f83741162e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 >> 172.21.15.63:0/40945117 conn(0x7f83740f8980 msgr2=0x7f83740f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 shutdown_connections 2024-04-24T22:41:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:41:58.827+0000 7f8378e82700 1 -- 172.21.15.63:0/40945117 wait complete. 2024-04-24T22:41:58.830 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-04-24T22:41:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:58 smithi063 ceph-mon[26912]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:58 smithi192 ceph-mon[33991]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:41:59.456 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-24T22:41:57.608495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2603946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002620"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":188,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-24T22:41:05.844294+0000","last_change":"2024-04-24T22:40:41.359242+0000","last_active":"2024-04-24T22:41:05.844294+0000","last_peered":"2024-04-24T22:41:05.844294+0000","last_clean":"2024-04-24T22:41:05.844294+0000","last_became_active":"2024-04-24T22:40:41.358961+0000","last_became_peered":"2024-04-24T22:40:41.358961+0000","last_unstale":"2024-04-24T22:41:05.844294+0000","last_undegraded":"2024-04-24T22:41:05.844294+0000","last_fullsized":"2024-04-24T22:41:05.844294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:54.337667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:54.337667+0000","last_clean_scrub_stamp":"2024-04-24T22:39:54.337667+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:20.685146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.499}]},{"osd":1,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":6.8879999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45600000000000002}]},{"osd":3,"last update":"Wed Apr 24 22:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.32600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Apr 24 22:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.498},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51400000000000001}]},{"osd":4,"last update":"Wed Apr 24 22:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":138807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Apr 24 22:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.308},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.29799999999999999}]},{"osd":3,"last update":"Wed Apr 24 22:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Apr 24 22:41:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 22:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.26700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.249,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.27500000000000002}]},{"osd":2,"last update":"Wed Apr 24 22:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.223},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.317}]},{"osd":3,"last update":"Wed Apr 24 22:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Apr 24 22:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 22:41:27 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Apr 24 22:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.48899999999999999}]},{"osd":3,"last update":"Wed Apr 24 22:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Apr 24 22:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:41:59.457 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T22:41:59.457 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T22:41:59.457 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T22:41:59.457 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T22:41:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:41:59.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:41:59 smithi063 ceph-mon[26912]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:42:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:41:59 smithi192 ceph-mon[33991]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T22:42:00.743 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:00 smithi063 ceph-mon[26912]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:00 smithi192 ceph-mon[33991]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 -- 172.21.15.63:0/1787921072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 msgr2=0x7f03ac106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 --2- 172.21.15.63:0/1787921072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac106dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0394004940 tx=0x7f0394009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 -- 172.21.15.63:0/1787921072 shutdown_connections 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 --2- 172.21.15.63:0/1787921072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac106dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 --2- 172.21.15.63:0/1787921072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 0x7f03ac0fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 -- 172.21.15.63:0/1787921072 >> 172.21.15.63:0/1787921072 conn(0x7f03ac0f7890 msgr2=0x7f03ac0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 -- 172.21.15.63:0/1787921072 shutdown_connections 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.236+0000 7f03b1756700 1 -- 172.21.15.63:0/1787921072 wait complete. 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 Processor -- start 2024-04-24T22:42:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 -- start start 2024-04-24T22:42:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 0x7f03ac115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1168c0 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03b1756700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac10fc90 con 0x7f03ac0fdde0 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42360/0 (socket says 172.21.15.63:42360) 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 -- 172.21.15.63:0/191930699 learned_addr learned my addr 172.21.15.63:0/191930699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aaffd700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 0x7f03ac115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 -- 172.21.15.63:0/191930699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 msgr2=0x7f03ac115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 0x7f03ac115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 -- 172.21.15.63:0/191930699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394005040 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03aa7fc700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0394009380 tx=0x7f0394009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.237+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039400e040 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.238+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac10ff10 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.238+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0394009af0 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.238+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039401c3f0 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.238+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac110450 con 0x7f03ac0fe780 2024-04-24T22:42:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.238+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac066750 con 0x7f03ac0fe780 2024-04-24T22:42:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.239+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0394013420 con 0x7f03ac0fe780 2024-04-24T22:42:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.239+0000 7f03a37fe700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 0x7f039806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.240+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f03940881d0 con 0x7f03ac0fe780 2024-04-24T22:42:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.240+0000 7f03aaffd700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 0x7f039806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.240+0000 7f03aaffd700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 0x7f039806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004550 tx=0x7f03a4007ed0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.244+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0394058300 con 0x7f03ac0fe780 2024-04-24T22:42:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.484+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f03ac05f5c0 con 0x7f03ac0fe780 2024-04-24T22:42:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.485+0000 7f03a37fe700 1 -- 172.21.15.63:0/191930699 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0394057e90 con 0x7f03ac0fe780 2024-04-24T22:42:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:01.487 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T22:42:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 msgr2=0x7f039806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 0x7f039806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004550 tx=0x7f03a4007ed0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 msgr2=0x7f03ac116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0394009380 tx=0x7f0394009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 shutdown_connections 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f03980692a0 0x7f039806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03ac0fdde0 0x7f03ac115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 --2- 172.21.15.63:0/191930699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03ac0fe780 0x7f03ac116270 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.486+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 >> 172.21.15.63:0/191930699 conn(0x7f03ac0f7890 msgr2=0x7f03ac0f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.487+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 shutdown_connections 2024-04-24T22:42:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:01.487+0000 7f03b1756700 1 -- 172.21.15.63:0/191930699 wait complete. 2024-04-24T22:42:01.784 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:01 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/191930699' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T22:42:02.095 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T22:42:02.096 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T22:42:02.096 INFO:teuthology.run_tasks:Running task print... 2024-04-24T22:42:02.105 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-24T22:42:02.105 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:02.114 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:02.115 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-24T22:42:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:01 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/191930699' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T22:42:02.382 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:02.779 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:02 smithi063 ceph-mon[26912]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:02 smithi192 ceph-mon[33991]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:04.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 -- 172.21.15.63:0/2654464103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fe080 msgr2=0x7f864c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 --2- 172.21.15.63:0/2654464103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fe080 0x7f864c0fe4f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8634004940 tx=0x7f8634009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 -- 172.21.15.63:0/2654464103 shutdown_connections 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 --2- 172.21.15.63:0/2654464103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fe080 0x7f864c0fe4f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 --2- 172.21.15.63:0/2654464103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fce80 0x7f864c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 -- 172.21.15.63:0/2654464103 >> 172.21.15.63:0/2654464103 conn(0x7f864c0f8590 msgr2=0x7f864c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 -- 172.21.15.63:0/2654464103 shutdown_connections 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 -- 172.21.15.63:0/2654464103 wait complete. 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.074+0000 7f865220e700 1 Processor -- start 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f865220e700 1 -- start start 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f865220e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f865220e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 0x7f864c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f865220e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c10e5f0 con 0x7f864c0fce80 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f865220e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c10ede0 con 0x7f864c0fe080 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:04.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42378/0 (socket says 172.21.15.63:42378) 2024-04-24T22:42:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 -- 172.21.15.63:0/1053998558 learned_addr learned my addr 172.21.15.63:0/1053998558 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864affd700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 0x7f864c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 -- 172.21.15.63:0/1053998558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 msgr2=0x7f864c10df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 0x7f864c10df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 -- 172.21.15.63:0/1053998558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634005040 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864affd700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 0x7f864c10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.075+0000 7f864b7fe700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f863c00bae0 tx=0x7f863c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.076+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863c00c8c0 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.076+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c10f0c0 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.076+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f863c014410 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.076+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863c007610 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.076+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c10f680 con 0x7f864c0fce80 2024-04-24T22:42:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.077+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c05f5c0 con 0x7f864c0fce80 2024-04-24T22:42:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.077+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f863c014580 con 0x7f864c0fce80 2024-04-24T22:42:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.078+0000 7f8643fff700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 0x7f863806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.078+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f863c086f60 con 0x7f864c0fce80 2024-04-24T22:42:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.078+0000 7f864affd700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 0x7f863806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.078+0000 7f864affd700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 0x7f863806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f863400b3b0 tx=0x7f8634009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.082+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f863c04d250 con 0x7f864c0fce80 2024-04-24T22:42:04.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.276+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f864c04bed0 con 0x7f864c0fce80 2024-04-24T22:42:04.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.280+0000 7f8643fff700 1 -- 172.21.15.63:0/1053998558 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f863c056fe0 con 0x7f864c0fce80 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 msgr2=0x7f863806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 0x7f863806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f863400b3b0 tx=0x7f8634009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 msgr2=0x7f864c10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f863c00bae0 tx=0x7f863c009810 comp rx=0 tx=0).stop 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 shutdown_connections 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86380692a0 0x7f863806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c0fce80 0x7f864c10da20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 --2- 172.21.15.63:0/1053998558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f864c0fe080 0x7f864c10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:04.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 >> 172.21.15.63:0/1053998558 conn(0x7f864c0f8590 msgr2=0x7f864c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:04.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 shutdown_connections 2024-04-24T22:42:04.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:04.283+0000 7f865220e700 1 -- 172.21.15.63:0/1053998558 wait complete. 2024-04-24T22:42:04.832 INFO:teuthology.run_tasks:Running task print... 2024-04-24T22:42:04.841 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-24T22:42:04.841 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:04.856 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:04.856 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T22:42:04.879 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:04.879 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1053998558' entity='client.admin' 2024-04-24T22:42:04.879 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:04.879 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:04.880 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:04.880 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1053998558' entity='client.admin' 2024-04-24T22:42:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:05.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:05.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:05.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 -- 172.21.15.63:0/2102870805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 msgr2=0x7f49e80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 --2- 172.21.15.63:0/2102870805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e80fd270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004940 tx=0x7f49e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 -- 172.21.15.63:0/2102870805 shutdown_connections 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 --2- 172.21.15.63:0/2102870805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 0x7f49e80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 --2- 172.21.15.63:0/2102870805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e80fd270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 -- 172.21.15.63:0/2102870805 >> 172.21.15.63:0/2102870805 conn(0x7f49e80f8520 msgr2=0x7f49e80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 -- 172.21.15.63:0/2102870805 shutdown_connections 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.734+0000 7f49ef693700 1 -- 172.21.15.63:0/2102870805 wait complete. 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 Processor -- start 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 -- start start 2024-04-24T22:42:06.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 0x7f49e810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e810e530 con 0x7f49e80fce60 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ef693700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8112ed0 con 0x7f49e80fdfd0 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ed42f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ed42f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42410/0 (socket says 172.21.15.63:42410) 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.735+0000 7f49ed42f700 1 -- 172.21.15.63:0/3415574182 learned_addr learned my addr 172.21.15.63:0/3415574182 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ecc2e700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 0x7f49e810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ed42f700 1 -- 172.21.15.63:0/3415574182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 msgr2=0x7f49e810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ed42f700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 0x7f49e810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ed42f700 1 -- 172.21.15.63:0/3415574182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4005040 con 0x7f49e80fce60 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ed42f700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49e400b3b0 tx=0x7f49e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e400e040 con 0x7f49e80fce60 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e400bcc0 con 0x7f49e80fce60 2024-04-24T22:42:06.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e81130d0 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e401c630 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.736+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e8113640 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.737+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e804c710 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.738+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f49e4013980 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.738+0000 7f49ddffb700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 0x7f49d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.738+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f49e4087bd0 con 0x7f49e80fce60 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.738+0000 7f49ecc2e700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 0x7f49d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.738+0000 7f49ecc2e700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 0x7f49d406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006fd0 tx=0x7f49d8008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:06.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.743+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f49e4057cd0 con 0x7f49e80fce60 2024-04-24T22:42:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:06 smithi192 ceph-mon[33991]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:06.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.957+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e81022a0 con 0x7f49d40692a0 2024-04-24T22:42:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.958+0000 7f49ddffb700 1 -- 172.21.15.63:0/3415574182 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f49e81022a0 con 0x7f49d40692a0 2024-04-24T22:42:06.960 INFO:teuthology.orchestra.run.smithi063.stdout:Backend: cephadm 2024-04-24T22:42:06.960 INFO:teuthology.orchestra.run.smithi063.stdout:Available: Yes 2024-04-24T22:42:06.960 INFO:teuthology.orchestra.run.smithi063.stdout:Paused: No 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 msgr2=0x7f49d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 0x7f49d406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006fd0 tx=0x7f49d8008040 comp rx=0 tx=0).stop 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 msgr2=0x7f49e810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49e400b3b0 tx=0x7f49e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 shutdown_connections 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f49d40692a0 0x7f49d406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49e80fce60 0x7f49e810d960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 --2- 172.21.15.63:0/3415574182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49e80fdfd0 0x7f49e810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 >> 172.21.15.63:0/3415574182 conn(0x7f49e80f8520 msgr2=0x7f49e80f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:06.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 shutdown_connections 2024-04-24T22:42:06.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:06.960+0000 7f49ef693700 1 -- 172.21.15.63:0/3415574182 wait complete. 2024-04-24T22:42:06.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:06 smithi063 ceph-mon[26912]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:07.581 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T22:42:07.893 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:09.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:08 smithi063 ceph-mon[26912]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:09.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:08 smithi063 ceph-mon[26912]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:08 smithi192 ceph-mon[33991]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:08 smithi192 ceph-mon[33991]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:09.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 -- 172.21.15.63:0/4007624557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 msgr2=0x7f14640fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 --2- 172.21.15.63:0/4007624557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f14640fc050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1460004940 tx=0x7f1460009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:09.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 -- 172.21.15.63:0/4007624557 shutdown_connections 2024-04-24T22:42:09.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 --2- 172.21.15.63:0/4007624557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14640fcef0 0x7f14640fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 --2- 172.21.15.63:0/4007624557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f14640fc050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 -- 172.21.15.63:0/4007624557 >> 172.21.15.63:0/4007624557 conn(0x7f14640f73a0 msgr2=0x7f14640f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 -- 172.21.15.63:0/4007624557 shutdown_connections 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.566+0000 7f146c1e0700 1 -- 172.21.15.63:0/4007624557 wait complete. 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 Processor -- start 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 -- start start 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14640fcef0 0x7f1464112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1464066f50 con 0x7f14640fbc40 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f146c1e0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1464067090 con 0x7f14640fcef0 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49596/0 (socket says 172.21.15.63:49596) 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 -- 172.21.15.63:0/729017118 learned_addr learned my addr 172.21.15.63:0/729017118 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 -- 172.21.15.63:0/729017118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14640fcef0 msgr2=0x7f1464112b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14640fcef0 0x7f1464112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 -- 172.21.15.63:0/729017118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1460005040 con 0x7f14640fbc40 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f1469f7c700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f146000b3b0 tx=0x7f1460009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.567+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f146000e040 con 0x7f14640fbc40 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.568+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f146000bcc0 con 0x7f14640fbc40 2024-04-24T22:42:09.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.568+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464113180 con 0x7f14640fbc40 2024-04-24T22:42:09.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.568+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f146001c630 con 0x7f14640fbc40 2024-04-24T22:42:09.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.568+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1464113740 con 0x7f14640fbc40 2024-04-24T22:42:09.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.569+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14640fb040 con 0x7f14640fbc40 2024-04-24T22:42:09.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.569+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1460013980 con 0x7f14640fbc40 2024-04-24T22:42:09.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.569+0000 7f145a7fc700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 0x7f145006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.569+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1460087c80 con 0x7f14640fbc40 2024-04-24T22:42:09.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.570+0000 7f146977b700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 0x7f145006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.570+0000 7f146977b700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 0x7f145006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454006db0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:09.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.574+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1460054270 con 0x7f14640fbc40 2024-04-24T22:42:09.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.780+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f14641011e0 con 0x7f14500692a0 2024-04-24T22:42:09.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.789+0000 7f145a7fc700 1 -- 172.21.15.63:0/729017118 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f14641011e0 con 0x7f14500692a0 2024-04-24T22:42:09.792 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 2m ago 5m 25.7M - 0.25.0 c8568f914cd2 f1e9967ae2bb 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 2m ago 5m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (4m) 65s ago 4m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 2m ago 4m 78.3M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (6m) 2m ago 6m 470M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (4m) 65s ago 4m 431M - 17.2.7-902-gc61472c8 0de407c1b3af afb9d2db7487 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 2m ago 6m 41.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (4m) 65s ago 4m 40.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 b97c5a0b7f9e 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 65s ago 4m 16.1M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:42:09.793 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 2m ago 3m 39.0M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 2m ago 2m 35.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 2m ago 2m 35.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (2m) 65s ago 2m 38.3M 5312M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (95s) 65s ago 95s 36.3M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (70s) 65s ago 70s 11.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:42:09.794 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 2m ago 4m 35.6M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 msgr2=0x7f145006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 0x7f145006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454006db0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 msgr2=0x7f1464110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f146000b3b0 tx=0x7f1460009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 shutdown_connections 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f14500692a0 0x7f145006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14640fbc40 0x7f1464110600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 --2- 172.21.15.63:0/729017118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14640fcef0 0x7f1464112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 >> 172.21.15.63:0/729017118 conn(0x7f14640f73a0 msgr2=0x7f14640f89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:09.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 shutdown_connections 2024-04-24T22:42:09.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:09.792+0000 7f146c1e0700 1 -- 172.21.15.63:0/729017118 wait complete. 2024-04-24T22:42:10.497 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T22:42:10.762 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:10 smithi063 ceph-mon[26912]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:10 smithi063 ceph-mon[26912]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:10 smithi192 ceph-mon[33991]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:10 smithi192 ceph-mon[33991]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:11.977 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.466+0000 7f6c375ca700 1 -- 172.21.15.63:0/3188116241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fe420 msgr2=0x7f6c300fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.466+0000 7f6c375ca700 1 --2- 172.21.15.63:0/3188116241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fe420 0x7f6c300fe830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6c20004940 tx=0x7f6c20009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- 172.21.15.63:0/3188116241 shutdown_connections 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 --2- 172.21.15.63:0/3188116241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fed70 0x7f6c300faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 --2- 172.21.15.63:0/3188116241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fe420 0x7f6c300fe830 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- 172.21.15.63:0/3188116241 >> 172.21.15.63:0/3188116241 conn(0x7f6c300f6be0 msgr2=0x7f6c300f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- 172.21.15.63:0/3188116241 shutdown_connections 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- 172.21.15.63:0/3188116241 wait complete. 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 Processor -- start 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- start start 2024-04-24T22:42:12.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fe420 0x7f6c300fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300fe030 con 0x7f6c300fed70 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.467+0000 7f6c375ca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300fc620 con 0x7f6c300fe420 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49622/0 (socket says 172.21.15.63:49622) 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 -- 172.21.15.63:0/4219402851 learned_addr learned my addr 172.21.15.63:0/4219402851 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c35366700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fe420 0x7f6c300fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 -- 172.21.15.63:0/4219402851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fe420 msgr2=0x7f6c300fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fe420 0x7f6c300fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 -- 172.21.15.63:0/4219402851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20005040 con 0x7f6c300fed70 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c34b65700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c002a50 tx=0x7f6c2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2c007cc0 con 0x7f6c300fed70 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300fc8d0 con 0x7f6c300fed70 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c2c00f040 con 0x7f6c300fed70 2024-04-24T22:42:12.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.468+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2c00da30 con 0x7f6c300fed70 2024-04-24T22:42:12.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.469+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300fce90 con 0x7f6c300fed70 2024-04-24T22:42:12.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.469+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3004c710 con 0x7f6c300fed70 2024-04-24T22:42:12.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.470+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6c2c00db90 con 0x7f6c300fed70 2024-04-24T22:42:12.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.470+0000 7f6c25ffb700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 0x7f6c1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:12.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.470+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6c2c086b30 con 0x7f6c300fed70 2024-04-24T22:42:12.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.470+0000 7f6c35366700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 0x7f6c1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:12.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.471+0000 7f6c35366700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 0x7f6c1c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000b970 tx=0x7f6c20009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:12.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.475+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6c2c056c30 con 0x7f6c300fed70 2024-04-24T22:42:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.681+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c301024f0 con 0x7f6c1c0692a0 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.685+0000 7f6c25ffb700 1 -- 172.21.15.63:0/4219402851 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f6c301024f0 con 0x7f6c1c0692a0 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:crash 2/2 2m ago 6m * 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:mgr 2/2 2m ago 6m count:2 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:mon 2/2 2m ago 5m smithi063:172.21.15.63=smithi063;smithi192:172.21.15.192=smithi192;count:2 2024-04-24T22:42:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-24T22:42:12.689 INFO:teuthology.orchestra.run.smithi063.stdout:osd 6 2m ago - 2024-04-24T22:42:12.689 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-24T22:42:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 msgr2=0x7f6c1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 0x7f6c1c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000b970 tx=0x7f6c20009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 msgr2=0x7f6c300fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c002a50 tx=0x7f6c2c0046b0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 shutdown_connections 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6c1c0692a0 0x7f6c1c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c300fe420 0x7f6c300fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 --2- 172.21.15.63:0/4219402851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c300fed70 0x7f6c300fc0e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 >> 172.21.15.63:0/4219402851 conn(0x7f6c300f6be0 msgr2=0x7f6c300f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 shutdown_connections 2024-04-24T22:42:12.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:12.687+0000 7f6c375ca700 1 -- 172.21.15.63:0/4219402851 wait complete. 2024-04-24T22:42:13.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:12 smithi063 ceph-mon[26912]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:12 smithi192 ceph-mon[33991]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:13.376 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T22:42:13.640 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:14.003 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:13 smithi063 ceph-mon[26912]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:14.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:13 smithi192 ceph-mon[33991]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:15.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:14 smithi063 ceph-mon[26912]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- 172.21.15.63:0/3776566635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 msgr2=0x7f74900ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 --2- 172.21.15.63:0/3776566635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900ffa10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7480004940 tx=0x7f7480009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- 172.21.15.63:0/3776566635 shutdown_connections 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 --2- 172.21.15.63:0/3776566635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74900fff50 0x7f7490102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 --2- 172.21.15.63:0/3776566635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900ffa10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- 172.21.15.63:0/3776566635 >> 172.21.15.63:0/3776566635 conn(0x7f74900f7440 msgr2=0x7f74900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:15.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- 172.21.15.63:0/3776566635 shutdown_connections 2024-04-24T22:42:15.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- 172.21.15.63:0/3776566635 wait complete. 2024-04-24T22:42:15.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 Processor -- start 2024-04-24T22:42:15.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.069+0000 7f7497e33700 1 -- start start 2024-04-24T22:42:15.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7497e33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7497e33700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74900fff50 0x7f74900fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7497e33700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74900fd860 con 0x7f7490066ef0 2024-04-24T22:42:15.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7497e33700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74900fbe50 con 0x7f74900fff50 2024-04-24T22:42:15.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49642/0 (socket says 172.21.15.63:49642) 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 -- 172.21.15.63:0/2797207759 learned_addr learned my addr 172.21.15.63:0/2797207759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 -- 172.21.15.63:0/2797207759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74900fff50 msgr2=0x7f74900fb910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74900fff50 0x7f74900fb910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 -- 172.21.15.63:0/2797207759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480005040 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7495bcf700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7480009150 tx=0x7f7480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748000e040 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7480004290 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.070+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74900fc0a0 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.071+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748001c630 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.071+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74900fc660 con 0x7f7490066ef0 2024-04-24T22:42:15.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.071+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749004c710 con 0x7f7490066ef0 2024-04-24T22:42:15.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.072+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f748000bbf0 con 0x7f7490066ef0 2024-04-24T22:42:15.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.072+0000 7f74867fc700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 0x7f747c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.072+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7480088190 con 0x7f7490066ef0 2024-04-24T22:42:15.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.072+0000 7f74953ce700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 0x7f747c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.073+0000 7f74953ce700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 0x7f747c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f748c004b70 tx=0x7f748c007cc0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:15.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.077+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7480058290 con 0x7f7490066ef0 2024-04-24T22:42:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:14 smithi192 ceph-mon[33991]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.278+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f74900f8f30 con 0x7f747c0692a0 2024-04-24T22:42:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.280+0000 7f74867fc700 1 -- 172.21.15.63:0/2797207759 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f74900f8f30 con 0x7f747c0692a0 2024-04-24T22:42:15.282 INFO:teuthology.orchestra.run.smithi063.stdout:HOST ADDR LABELS STATUS 2024-04-24T22:42:15.283 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 172.21.15.63 2024-04-24T22:42:15.283 INFO:teuthology.orchestra.run.smithi063.stdout:smithi192 172.21.15.192 2024-04-24T22:42:15.283 INFO:teuthology.orchestra.run.smithi063.stdout:2 hosts in cluster 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 msgr2=0x7f747c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 0x7f747c06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f748c004b70 tx=0x7f748c007cc0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 msgr2=0x7f74900fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7480009150 tx=0x7f7480009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 shutdown_connections 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f747c0692a0 0x7f747c06b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7490066ef0 0x7f74900fd260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 --2- 172.21.15.63:0/2797207759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74900fff50 0x7f74900fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 >> 172.21.15.63:0/2797207759 conn(0x7f74900f7440 msgr2=0x7f74900f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 shutdown_connections 2024-04-24T22:42:15.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:15.282+0000 7f7497e33700 1 -- 172.21.15.63:0/2797207759 wait complete. 2024-04-24T22:42:15.961 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T22:42:16.231 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:16 smithi192 ceph-mon[33991]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:16 smithi192 ceph-mon[33991]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:17.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:16 smithi063 ceph-mon[26912]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:17.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:16 smithi063 ceph-mon[26912]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 -- 172.21.15.63:0/1972531796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fcf60 msgr2=0x7f46580fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 --2- 172.21.15.63:0/1972531796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fcf60 0x7f46580fd3d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f464c004940 tx=0x7f464c009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 -- 172.21.15.63:0/1972531796 shutdown_connections 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 --2- 172.21.15.63:0/1972531796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fcf60 0x7f46580fd3d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 --2- 172.21.15.63:0/1972531796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fbcb0 0x7f46580fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 -- 172.21.15.63:0/1972531796 >> 172.21.15.63:0/1972531796 conn(0x7f46580f7430 msgr2=0x7f46580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 -- 172.21.15.63:0/1972531796 shutdown_connections 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.819+0000 7f465f570700 1 -- 172.21.15.63:0/1972531796 wait complete. 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 Processor -- start 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 -- start start 2024-04-24T22:42:17.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fcf60 0x7f46580fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46580fe8e0 con 0x7f46580fbcb0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465f570700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46580fea50 con 0x7f46580fcf60 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49656/0 (socket says 172.21.15.63:49656) 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 -- 172.21.15.63:0/4176574397 learned_addr learned my addr 172.21.15.63:0/4176574397 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465cb0b700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fcf60 0x7f46580fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 -- 172.21.15.63:0/4176574397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fcf60 msgr2=0x7f46580fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fcf60 0x7f46580fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 -- 172.21.15.63:0/4176574397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c005040 con 0x7f46580fbcb0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.820+0000 7f465d30c700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f465000bbb0 tx=0x7f4650009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.821+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46500092e0 con 0x7f46580fbcb0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.821+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46580fed30 con 0x7f46580fbcb0 2024-04-24T22:42:17.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.821+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f465000cbd0 con 0x7f46580fbcb0 2024-04-24T22:42:17.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.821+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4650007400 con 0x7f46580fbcb0 2024-04-24T22:42:17.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.821+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4658115ad0 con 0x7f46580fbcb0 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.822+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465805f5c0 con 0x7f46580fbcb0 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.822+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4650012070 con 0x7f46580fbcb0 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.823+0000 7f4655ffb700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 0x7f464006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.823+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4650086c70 con 0x7f46580fbcb0 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.823+0000 7f465cb0b700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 0x7f464006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:17.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.823+0000 7f465cb0b700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 0x7f464006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:17.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:17.827+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4650053260 con 0x7f46580fbcb0 2024-04-24T22:42:18.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.029+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f465805e3a0 con 0x7f46400692a0 2024-04-24T22:42:18.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.031+0000 7f4655ffb700 1 -- 172.21.15.63:0/4176574397 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f465805e3a0 con 0x7f46400692a0 2024-04-24T22:42:18.034 INFO:teuthology.orchestra.run.smithi063.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T22:42:18.034 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T22:42:18.034 INFO:teuthology.orchestra.run.smithi063.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 70s ago Has a FileSystem, LVM detected 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 msgr2=0x7f464006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 0x7f464006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 msgr2=0x7f46580ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f465000bbb0 tx=0x7f4650009840 comp rx=0 tx=0).stop 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 shutdown_connections 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f46400692a0 0x7f464006b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46580fbcb0 0x7f46580ffdd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 --2- 172.21.15.63:0/4176574397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46580fcf60 0x7f46580fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 >> 172.21.15.63:0/4176574397 conn(0x7f46580f7430 msgr2=0x7f4658065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:18.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 shutdown_connections 2024-04-24T22:42:18.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:18.033+0000 7f465f570700 1 -- 172.21.15.63:0/4176574397 wait complete. 2024-04-24T22:42:18.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:18.638 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:18.638 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-24T22:42:18.907 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:19.030 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:18 smithi063 ceph-mon[26912]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:18 smithi192 ceph-mon[33991]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:20.002 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:19 smithi063 ceph-mon[26912]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:19 smithi192 ceph-mon[33991]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:20.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.341+0000 7f4d26e11700 1 -- 172.21.15.63:0/4056895145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 msgr2=0x7f4d200fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.341+0000 7f4d26e11700 1 --2- 172.21.15.63:0/4056895145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d200fc0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d14004940 tx=0x7f4d14009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- 172.21.15.63:0/4056895145 shutdown_connections 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 --2- 172.21.15.63:0/4056895145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d200fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 --2- 172.21.15.63:0/4056895145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d200fc0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- 172.21.15.63:0/4056895145 >> 172.21.15.63:0/4056895145 conn(0x7f4d200f7440 msgr2=0x7f4d200f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:20.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- 172.21.15.63:0/4056895145 shutdown_connections 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- 172.21.15.63:0/4056895145 wait complete. 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 Processor -- start 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- start start 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d20112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20112c70 con 0x7f4d200fbce0 2024-04-24T22:42:20.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.342+0000 7f4d26e11700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20112de0 con 0x7f4d200fcf90 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38304/0 (socket says 172.21.15.63:38304) 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 -- 172.21.15.63:0/3946070995 learned_addr learned my addr 172.21.15.63:0/3946070995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d1ffff700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d20112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 -- 172.21.15.63:0/3946070995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 msgr2=0x7f4d20112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d20112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:20.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 -- 172.21.15.63:0/3946070995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14005040 con 0x7f4d200fbce0 2024-04-24T22:42:20.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d1ffff700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d20112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:42:20.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d24bad700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009150 tx=0x7f4d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:20.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1400e040 con 0x7f4d200fbce0 2024-04-24T22:42:20.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1400bbf0 con 0x7f4d200fbce0 2024-04-24T22:42:20.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1401c630 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d20113060 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.343+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d20113620 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.344+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2004c710 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.345+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4d1400bd60 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.345+0000 7f4d1d7fa700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 0x7f4d1006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.345+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4d14088190 con 0x7f4d200fbce0 2024-04-24T22:42:20.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.345+0000 7f4d1ffff700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 0x7f4d1006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:20.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.346+0000 7f4d1ffff700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 0x7f4d1006b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:20.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.350+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4d14058330 con 0x7f4d200fbce0 2024-04-24T22:42:20.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:20.575+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d20101260 con 0x7f4d100692a0 2024-04-24T22:42:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:21 smithi063 ceph-mon[26912]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:21 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T22:42:21.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:21 smithi192 ceph-mon[33991]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:21.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:21 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T22:42:22.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.098+0000 7f4d1d7fa700 1 -- 172.21.15.63:0/3946070995 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d20101260 con 0x7f4d100692a0 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 msgr2=0x7f4d1006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 0x7f4d1006b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).stop 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 msgr2=0x7f4d20066c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009150 tx=0x7f4d14009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 shutdown_connections 2024-04-24T22:42:22.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4d100692a0 0x7f4d1006b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:22.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d200fbce0 0x7f4d20066c90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:22.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 --2- 172.21.15.63:0/3946070995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d200fcf90 0x7f4d20112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:22.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 >> 172.21.15.63:0/3946070995 conn(0x7f4d200f7440 msgr2=0x7f4d200f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:22.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 shutdown_connections 2024-04-24T22:42:22.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:22.100+0000 7f4d26e11700 1 -- 172.21.15.63:0/3946070995 wait complete. 2024-04-24T22:42:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-mon[26912]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T22:42:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-mon[26912]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T22:42:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T22:42:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-mon[26912]: pgmap v169: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:22.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:22 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[26908]: 2024-04-24T22:42:22.071+0000 7fd02ccd4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:42:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:22 smithi192 ceph-mon[33991]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:42:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:22 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T22:42:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:22 smithi192 ceph-mon[33991]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T22:42:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:22 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T22:42:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:22 smithi192 ceph-mon[33991]: pgmap v169: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:22.723 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T22:42:22.992 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:23.016 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T22:42:23.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: fsmap cephfs:0 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: Saving service mds.cephfs spec with placement count:4 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:23.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:23 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T22:42:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: fsmap cephfs:0 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: Saving service mds.cephfs spec with placement count:4 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:23.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:23 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:24.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:24 smithi063 ceph-mon[26912]: Deploying daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:42:24.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:24 smithi063 ceph-mon[26912]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T22:42:24.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:24 smithi063 ceph-mon[26912]: pgmap v173: 65 pgs: 12 creating+peering, 12 active+clean, 41 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:24.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:24 smithi192 ceph-mon[33991]: Deploying daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:42:24.430 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:24 smithi192 ceph-mon[33991]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T22:42:24.430 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:24 smithi192 ceph-mon[33991]: pgmap v173: 65 pgs: 12 creating+peering, 12 active+clean, 41 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:24.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 -- 172.21.15.63:0/1806390304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 msgr2=0x7fceb40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/1806390304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40fc0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004940 tx=0x7fcea8009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 -- 172.21.15.63:0/1806390304 shutdown_connections 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/1806390304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/1806390304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40fc0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 -- 172.21.15.63:0/1806390304 >> 172.21.15.63:0/1806390304 conn(0x7fceb40f7430 msgr2=0x7fceb40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 -- 172.21.15.63:0/1806390304 shutdown_connections 2024-04-24T22:42:24.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.742+0000 7fceb9fd4700 1 -- 172.21.15.63:0/1806390304 wait complete. 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 Processor -- start 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 -- start start 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb40fe7e0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb9fd4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb40fe920 con 0x7fceb40fcf60 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38328/0 (socket says 172.21.15.63:38328) 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb37fe700 1 -- 172.21.15.63:0/4206276160 learned_addr learned my addr 172.21.15.63:0/4206276160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.743+0000 7fceb2ffd700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb37fe700 1 -- 172.21.15.63:0/4206276160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 msgr2=0x7fceb40fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:24.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb37fe700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb37fe700 1 -- 172.21.15.63:0/4206276160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8005040 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb2ffd700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb37fe700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcea800b3b0 tx=0x7fcea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcea800e040 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcea800bcc0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb40feba0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcea801c630 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.744+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb4115ae0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.745+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb4101af0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.746+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcea8013980 con 0x7fceb40fbcb0 2024-04-24T22:42:24.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.746+0000 7fce9bfff700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 0x7fce9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:24.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.746+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcea8087fd0 con 0x7fceb40fbcb0 2024-04-24T22:42:24.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.746+0000 7fceb2ffd700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 0x7fce9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:24.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.747+0000 7fceb2ffd700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 0x7fce9c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009fd0 tx=0x7fcea4009ee0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:24.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:24.751+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcea8054280 con 0x7fceb40fbcb0 2024-04-24T22:42:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.003+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fceb4104930 con 0x7fceb40fbcb0 2024-04-24T22:42:25.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.003+0000 7fce9bfff700 1 -- 172.21.15.63:0/4206276160 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fceb4104930 con 0x7fceb40fbcb0 2024-04-24T22:42:25.006 INFO:teuthology.orchestra.run.smithi063.stdout:e2 2024-04-24T22:42:25.006 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:42:25.006 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 2 2024-04-24T22:42:25.007 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:22.073654+0000 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:42:25.008 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:in 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:up {} 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:42:25.009 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 0 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 msgr2=0x7fce9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 0x7fce9c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009fd0 tx=0x7fcea4009ee0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 msgr2=0x7fceb40ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcea800b3b0 tx=0x7fcea8009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 shutdown_connections 2024-04-24T22:42:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fce9c0692a0 0x7fce9c06b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fceb40fbcb0 0x7fceb40ffd30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 --2- 172.21.15.63:0/4206276160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceb40fcf60 0x7fceb40fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 >> 172.21.15.63:0/4206276160 conn(0x7fceb40f7430 msgr2=0x7fceb4065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.005+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 shutdown_connections 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:25.006+0000 7fceb9fd4700 1 -- 172.21.15.63:0/4206276160 wait complete. 2024-04-24T22:42:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 2 2024-04-24T22:42:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:25 smithi063 ceph-mon[26912]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T22:42:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:25 smithi063 ceph-mon[26912]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T22:42:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:25 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4206276160' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:42:25.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:25 smithi192 ceph-mon[33991]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T22:42:25.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:25 smithi192 ceph-mon[33991]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T22:42:25.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:25 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4206276160' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:42:26.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:26 smithi192 ceph-mon[33991]: pgmap v175: 65 pgs: 12 creating+peering, 24 active+clean, 29 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:26.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:26 smithi063 ceph-mon[26912]: pgmap v175: 65 pgs: 12 creating+peering, 24 active+clean, 29 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:26.483 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:26.492 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:26.492 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-24T22:42:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:27.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:27.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:27 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:27.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:27.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:27.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:27 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 -- 172.21.15.63:0/311165234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 msgr2=0x7fb10c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 --2- 172.21.15.63:0/311165234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c0ffa00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ce0 tx=0x7fb104004d10 comp rx=0 tx=0).stop 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 -- 172.21.15.63:0/311165234 shutdown_connections 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 --2- 172.21.15.63:0/311165234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 0x7fb10c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 --2- 172.21.15.63:0/311165234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c0ffa00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 -- 172.21.15.63:0/311165234 >> 172.21.15.63:0/311165234 conn(0x7fb10c0f7430 msgr2=0x7fb10c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 -- 172.21.15.63:0/311165234 shutdown_connections 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.780+0000 7fb110c80700 1 -- 172.21.15.63:0/311165234 wait complete. 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 Processor -- start 2024-04-24T22:42:28.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 -- start start 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 0x7fb10c10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c10e750 con 0x7fb10c066260 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb110c80700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c10efa0 con 0x7fb10c0fff40 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37806/0 (socket says 172.21.15.63:37806) 2024-04-24T22:42:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 -- 172.21.15.63:0/3715066 learned_addr learned my addr 172.21.15.63:0/3715066 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:28.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb109d9b700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 0x7fb10c10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:28.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 -- 172.21.15.63:0/3715066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 msgr2=0x7fb10c10e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:28.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 0x7fb10c10e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:28.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 -- 172.21.15.63:0/3715066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009940 con 0x7fb10c066260 2024-04-24T22:42:28.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb10a59c700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb104009910 tx=0x7fb104004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.781+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10400e040 con 0x7fb10c066260 2024-04-24T22:42:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.782+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb104008da0 con 0x7fb10c066260 2024-04-24T22:42:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.782+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c10f220 con 0x7fb10c066260 2024-04-24T22:42:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.782+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10401c620 con 0x7fb10c066260 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.782+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c10f760 con 0x7fb10c066260 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.783+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c05f5c0 con 0x7fb10c066260 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.783+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb104012a50 con 0x7fb10c066260 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.783+0000 7fb102ffd700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 0x7fb0f806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.784+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb104088d00 con 0x7fb10c066260 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.784+0000 7fb109d9b700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 0x7fb0f806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.784+0000 7fb109d9b700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 0x7fb0f806b760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004f40 tx=0x7fb0f400b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:28.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:28.788+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb10408c030 con 0x7fb10c066260 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: Deploying daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: pgmap v176: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.63:6826/776980192,v1:172.21.15.63:6827/776980192] up:boot 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: daemon mds.cephfs.smithi063.twfoxu assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: Cluster is now healthy 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: fsmap cephfs:0 1 up:standby 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:creating} 2024-04-24T22:42:28.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:28 smithi063 ceph-mon[26912]: daemon mds.cephfs.smithi063.twfoxu is now active in filesystem cephfs as rank 0 2024-04-24T22:42:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: Deploying daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:42:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: pgmap v176: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:42:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.63:6826/776980192,v1:172.21.15.63:6827/776980192] up:boot 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: daemon mds.cephfs.smithi063.twfoxu assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: Cluster is now healthy 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: fsmap cephfs:0 1 up:standby 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:creating} 2024-04-24T22:42:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:28 smithi192 ceph-mon[33991]: daemon mds.cephfs.smithi063.twfoxu is now active in filesystem cephfs as rank 0 2024-04-24T22:42:29.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.029+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fb10c10fa80 con 0x7fb10c066260 2024-04-24T22:42:29.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.638+0000 7fb102ffd700 1 -- 172.21.15.63:0/3715066 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb104058a40 con 0x7fb10c066260 2024-04-24T22:42:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 msgr2=0x7fb0f806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 0x7fb0f806b760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004f40 tx=0x7fb0f400b040 comp rx=0 tx=0).stop 2024-04-24T22:42:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 msgr2=0x7fb10c10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb104009910 tx=0x7fb104004d10 comp rx=0 tx=0).stop 2024-04-24T22:42:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 shutdown_connections 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb0f80692b0 0x7fb0f806b760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c066260 0x7fb10c10dc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 --2- 172.21.15.63:0/3715066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c0fff40 0x7fb10c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 >> 172.21.15.63:0/3715066 conn(0x7fb10c0f7430 msgr2=0x7fb10c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.641+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 shutdown_connections 2024-04-24T22:42:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:29.642+0000 7fb110c80700 1 -- 172.21.15.63:0/3715066 wait complete. 2024-04-24T22:42:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:29 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.63:6826/776980192,v1:172.21.15.63:6827/776980192] up:active 2024-04-24T22:42:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:29 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 2024-04-24T22:42:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:29 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3715066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T22:42:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:29.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:29 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:30.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:29 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.63:6826/776980192,v1:172.21.15.63:6827/776980192] up:active 2024-04-24T22:42:30.045 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:29 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 2024-04-24T22:42:30.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:29 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3715066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T22:42:30.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:30.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:29 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:30.472 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:30.487 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:30.487 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 271 B/s wr, 2 op/s 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3715066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:boot 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 1 up:standby 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:42:30.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 1 up:standby 2024-04-24T22:42:30.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:30.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:30.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:30.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:30.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:30 smithi063 ceph-mon[26912]: Deploying daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:42:30.882 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 271 B/s wr, 2 op/s 2024-04-24T22:42:30.882 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3715066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T22:42:30.882 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:boot 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 1 up:standby 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 1 up:standby 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:30.883 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:30 smithi192 ceph-mon[33991]: Deploying daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:42:31.812 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:32.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:32 smithi192 ceph-mon[33991]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 599 B/s wr, 3 op/s 2024-04-24T22:42:32.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:32 smithi063 ceph-mon[26912]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 599 B/s wr, 3 op/s 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 -- 172.21.15.63:0/90758905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fcf60 msgr2=0x7f24900fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 --2- 172.21.15.63:0/90758905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fcf60 0x7f24900fd3d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2478004940 tx=0x7f2478009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 -- 172.21.15.63:0/90758905 shutdown_connections 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 --2- 172.21.15.63:0/90758905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fcf60 0x7f24900fd3d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 --2- 172.21.15.63:0/90758905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fbcb0 0x7f24900fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 -- 172.21.15.63:0/90758905 >> 172.21.15.63:0/90758905 conn(0x7f24900f7430 msgr2=0x7f24900f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 -- 172.21.15.63:0/90758905 shutdown_connections 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.649+0000 7f2494c68700 1 -- 172.21.15.63:0/90758905 wait complete. 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 Processor -- start 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 -- start start 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fcf60 0x7f2490112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490066350 con 0x7f24900fbcb0 2024-04-24T22:42:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f2494c68700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490066490 con 0x7f24900fcf60 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37822/0 (socket says 172.21.15.63:37822) 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 -- 172.21.15.63:0/211613448 learned_addr learned my addr 172.21.15.63:0/211613448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248dd9b700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fcf60 0x7f2490112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 -- 172.21.15.63:0/211613448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fcf60 msgr2=0x7f2490112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fcf60 0x7f2490112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 -- 172.21.15.63:0/211613448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478005040 con 0x7f24900fbcb0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.650+0000 7f248e59c700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f248000c830 tx=0x7f2480004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.651+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248000a760 con 0x7f24900fbcb0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.651+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248000ad90 con 0x7f24900fbcb0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.651+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2490113280 con 0x7f24900fbcb0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.651+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24800113a0 con 0x7f24900fbcb0 2024-04-24T22:42:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.651+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490113810 con 0x7f24900fbcb0 2024-04-24T22:42:33.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.652+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249005f5c0 con 0x7f24900fbcb0 2024-04-24T22:42:33.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.652+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2480012400 con 0x7f24900fbcb0 2024-04-24T22:42:33.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.652+0000 7f2486ffd700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 0x7f247c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:33.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.653+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2480087200 con 0x7f24900fbcb0 2024-04-24T22:42:33.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.653+0000 7f248dd9b700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 0x7f247c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:33.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.653+0000 7f248dd9b700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 0x7f247c06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f247800b3b0 tx=0x7f2478009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:33.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.657+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2480053530 con 0x7f24900fbcb0 2024-04-24T22:42:33.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:33.905+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f24901048d0 con 0x7f24900fbcb0 2024-04-24T22:42:34.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.647+0000 7f2486ffd700 1 -- 172.21.15.63:0/211613448 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2480056b50 con 0x7f24900fbcb0 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 msgr2=0x7f247c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 0x7f247c06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f247800b3b0 tx=0x7f2478009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 msgr2=0x7f2490110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f248000c830 tx=0x7f2480004470 comp rx=0 tx=0).stop 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 shutdown_connections 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f247c0692a0 0x7f247c06b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:34.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24900fbcb0 0x7f2490110730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:34.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 --2- 172.21.15.63:0/211613448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24900fcf60 0x7f2490112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:34.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 >> 172.21.15.63:0/211613448 conn(0x7f24900f7430 msgr2=0x7f24900f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:34.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 shutdown_connections 2024-04-24T22:42:34.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:34.650+0000 7f2494c68700 1 -- 172.21.15.63:0/211613448 wait complete. 2024-04-24T22:42:34.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/211613448' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:34.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:34 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:34.855 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T22:42:34.855 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/211613448' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-24T22:42:34.855 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.855 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.856 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:34.856 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:42:34.856 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T22:42:34.856 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:34 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:35.324 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:35.335 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:35.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-24T22:42:35.608 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:35.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:35 smithi063 ceph-mon[26912]: Deploying daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:42:35.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:35 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/211613448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-24T22:42:35.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:35 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] up:boot 2024-04-24T22:42:35.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:35 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 2 up:standby 2024-04-24T22:42:35.921 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:35 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:42:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:35 smithi192 ceph-mon[33991]: Deploying daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:42:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:35 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/211613448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-24T22:42:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:35 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] up:boot 2024-04-24T22:42:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:35 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 2 up:standby 2024-04-24T22:42:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:35 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:42:36.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T22:42:36.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T22:42:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:36.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 -- 172.21.15.63:0/3564042555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280066ad0 msgr2=0x7f1280101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 --2- 172.21.15.63:0/3564042555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280066ad0 0x7f1280101e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1270004940 tx=0x7f1270009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 -- 172.21.15.63:0/3564042555 shutdown_connections 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 --2- 172.21.15.63:0/3564042555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280102370 0x7f1280104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 --2- 172.21.15.63:0/3564042555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280066ad0 0x7f1280101e30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.236+0000 7f12860d8700 1 -- 172.21.15.63:0/3564042555 >> 172.21.15.63:0/3564042555 conn(0x7f12800f7420 msgr2=0x7f12800f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 -- 172.21.15.63:0/3564042555 shutdown_connections 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 -- 172.21.15.63:0/3564042555 wait complete. 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 Processor -- start 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 -- start start 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 0x7f128010fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801108a0 con 0x7f1280102370 2024-04-24T22:42:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.237+0000 7f12860d8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280110fe0 con 0x7f1280066ad0 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 0x7f128010fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37858/0 (socket says 172.21.15.63:37858) 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 0x7f128010fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:40286/0 (socket says 172.21.15.63:40286) 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 -- 172.21.15.63:0/3813771811 learned_addr learned my addr 172.21.15.63:0/3813771811 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 -- 172.21.15.63:0/3813771811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 msgr2=0x7f128010fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 0x7f128010fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 -- 172.21.15.63:0/3813771811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270005040 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f127effd700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f127400c830 tx=0x7f1274004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12740106f0 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1274010d20 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12801112c0 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12740113a0 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.238+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1280111830 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.240+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128005f5c0 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.240+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1274011500 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.240+0000 7f1267fff700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 0x7f126806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.241+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f12740871b0 con 0x7f1280102370 2024-04-24T22:42:37.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.241+0000 7f127f7fe700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 0x7f126806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:37.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.241+0000 7f127f7fe700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 0x7f126806b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1270004890 tx=0x7f1270009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:37.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.246+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1274056f70 con 0x7f1280102370 2024-04-24T22:42:37.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.505+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f128004bed0 con 0x7f1280102370 2024-04-24T22:42:37.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.667+0000 7f1267fff700 1 -- 172.21.15.63:0/3813771811 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7f1274056b00 con 0x7f1280102370 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 msgr2=0x7f126806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 0x7f126806b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1270004890 tx=0x7f1270009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 msgr2=0x7f12801102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f127400c830 tx=0x7f1274004470 comp rx=0 tx=0).stop 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 shutdown_connections 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f12680692b0 0x7f126806b760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1280066ad0 0x7f128010fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 --2- 172.21.15.63:0/3813771811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1280102370 0x7f12801102a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 >> 172.21.15.63:0/3813771811 conn(0x7f12800f7420 msgr2=0x7f12800f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 shutdown_connections 2024-04-24T22:42:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:37.669+0000 7f12860d8700 1 -- 172.21.15.63:0/3813771811 wait complete. 2024-04-24T22:42:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:inline data enabled 2024-04-24T22:42:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:37 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:boot 2024-04-24T22:42:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:37 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:42:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:37 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:42:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:37 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:37.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:37 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3813771811' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:42:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:37 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:boot 2024-04-24T22:42:37.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:37 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:42:37.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:37 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:42:37.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:37 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:37.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:37 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3813771811' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T22:42:38.711 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T22:42:38.711 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:standby 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3813771811' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: Dropping low affinity active daemon mds.cephfs.smithi063.twfoxu in favor of higher affinity standby. 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: Replacing daemon mds.cephfs.smithi063.twfoxu as rank 0 with standby daemon mds.cephfs.smithi192.mnfuxk 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T22:42:38.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:38 smithi063 ceph-mon[26912]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:replay} 2 up:standby 2024-04-24T22:42:38.740 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:42:38.751 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:42:38.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T22:42:38.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:standby 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3813771811' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: Dropping low affinity active daemon mds.cephfs.smithi063.twfoxu in favor of higher affinity standby. 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: Replacing daemon mds.cephfs.smithi063.twfoxu as rank 0 with standby daemon mds.cephfs.smithi192.mnfuxk 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T22:42:38.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:38 smithi192 ceph-mon[33991]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:replay} 2 up:standby 2024-04-24T22:42:39.654 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:reconnect 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] up:boot 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] up:standby 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:reconnect} 3 up:standby 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:42:39.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:39.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:39 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:reconnect 2024-04-24T22:42:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] up:boot 2024-04-24T22:42:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] up:standby 2024-04-24T22:42:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:reconnect} 3 up:standby 2024-04-24T22:42:40.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:42:40.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:40.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:39 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:41.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:40 smithi063 ceph-mon[26912]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-24T22:42:41.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:40 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:rejoin 2024-04-24T22:42:41.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:40 smithi063 ceph-mon[26912]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:rejoin} 3 up:standby 2024-04-24T22:42:41.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:40 smithi063 ceph-mon[26912]: daemon mds.cephfs.smithi192.mnfuxk is now active in filesystem cephfs as rank 0 2024-04-24T22:42:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:40 smithi192 ceph-mon[33991]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-24T22:42:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:40 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:rejoin 2024-04-24T22:42:41.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:40 smithi192 ceph-mon[33991]: fsmap cephfs:1/1 {0=cephfs.smithi192.mnfuxk=up:rejoin} 3 up:standby 2024-04-24T22:42:41.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:40 smithi192 ceph-mon[33991]: daemon mds.cephfs.smithi192.mnfuxk is now active in filesystem cephfs as rank 0 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:active 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:standby 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:41.953 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] up:active 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:standby 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:42.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:42.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 -- 172.21.15.63:0/761324068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140662c0 msgr2=0x7f4f140ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 --2- 172.21.15.63:0/761324068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140662c0 0x7f4f140ff9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f10004940 tx=0x7f4f10009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 -- 172.21.15.63:0/761324068 shutdown_connections 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 --2- 172.21.15.63:0/761324068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140ffee0 0x7f4f14102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 --2- 172.21.15.63:0/761324068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140662c0 0x7f4f140ff9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 -- 172.21.15.63:0/761324068 >> 172.21.15.63:0/761324068 conn(0x7f4f140f7430 msgr2=0x7f4f140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 -- 172.21.15.63:0/761324068 shutdown_connections 2024-04-24T22:42:42.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.446+0000 7f4f1b956700 1 -- 172.21.15.63:0/761324068 wait complete. 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 Processor -- start 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 -- start start 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140662c0 0x7f4f1410da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1410e560 con 0x7f4f140ffee0 2024-04-24T22:42:42.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f1b956700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1410ed40 con 0x7f4f140662c0 2024-04-24T22:42:42.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f18ef1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:42.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f18ef1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58298/0 (socket says 172.21.15.63:58298) 2024-04-24T22:42:42.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f18ef1700 1 -- 172.21.15.63:0/921608555 learned_addr learned my addr 172.21.15.63:0/921608555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:42.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.447+0000 7f4f196f2700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140662c0 0x7f4f1410da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f18ef1700 1 -- 172.21.15.63:0/921608555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140662c0 msgr2=0x7f4f1410da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f18ef1700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140662c0 0x7f4f1410da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f18ef1700 1 -- 172.21.15.63:0/921608555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10005040 con 0x7f4f140ffee0 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f18ef1700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c8c0 tx=0x7f4f040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f0400a760 con 0x7f4f140ffee0 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f0400ad90 con 0x7f4f140ffee0 2024-04-24T22:42:42.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1410f020 con 0x7f4f140ffee0 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f040113a0 con 0x7f4f140ffee0 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.448+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1410f560 con 0x7f4f140ffee0 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.449+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1405f5c0 con 0x7f4f140ffee0 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.450+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4f04012400 con 0x7f4f140ffee0 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.450+0000 7f4f09ffb700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 0x7f4f0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.450+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4f04087200 con 0x7f4f140ffee0 2024-04-24T22:42:42.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.450+0000 7f4f196f2700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 0x7f4f0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:42.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.451+0000 7f4f196f2700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 0x7f4f0006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b970 tx=0x7f4f10009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:42.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.455+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4f04056f70 con 0x7f4f140ffee0 2024-04-24T22:42:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.704+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f1404bed0 con 0x7f4f140ffee0 2024-04-24T22:42:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.705+0000 7f4f09ffb700 1 -- 172.21.15.63:0/921608555 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1809 (secure 0 0 0) 0x7f4f04056b00 con 0x7f4f140ffee0 2024-04-24T22:42:42.707 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:42:42.708 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:42:42.709 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:42.710 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:42:42.711 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:42.711 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:42:42.711 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:42:42.711 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:42:42.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 msgr2=0x7f4f0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:42.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 0x7f4f0006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b970 tx=0x7f4f10009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:42.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 msgr2=0x7f4f1410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:42.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c8c0 tx=0x7f4f040044c0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 shutdown_connections 2024-04-24T22:42:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f4f000692a0 0x7f4f0006b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f140662c0 0x7f4f1410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 --2- 172.21.15.63:0/921608555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f140ffee0 0x7f4f1410df60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 >> 172.21.15.63:0/921608555 conn(0x7f4f140f7430 msgr2=0x7f4f140f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 shutdown_connections 2024-04-24T22:42:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:42.707+0000 7f4f1b956700 1 -- 172.21.15.63:0/921608555 wait complete. 2024-04-24T22:42:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:42:43.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:42 smithi063 ceph-mon[26912]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 819 B/s wr, 3 op/s 2024-04-24T22:42:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:42 smithi192 ceph-mon[33991]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 819 B/s wr, 3 op/s 2024-04-24T22:42:43.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-24T22:42:43.832 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:43.958 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:43 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/921608555' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:42:44.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:43 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/921608555' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:42:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:44 smithi192 ceph-mon[33991]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T22:42:44.959 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:44 smithi063 ceph-mon[26912]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2255635357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 msgr2=0x7f1b840fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2255635357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b840fd270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b74004940 tx=0x7f1b74009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2255635357 shutdown_connections 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2255635357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 0x7f1b840fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2255635357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b840fd270 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2255635357 >> 172.21.15.63:0/2255635357 conn(0x7f1b840f8540 msgr2=0x7f1b840fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2255635357 shutdown_connections 2024-04-24T22:42:45.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2255635357 wait complete. 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 Processor -- start 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.473+0000 7f1b88c4e700 1 -- start start 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b88c4e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b88c4e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 0x7f1b840702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b88c4e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84070860 con 0x7f1b840fce60 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b88c4e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840709d0 con 0x7f1b840fdfd0 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58312/0 (socket says 172.21.15.63:58312) 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 -- 172.21.15.63:0/2987700686 learned_addr learned my addr 172.21.15.63:0/2987700686 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b81d9b700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 0x7f1b840702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 -- 172.21.15.63:0/2987700686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 msgr2=0x7f1b840702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 0x7f1b840702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 -- 172.21.15.63:0/2987700686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b74005040 con 0x7f1b840fce60 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b8259c700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400b3b0 tx=0x7f1b74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.474+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7400e040 con 0x7f1b840fce60 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.475+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84070c50 con 0x7f1b840fce60 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.475+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b7400bbf0 con 0x7f1b840fce60 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.475+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84115a20 con 0x7f1b840fce60 2024-04-24T22:42:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.475+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7401c630 con 0x7f1b840fce60 2024-04-24T22:42:45.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.476+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8405f5c0 con 0x7f1b840fce60 2024-04-24T22:42:45.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.476+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1b7400bd60 con 0x7f1b840fce60 2024-04-24T22:42:45.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.477+0000 7f1b72ffd700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 0x7f1b6c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:45.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.477+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1b74087f10 con 0x7f1b840fce60 2024-04-24T22:42:45.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.477+0000 7f1b81d9b700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 0x7f1b6c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:45.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.477+0000 7f1b81d9b700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 0x7f1b6c06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b780102e0 tx=0x7f1b7800b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:45.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.482+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1b740541f0 con 0x7f1b840fce60 2024-04-24T22:42:45.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.736+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b8404bed0 con 0x7f1b840fce60 2024-04-24T22:42:45.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.737+0000 7f1b72ffd700 1 -- 172.21.15.63:0/2987700686 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4691 (secure 0 0 0) 0x7f1b74057810 con 0x7f1b840fce60 2024-04-24T22:42:45.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.740+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 msgr2=0x7f1b6c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:45.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.740+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 0x7f1b6c06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b780102e0 tx=0x7f1b7800b040 comp rx=0 tx=0).stop 2024-04-24T22:42:45.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.740+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 msgr2=0x7f1b84071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:45.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.740+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400b3b0 tx=0x7f1b74009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 shutdown_connections 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1b6c0692a0 0x7f1b6c06b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840fce60 0x7f1b84071de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 --2- 172.21.15.63:0/2987700686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b840fdfd0 0x7f1b840702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 >> 172.21.15.63:0/2987700686 conn(0x7f1b840f8540 msgr2=0x7f1b840f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 shutdown_connections 2024-04-24T22:42:45.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:45.741+0000 7f1b88c4e700 1 -- 172.21.15.63:0/2987700686 wait complete. 2024-04-24T22:42:45.744 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:42:45.755 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:42:46.391 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.391 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.391 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:46.392 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:46.392 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.392 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:46.392 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 4 op/s 2024-04-24T22:42:46.392 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:46 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2987700686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:42:46.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-24T22:42:46.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:46.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:46.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:46.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:42:46.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 4 op/s 2024-04-24T22:42:46.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:46 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2987700686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:42:47.099 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:48.527 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:48.527 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:48 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:48.527 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:48 smithi063 ceph-mon[26912]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 4 op/s 2024-04-24T22:42:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:48 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:48 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:48 smithi192 ceph-mon[33991]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 4 op/s 2024-04-24T22:42:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- 172.21.15.63:0/535165812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fce80 msgr2=0x7fb9200fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 --2- 172.21.15.63:0/535165812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fce80 0x7fb9200fd290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb914004940 tx=0x7fb914009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- 172.21.15.63:0/535165812 shutdown_connections 2024-04-24T22:42:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 --2- 172.21.15.63:0/535165812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fdff0 0x7fb9200fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 --2- 172.21.15.63:0/535165812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fce80 0x7fb9200fd290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- 172.21.15.63:0/535165812 >> 172.21.15.63:0/535165812 conn(0x7fb9200f8580 msgr2=0x7fb9200fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- 172.21.15.63:0/535165812 shutdown_connections 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- 172.21.15.63:0/535165812 wait complete. 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 Processor -- start 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- start start 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fce80 0x7fb92010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92010e580 con 0x7fb9200fdff0 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.482+0000 7fb924936700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920112f20 con 0x7fb9200fce80 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58778/0 (socket says 172.21.15.63:58778) 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 -- 172.21.15.63:0/2203318160 learned_addr learned my addr 172.21.15.63:0/2203318160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91e59c700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fce80 0x7fb92010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 -- 172.21.15.63:0/2203318160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fce80 msgr2=0x7fb92010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fce80 0x7fb92010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 -- 172.21.15.63:0/2203318160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914005040 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb91dd9b700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b580 tx=0x7fb9100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91000f6f0 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb91000fd20 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920113180 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb910011ea0 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.483+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9201136f0 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.484+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92004c710 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.485+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb91000f850 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.485+0000 7fb90effd700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 0x7fb90806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.485+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb910086e80 con 0x7fb9200fdff0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.485+0000 7fb91e59c700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 0x7fb90806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.486+0000 7fb91e59c700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 0x7fb90806b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb91400b3b0 tx=0x7fb914009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.490+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb910056bf0 con 0x7fb9200fdff0 2024-04-24T22:42:49.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.777+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fb920113bc0 con 0x7fb9200fdff0 2024-04-24T22:42:49.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.777+0000 7fb90effd700 1 -- 172.21.15.63:0/2203318160 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v14) v1 ==== 78+0+99 (secure 0 0 0) 0x7fb910056780 con 0x7fb9200fdff0 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 msgr2=0x7fb90806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 0x7fb90806b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb91400b3b0 tx=0x7fb914009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 msgr2=0x7fb92010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b580 tx=0x7fb9100091a0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 shutdown_connections 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb9080692a0 0x7fb90806b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9200fce80 0x7fb92010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 --2- 172.21.15.63:0/2203318160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9200fdff0 0x7fb92010def0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.779+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 >> 172.21.15.63:0/2203318160 conn(0x7fb9200f8580 msgr2=0x7fb9200f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.780+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 shutdown_connections 2024-04-24T22:42:49.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:49.780+0000 7fb924936700 1 -- 172.21.15.63:0/2203318160 wait complete. 2024-04-24T22:42:49.794 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:42:50.480 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-24T22:42:50.516 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:42:50.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2024213490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fe110 msgr2=0x7fb5c40fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2024213490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fe110 0x7fb5c40fa270 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80048d0 tx=0x7fb5b8009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2024213490 shutdown_connections 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2024213490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fe110 0x7fb5c40fa270 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2024213490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c40fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2024213490 >> 172.21.15.63:0/2024213490 conn(0x7fb5c40f5fc0 msgr2=0x7fb5c40f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2024213490 shutdown_connections 2024-04-24T22:42:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2024213490 wait complete. 2024-04-24T22:42:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 Processor -- start 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- start start 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 0x7fb5c410d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c410d850 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.653+0000 7fb5cbf9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4112210 con 0x7fb5c40fe110 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58790/0 (socket says 172.21.15.63:58790) 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 -- 172.21.15.63:0/2177001713 learned_addr learned my addr 172.21.15.63:0/2177001713 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9538640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 0x7fb5c410d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 -- 172.21.15.63:0/2177001713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 msgr2=0x7fb5c410d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 0x7fb5c410d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 -- 172.21.15.63:0/2177001713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8005040 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9538640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 0x7fb5c410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c9d39640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc002a80 tx=0x7fb5bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5bc007c30 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5bc00f040 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5bc009670 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c41123b0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.654+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41128a0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.655+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c404bcc0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.656+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb5bc00d6b0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.656+0000 7fb5c27fc640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 0x7fb5ac06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.656+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb5bc087160 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.656+0000 7fb5c9538640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 0x7fb5ac06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.657+0000 7fb5c9538640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 0x7fb5ac06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000c00 tx=0x7fb5b8009da0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:50.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.661+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb5bc056e50 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:50 smithi063 ceph-mon[26912]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T22:42:50.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:50 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2203318160' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T22:42:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.838+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5c4104ce0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.839+0000 7fb5c27fc640 1 -- 172.21.15.63:0/2177001713 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14396 (secure 0 0 0) 0x7fb5bc0569e0 con 0x7fb5c40fd7c0 2024-04-24T22:42:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:50.842 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":46,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:42:37.668022+0000","last_up_change":"2024-04-24T22:41:05.837531+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T22:42:21.011439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T22:42:21.501399+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6803","nonce":758545231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6805","nonce":758545231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6809","nonce":758545231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":758545231},{"type":"v1","addr":"172.21.15.63:6807","nonce":758545231}]},"public_addr":"172.21.15.63:6803/758545231","cluster_addr":"172.21.15.63:6805/758545231","heartbeat_back_addr":"172.21.15.63:6809/758545231","heartbeat_front_addr":"172.21.15.63:6807/758545231","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6811","nonce":1039875891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6813","nonce":1039875891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6817","nonce":1039875891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1039875891},{"type":"v1","addr":"172.21.15.63:6815","nonce":1039875891}]},"public_addr":"172.21.15.63:6811/1039875891","cluster_addr":"172.21.15.63:6813/1039875891","heartbeat_back_addr":"172.21.15.63:6817/1039875891","heartbeat_front_addr":"172.21.15.63:6815/1039875891","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6819","nonce":2431699450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6821","nonce":2431699450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6825","nonce":2431699450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2431699450},{"type":"v1","addr":"172.21.15.63:6823","nonce":2431699450}]},"public_addr":"172.21.15.63:6819/2431699450","cluster_addr":"172.21.15.63:6821/2431699450","heartbeat_back_addr":"172.21.15.63:6825/2431699450","heartbeat_front_addr":"172.21.15.63:6823/2431699450","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6801","nonce":2771944170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6803","nonce":2771944170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6807","nonce":2771944170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2771944170},{"type":"v1","addr":"172.21.15.192:6805","nonce":2771944170}]},"public_addr":"172.21.15.192:6801/2771944170","cluster_addr":"172.21.15.192:6803/2771944170","heartbeat_back_addr":"172.21.15.192:6807/2771944170","heartbeat_front_addr":"172.21.15.192:6805/2771944170","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6809","nonce":1725358058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6811","nonce":1725358058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6815","nonce":1725358058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1725358058},{"type":"v1","addr":"172.21.15.192:6813","nonce":1725358058}]},"public_addr":"172.21.15.192:6809/1725358058","cluster_addr":"172.21.15.192:6811/1725358058","heartbeat_back_addr":"172.21.15.192:6815/1725358058","heartbeat_front_addr":"172.21.15.192:6813/1725358058","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6817","nonce":2882470257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6819","nonce":2882470257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6823","nonce":2882470257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2882470257},{"type":"v1","addr":"172.21.15.192:6821","nonce":2882470257}]},"public_addr":"172.21.15.192:6817/2882470257","cluster_addr":"172.21.15.192:6819/2882470257","heartbeat_back_addr":"172.21.15.192:6823/2882470257","heartbeat_front_addr":"172.21.15.192:6821/2882470257","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6827/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6826/776980192":"2024-04-25T22:42:37.667991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:42:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 msgr2=0x7fb5ac06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 0x7fb5ac06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000c00 tx=0x7fb5b8009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 msgr2=0x7fb5c410cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc002a80 tx=0x7fb5bc004600 comp rx=0 tx=0).stop 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 shutdown_connections 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fb5ac069210 0x7fb5ac06b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c40fe110 0x7fb5c410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 --2- 172.21.15.63:0/2177001713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c40fd7c0 0x7fb5c410cc90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 >> 172.21.15.63:0/2177001713 conn(0x7fb5c40f5fc0 msgr2=0x7fb5c40f7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 shutdown_connections 2024-04-24T22:42:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.841+0000 7fb5cbf9d640 1 -- 172.21.15.63:0/2177001713 wait complete. 2024-04-24T22:42:50.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:54.076844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T22:42:21.011439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T22:42:21.501399+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T22:42:50.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:42:50.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:50 smithi192 ceph-mon[33991]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T22:42:50.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:50 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2203318160' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 -- 172.21.15.63:0/4101287667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 msgr2=0x7fda200fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 --2- 172.21.15.63:0/4101287667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda200fc610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda180048d0 tx=0x7fda18009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 -- 172.21.15.63:0/4101287667 shutdown_connections 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 --2- 172.21.15.63:0/4101287667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 0x7fda200fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 --2- 172.21.15.63:0/4101287667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda200fc610 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 -- 172.21.15.63:0/4101287667 >> 172.21.15.63:0/4101287667 conn(0x7fda200f7900 msgr2=0x7fda200f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 -- 172.21.15.63:0/4101287667 shutdown_connections 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 -- 172.21.15.63:0/4101287667 wait complete. 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.964+0000 7fda24d9d640 1 Processor -- start 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 -- start start 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 0x7fda2010d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2010d890 con 0x7fda200fc200 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda20112250 con 0x7fda200fd430 2024-04-24T22:42:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58794/0 (socket says 172.21.15.63:58794) 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 -- 172.21.15.63:0/1511766879 learned_addr learned my addr 172.21.15.63:0/1511766879 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1dd9b640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 0x7fda2010d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 -- 172.21.15.63:0/1511766879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 msgr2=0x7fda2010d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 0x7fda2010d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 -- 172.21.15.63:0/1511766879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda18005040 con 0x7fda200fc200 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda1e59c640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fda180048a0 tx=0x7fda18009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda180094c0 con 0x7fda200fc200 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda18007cc0 con 0x7fda200fc200 2024-04-24T22:42:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda20112390 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda1803b400 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.965+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda20112860 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.966+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2004bcc0 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.967+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fda1802f070 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.967+0000 7fda16ffd640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 0x7fda0c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.967+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fda180a9630 con 0x7fda200fc200 2024-04-24T22:42:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.967+0000 7fda1dd9b640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 0x7fda0c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.968+0000 7fda1dd9b640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 0x7fda0c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda08004a70 tx=0x7fda0800b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:50.972+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fda18075910 con 0x7fda200fc200 2024-04-24T22:42:51.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.151+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda20112d70 con 0x7fda200fc200 2024-04-24T22:42:51.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.152+0000 7fda16ffd640 1 -- 172.21.15.63:0/1511766879 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda18078f30 con 0x7fda200fc200 2024-04-24T22:42:51.155 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-04-24T22:42:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.153+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 msgr2=0x7fda0c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.153+0000 7fda24d9d640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 0x7fda0c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda08004a70 tx=0x7fda0800b040 comp rx=0 tx=0).stop 2024-04-24T22:42:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.153+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 msgr2=0x7fda2010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.153+0000 7fda24d9d640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fda180048a0 tx=0x7fda18009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 shutdown_connections 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda0c069210 0x7fda0c06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda200fd430 0x7fda2010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 --2- 172.21.15.63:0/1511766879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda200fc200 0x7fda2010cd00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 >> 172.21.15.63:0/1511766879 conn(0x7fda200f7900 msgr2=0x7fda200f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 shutdown_connections 2024-04-24T22:42:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.154+0000 7fda24d9d640 1 -- 172.21.15.63:0/1511766879 wait complete. 2024-04-24T22:42:51.167 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T22:42:51.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 -- 172.21.15.63:0/3783398524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fb060 msgr2=0x7f87040fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 --2- 172.21.15.63:0/3783398524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fb060 0x7f87040fb470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f86f80048d0 tx=0x7f86f8009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 -- 172.21.15.63:0/3783398524 shutdown_connections 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 --2- 172.21.15.63:0/3783398524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fc330 0x7f87040fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 --2- 172.21.15.63:0/3783398524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fb060 0x7f87040fb470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 -- 172.21.15.63:0/3783398524 >> 172.21.15.63:0/3783398524 conn(0x7f87040f6820 msgr2=0x7f87040f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 -- 172.21.15.63:0/3783398524 shutdown_connections 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.273+0000 7f870b06e640 1 -- 172.21.15.63:0/3783398524 wait complete. 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 Processor -- start 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 -- start start 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fb060 0x7f870410cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870410d860 con 0x7f87040fc330 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f870b06e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8704112220 con 0x7f87040fb060 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58816/0 (socket says 172.21.15.63:58816) 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 -- 172.21.15.63:0/1448242735 learned_addr learned my addr 172.21.15.63:0/1448242735 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8708e0a640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fb060 0x7f870410cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 -- 172.21.15.63:0/1448242735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fb060 msgr2=0x7f870410cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fb060 0x7f870410cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 -- 172.21.15.63:0/1448242735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8005040 con 0x7f87040fc330 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.274+0000 7f8703fff640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86f000eba0 tx=0x7f86f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.275+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f0007c30 con 0x7f87040fc330 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.275+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86f000f040 con 0x7f87040fc330 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.275+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8704112360 con 0x7f87040fc330 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.275+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f00095b0 con 0x7f87040fc330 2024-04-24T22:42:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.275+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8704112850 con 0x7f87040fc330 2024-04-24T22:42:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.276+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870404bcc0 con 0x7f87040fc330 2024-04-24T22:42:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.276+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f86f000c710 con 0x7f87040fc330 2024-04-24T22:42:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.276+0000 7f87017fa640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 0x7f86f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.277+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f86f0086ec0 con 0x7f87040fc330 2024-04-24T22:42:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.277+0000 7f8708e0a640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 0x7f86f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.277+0000 7f8708e0a640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 0x7f86f406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86f8000c00 tx=0x7f86f8009da0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.281+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86f00531d0 con 0x7f87040fc330 2024-04-24T22:42:51.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.456+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8704112aa0 con 0x7f87040fc330 2024-04-24T22:42:51.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.457+0000 7f87017fa640 1 -- 172.21.15.63:0/1448242735 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f86f00567f0 con 0x7f87040fc330 2024-04-24T22:42:51.459 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 msgr2=0x7f86f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 0x7f86f406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86f8000c00 tx=0x7f86f8009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 msgr2=0x7f870410d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86f000eba0 tx=0x7f86f000b040 comp rx=0 tx=0).stop 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 shutdown_connections 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87040fc330 0x7f870410d1e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f86f4069210 0x7f86f406b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 --2- 172.21.15.63:0/1448242735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87040fb060 0x7f870410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 >> 172.21.15.63:0/1448242735 conn(0x7f87040f6820 msgr2=0x7f87040f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 shutdown_connections 2024-04-24T22:42:51.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.459+0000 7f870b06e640 1 -- 172.21.15.63:0/1448242735 wait complete. 2024-04-24T22:42:51.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T22:42:51.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 -- 172.21.15.63:0/751509169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fb070 msgr2=0x7f83900fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 --2- 172.21.15.63:0/751509169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fb070 0x7f83900fb480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83840048d0 tx=0x7f8384009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 -- 172.21.15.63:0/751509169 shutdown_connections 2024-04-24T22:42:51.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 --2- 172.21.15.63:0/751509169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fc2a0 0x7f83900fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 --2- 172.21.15.63:0/751509169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fb070 0x7f83900fb480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 -- 172.21.15.63:0/751509169 >> 172.21.15.63:0/751509169 conn(0x7f83900f6770 msgr2=0x7f83900f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 -- 172.21.15.63:0/751509169 shutdown_connections 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.595+0000 7f8397061640 1 -- 172.21.15.63:0/751509169 wait complete. 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 Processor -- start 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 -- start start 2024-04-24T22:42:51.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 0x7f8390065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390112040 con 0x7f83900fc2a0 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8397061640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390112180 con 0x7f83900fb070 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58832/0 (socket says 172.21.15.63:58832) 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 -- 172.21.15.63:0/4191667947 learned_addr learned my addr 172.21.15.63:0/4191667947 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8394dfd640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 0x7f8390065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 -- 172.21.15.63:0/4191667947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 msgr2=0x7f8390065ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 0x7f8390065ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f838ffff640 1 -- 172.21.15.63:0/4191667947 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384005040 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.596+0000 7f8394dfd640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 0x7f8390065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f838ffff640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f837c002a50 tx=0x7f837c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c007c30 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f837c00f040 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901123e0 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c00db50 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.597+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8390112900 con 0x7f83900fc2a0 2024-04-24T22:42:51.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.598+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8390105570 con 0x7f83900fc2a0 2024-04-24T22:42:51.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.599+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f837c01c400 con 0x7f83900fc2a0 2024-04-24T22:42:51.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.599+0000 7f838d7fa640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 0x7f838006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.599+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f837c086b20 con 0x7f83900fc2a0 2024-04-24T22:42:51.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.599+0000 7f8394dfd640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 0x7f838006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.600+0000 7f8394dfd640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 0x7f838006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f838402bda0 tx=0x7f8384009da0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.604+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f837c04d250 con 0x7f83900fc2a0 2024-04-24T22:42:51.787 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:51 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2177001713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:42:51.787 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:51 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1511766879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:42:51.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:51 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1448242735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:42:51.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.785+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8390103be0 con 0x7f83900fc2a0 2024-04-24T22:42:51.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.785+0000 7f838d7fa640 1 -- 172.21.15.63:0/4191667947 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f837c052d80 con 0x7f83900fc2a0 2024-04-24T22:42:51.788 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 msgr2=0x7f838006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 0x7f838006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f838402bda0 tx=0x7f8384009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 msgr2=0x7f8390111b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f837c002a50 tx=0x7f837c004600 comp rx=0 tx=0).stop 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 shutdown_connections 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83900fc2a0 0x7f8390111b00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8380069210 0x7f838006b6c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 --2- 172.21.15.63:0/4191667947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83900fb070 0x7f8390065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.788+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 >> 172.21.15.63:0/4191667947 conn(0x7f83900f6770 msgr2=0x7f83900ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.789+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 shutdown_connections 2024-04-24T22:42:51.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.789+0000 7f8397061640 1 -- 172.21.15.63:0/4191667947 wait complete. 2024-04-24T22:42:51.806 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:42:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/2712579786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 msgr2=0x7fc2a40fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/2712579786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fed70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2940048d0 tx=0x7fc294009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/2712579786 shutdown_connections 2024-04-24T22:42:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/2712579786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 0x7fc2a4101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/2712579786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fed70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/2712579786 >> 172.21.15.63:0/2712579786 conn(0x7fc2a40f6820 msgr2=0x7fc2a40f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/2712579786 shutdown_connections 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/2712579786 wait complete. 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.906+0000 7fc2ac0ae640 1 Processor -- start 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2ac0ae640 1 -- start start 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2ac0ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2ac0ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 0x7fc2a40fac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2ac0ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40fcbf0 con 0x7fc2a4065010 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2ac0ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40fb1b0 con 0x7fc2a40ff2b0 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58844/0 (socket says 172.21.15.63:58844) 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 -- 172.21.15.63:0/3487966692 learned_addr learned my addr 172.21.15.63:0/3487966692 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9649640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 0x7fc2a40fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 -- 172.21.15.63:0/3487966692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 msgr2=0x7fc2a40fac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 0x7fc2a40fac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 -- 172.21.15.63:0/3487966692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294005040 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc2a9e4a640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc2940048a0 tx=0x7fc294009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.907+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2940094c0 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.908+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc294007cc0 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.908+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a40fb430 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.908+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29403b400 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.908+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a40fb950 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.909+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a404bcc0 con 0x7fc2a4065010 2024-04-24T22:42:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.909+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc29402f070 con 0x7fc2a4065010 2024-04-24T22:42:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.909+0000 7fc29a7fc640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 0x7fc29006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.909+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc2940a9670 con 0x7fc2a4065010 2024-04-24T22:42:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.909+0000 7fc2a9649640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 0x7fc29006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.910+0000 7fc2a9649640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 0x7fc29006b6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40fc350 tx=0x7fc2a000b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:51.914+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc2940758d0 con 0x7fc2a4065010 2024-04-24T22:42:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:51 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2177001713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:42:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:51 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1511766879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:42:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:51 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1448242735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:42:52.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.127+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a4104cd0 con 0x7fc2a4065010 2024-04-24T22:42:52.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.128+0000 7fc29a7fc640 1 -- 172.21.15.63:0/3487966692 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4691 (secure 0 0 0) 0x7fc294078ef0 con 0x7fc2a4065010 2024-04-24T22:42:52.131 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:42:52.131 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.63:6829/3016441784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3016441784},{"type":"v1","addr":"172.21.15.63:6829","nonce":3016441784}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":14482,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/3569757524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3569757524},{"type":"v1","addr":"172.21.15.63:6827","nonce":3569757524}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:42:40.679489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi192.mnfuxk","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6825/780449385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":780449385},{"type":"v1","addr":"172.21.15.192:6825","nonce":780449385}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-24T22:42:52.131 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:42:52.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 msgr2=0x7fc29006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 0x7fc29006b6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40fc350 tx=0x7fc2a000b040 comp rx=0 tx=0).stop 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 msgr2=0x7fc2a40fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc2940048a0 tx=0x7fc294009da0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 shutdown_connections 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fc290069210 0x7fc29006b6c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a40ff2b0 0x7fc2a40fac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 --2- 172.21.15.63:0/3487966692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a4065010 0x7fc2a40fc5a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:52.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 >> 172.21.15.63:0/3487966692 conn(0x7fc2a40f6820 msgr2=0x7fc2a40f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:52.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 shutdown_connections 2024-04-24T22:42:52.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:52.130+0000 7fc2ac0ae640 1 -- 172.21.15.63:0/3487966692 wait complete. 2024-04-24T22:42:52.145 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 14, 'max_mds': 1, 'flags': 18} 2024-04-24T22:42:52.145 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T22:42:52.190 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T22:42:52.190 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-24T22:42:52.191 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T22:42:52.191 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:42:52.191 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:42:52.191 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-24T22:42:52.191 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:42:52.191 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T22:42:52.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:52.191 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-04-24T22:42:52.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:52.218 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-04-24T22:42:52.290 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-04-24T22:42:52.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:42:52.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:52.292 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-04-24T22:42:52.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:52.309 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-04-24T22:42:52.378 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-04-24T22:42:52.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:42:52.380 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T22:42:52.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T22:42:52.699 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4191667947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3487966692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:52.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T22:42:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4191667947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3487966692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:42:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 -- 172.21.15.63:0/4096748561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798102e50 msgr2=0x7f8798103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 --2- 172.21.15.63:0/4096748561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798102e50 0x7f8798103220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8780004940 tx=0x7f8780009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 -- 172.21.15.63:0/4096748561 shutdown_connections 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 --2- 172.21.15.63:0/4096748561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798065640 0x7f8798065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 --2- 172.21.15.63:0/4096748561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798102e50 0x7f8798103220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 -- 172.21.15.63:0/4096748561 >> 172.21.15.63:0/4096748561 conn(0x7f8798071d00 msgr2=0x7f8798072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.264+0000 7f879e454700 1 -- 172.21.15.63:0/4096748561 shutdown_connections 2024-04-24T22:42:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.265+0000 7f879e454700 1 -- 172.21.15.63:0/4096748561 wait complete. 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 Processor -- start 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 -- start start 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798102e50 0x7f87981140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981147c0 con 0x7f8798065640 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f879e454700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879810db20 con 0x7f8798102e50 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f8797fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f8797fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58856/0 (socket says 172.21.15.63:58856) 2024-04-24T22:42:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.266+0000 7f8797fff700 1 -- 172.21.15.63:0/2929215901 learned_addr learned my addr 172.21.15.63:0/2929215901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f87977fe700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798102e50 0x7f87981140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8797fff700 1 -- 172.21.15.63:0/2929215901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798102e50 msgr2=0x7f87981140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8797fff700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798102e50 0x7f87981140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8797fff700 1 -- 172.21.15.63:0/2929215901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780005040 con 0x7f8798065640 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8797fff700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f878000b8b0 tx=0x7f8780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878000e040 con 0x7f8798065640 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879810dda0 con 0x7f8798065640 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8780004290 con 0x7f8798065640 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878001c3f0 con 0x7f8798065640 2024-04-24T22:42:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.267+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879810e360 con 0x7f8798065640 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.268+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87980fe540 con 0x7f8798065640 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.269+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f87800137b0 con 0x7f8798065640 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.269+0000 7f8794ff9700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 0x7f878406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.269+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8780088550 con 0x7f8798065640 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.269+0000 7f87977fe700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 0x7f878406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.270+0000 7f87977fe700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 0x7f878406fb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8788006fd0 tx=0x7f8788008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:54.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.274+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f87800582c0 con 0x7f8798065640 2024-04-24T22:42:54.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.515+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f879804bed0 con 0x7f8798065640 2024-04-24T22:42:54.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.516+0000 7f8794ff9700 1 -- 172.21.15.63:0/2929215901 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8780057e50 con 0x7f8798065640 2024-04-24T22:42:54.518 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 msgr2=0x7f878406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 0x7f878406fb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8788006fd0 tx=0x7f8788008040 comp rx=0 tx=0).stop 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 msgr2=0x7f8798113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f878000b8b0 tx=0x7f8780009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 shutdown_connections 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f878406d6b0 0x7f878406fb60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8798065640 0x7f8798113ba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 --2- 172.21.15.63:0/2929215901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8798102e50 0x7f87981140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 >> 172.21.15.63:0/2929215901 conn(0x7f8798071d00 msgr2=0x7f87980fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 shutdown_connections 2024-04-24T22:42:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:54.518+0000 7f879e454700 1 -- 172.21.15.63:0/2929215901 wait complete. 2024-04-24T22:42:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:54 smithi063 ceph-mon[26912]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T22:42:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:54 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2929215901' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:42:55.161 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:42:55.162 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:42:55.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:42:55.162 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:42:55.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:55.162 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-04-24T22:42:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:54 smithi192 ceph-mon[33991]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T22:42:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:54 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2929215901' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:42:55.178 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42235sec preferred_lft 42235sec 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::2474:b7ab:3c0a:9ab5/64 scope link noprefixroute 2024-04-24T22:42:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:55.180 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:42:55.180 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:55.180 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-04-24T22:42:55.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:42:55.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-04-24T22:42:55.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-04-24T22:42:55.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:42:55.181 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-04-24T22:42:55.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:55.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:55.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:55.364 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:42:55.389 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-04-24T22:42:55.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:55.391 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-04-24T22:42:55.463 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-04-24T22:42:55.464 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:42:55.464 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T22:42:55.464 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T22:42:55.464 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:42:55.464 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:55.464 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-04-24T22:42:55.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T22:42:55.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T22:42:55.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T22:42:55.465 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-04-24T22:42:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:55.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:55.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:55.737 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-04-24T22:42:55.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:55.753 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-04-24T22:42:55.808 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:55.808 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-04-24T22:42:55.808 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:42:55.808 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T22:42:55.808 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-04-24T22:42:55.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:55.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:55.925 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T22:42:55.926 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-04-24T22:42:55.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:56.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:56.054 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:56.054 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-04-24T22:42:56.054 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-04-24T22:42:56.054 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:42:56.054 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-04-24T22:42:56.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:56.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:56.131 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T22:42:56.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:42:56.131 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T22:42:56.146 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:42:56.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:42:56.146 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:42:56.201 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-04-24T22:42:56.267 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-04-24T22:42:56.323 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T22:42:56.324 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8021f813c5a4064af2b6e17a70eaad129a1f1b22d4fd269d9a8393cc9c01cc63/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b72bef2ec388bd9e6eb96b3692233b26e64b4b88c15d2cbd53c8c432c4efdb0f/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/c9a6b87ab28f15842d1e64a15f54f05cd8775dcf326de2ed8eb78dc08fad53b8/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7a4ea5ff563b171d28bb3f0618fdb98054ae4a6f008a156657a42fdd45d8012d/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/89003462ff42d340906c2113c267c93caf3d518f6bc647336e4f82eb45e093f9/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/79949af0aaff035a92f2174e37887a08ebc7bc9a8fe5c0375e193c212ddda299/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b4a53b6b1f0dd5a30de1defb9566dd8003badb32f6fce90cc62ceb10da9fcd22/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7923d1f0b9000084c5115fad283b1b18b9a9b2a182fab94ff6c8d74acffc05db/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/3f34a82869c8fdbf001837783f5edca3266670d54a95671b7e4351dafb38ae56/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d55dd9f9b66b1850b5dca67263ed383070e37325604a731a6c83b0e4769e524/merged 2024-04-24T22:42:56.325 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1e175de6253c6bd2334d8efbd2b8d61113ab5c758a70d8b5771149322ecdcda5/merged 2024-04-24T22:42:56.326 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1aedbafb8adf589473168db0faccecbaac4d835cdbb41b3cca5744b17f4a8cb6/merged 2024-04-24T22:42:56.326 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-04-24T22:42:56.326 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:42:56.326 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:42:56.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:56.326 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-04-24T22:42:56.380 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T22:42:56.380 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T22:42:56.423 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-04-24T22:42:56.446 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T22:42:56.502 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:42:56.503 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/8021f813c5a4064af2b6e17a70eaad129a1f1b22d4fd269d9a8393cc9c01cc63/merged 2024-04-24T22:42:56.504 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b72bef2ec388bd9e6eb96b3692233b26e64b4b88c15d2cbd53c8c432c4efdb0f/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/c9a6b87ab28f15842d1e64a15f54f05cd8775dcf326de2ed8eb78dc08fad53b8/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7a4ea5ff563b171d28bb3f0618fdb98054ae4a6f008a156657a42fdd45d8012d/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/89003462ff42d340906c2113c267c93caf3d518f6bc647336e4f82eb45e093f9/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/79949af0aaff035a92f2174e37887a08ebc7bc9a8fe5c0375e193c212ddda299/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b4a53b6b1f0dd5a30de1defb9566dd8003badb32f6fce90cc62ceb10da9fcd22/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7923d1f0b9000084c5115fad283b1b18b9a9b2a182fab94ff6c8d74acffc05db/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/3f34a82869c8fdbf001837783f5edca3266670d54a95671b7e4351dafb38ae56/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d55dd9f9b66b1850b5dca67263ed383070e37325604a731a6c83b0e4769e524/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1e175de6253c6bd2334d8efbd2b8d61113ab5c758a70d8b5771149322ecdcda5/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1aedbafb8adf589473168db0faccecbaac4d835cdbb41b3cca5744b17f4a8cb6/merged 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:42:56.505 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:42:56.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:56.506 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-04-24T22:42:56.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:2024-04-24T22:42:56.510+0000 7fe6d5ac1300 -1 init, newargv = 0x55bc05b11920 newargc=15 2024-04-24T22:42:56.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[74317]: starting ceph client 2024-04-24T22:42:56.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[74317]: starting fuse 2024-04-24T22:42:56.528 INFO:teuthology.orchestra.run.smithi063.stdout:134 2024-04-24T22:42:56.529 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [134] 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T22:42:56.529 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T22:42:56.530 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.0")) 2024-04-24T22:42:56.531 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-04-24T22:42:56.652 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.0.74317.asok 2024-04-24T22:42:56.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:56.663 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74317.asok 2024-04-24T22:42:56.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:56.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74317.asok status 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "0", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "74317", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 14522, 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 14522 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3741521470 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3741521470 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.14522 192.168.0.1:0/3741521470", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/3741521470", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 14, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 46, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:42:56.804 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T22:42:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:56 smithi192 ceph-mon[33991]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-24T22:42:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:56 smithi063 ceph-mon[26912]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-24T22:42:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:42:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:42:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/1999100488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 msgr2=0x7f7c480fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/1999100488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480fcdf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c004940 tx=0x7f7c3c009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/1999100488 shutdown_connections 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/1999100488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480fcdf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/1999100488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 0x7f7c481031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/1999100488 >> 172.21.15.63:0/1999100488 conn(0x7f7c480f8990 msgr2=0x7f7c480fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/1999100488 shutdown_connections 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.657+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/1999100488 wait complete. 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 Processor -- start 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 -- start start 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 0x7f7c48070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c480721b0 con 0x7f7c48103780 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c4d3a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48070ce0 con 0x7f7c48102de0 2024-04-24T22:42:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45366/0 (socket says 172.21.15.63:45366) 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 -- 172.21.15.63:0/3542365788 learned_addr learned my addr 172.21.15.63:0/3542365788 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c46ffd700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 0x7f7c48070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 -- 172.21.15.63:0/3542365788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 msgr2=0x7f7c48070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 0x7f7c48070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.658+0000 7f7c467fc700 1 -- 172.21.15.63:0/3542365788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c005040 con 0x7f7c48103780 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c467fc700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009b30 tx=0x7f7c3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c021070 con 0x7f7c48103780 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3c013cd0 con 0x7f7c48103780 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c48070f60 con 0x7f7c48103780 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c00e070 con 0x7f7c48103780 2024-04-24T22:42:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.659+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c48119fd0 con 0x7f7c48103780 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.660+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c480fe530 con 0x7f7c48103780 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.660+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7c3c01cad0 con 0x7f7c48103780 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.661+0000 7f7c2f7fe700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 0x7f7c3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.661+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7c3c088630 con 0x7f7c48103780 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.661+0000 7f7c46ffd700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 0x7f7c3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.661+0000 7f7c46ffd700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 0x7f7c3006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009b80 tx=0x7f7c3800b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.665+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c3c054910 con 0x7f7c48103780 2024-04-24T22:42:58.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:58 smithi063 ceph-mon[26912]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T22:42:58.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.907+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7c48109ec0 con 0x7f7c48103780 2024-04-24T22:42:58.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.907+0000 7f7c2f7fe700 1 -- 172.21.15.63:0/3542365788 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f7c3c004490 con 0x7f7c48103780 2024-04-24T22:42:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 msgr2=0x7f7c3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 0x7f7c3006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009b80 tx=0x7f7c3800b040 comp rx=0 tx=0).stop 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 msgr2=0x7f7c480707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009b30 tx=0x7f7c3c009e30 comp rx=0 tx=0).stop 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 shutdown_connections 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f7c300692a0 0x7f7c3006b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c48102de0 0x7f7c48070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 --2- 172.21.15.63:0/3542365788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c48103780 0x7f7c480707a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 >> 172.21.15.63:0/3542365788 conn(0x7f7c480f8990 msgr2=0x7f7c480f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 shutdown_connections 2024-04-24T22:42:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:42:58.909+0000 7f7c4d3a8700 1 -- 172.21.15.63:0/3542365788 wait complete. 2024-04-24T22:42:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:58 smithi192 ceph-mon[33991]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:42:59.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:42:59.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:42:59.554 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T22:42:59.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:59.554 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-04-24T22:42:59.574 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:42:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:42:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:42:59.574 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T22:42:59.575 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42209sec preferred_lft 42209sec 2024-04-24T22:42:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::3e8d:c0db:3d2e:ce80/64 scope link noprefixroute 2024-04-24T22:42:59.576 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:42:59.576 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:42:59.576 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-24T22:42:59.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:59.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:42:59.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:59.776 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:42:59.800 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-04-24T22:42:59.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:42:59.802 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-04-24T22:42:59.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:42:59 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3542365788' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:42:59.872 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-04-24T22:42:59.872 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:42:59.872 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T22:42:59.872 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T22:42:59.872 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:42:59.873 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-24T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-24T22:42:59.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:42:59.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:42:59 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3542365788' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T22:43:00.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:43:00.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:00.192 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-04-24T22:43:00.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:00.207 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-04-24T22:43:00.262 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:43:00.262 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-24T22:43:00.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:43:00.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T22:43:00.262 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-24T22:43:00.351 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:43:00.374 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:43:00.380 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T22:43:00.380 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:43:00.380 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-24T22:43:00.380 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T22:43:00.380 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:43:00.380 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T22:43:00.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T22:43:00.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T22:43:00.381 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-24T22:43:00.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:43:00.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:43:00.506 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:43:00.506 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-24T22:43:00.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-04-24T22:43:00.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:43:00.506 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-24T22:43:00.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:43:00.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:43:00.584 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-24T22:43:00.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:43:00.585 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:00.599 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T22:43:00.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:43:00.599 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:00.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-04-24T22:43:00.721 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T22:43:00.775 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-04-24T22:43:00.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:43:00.776 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-04-24T22:43:00.777 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/0 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/20236c6eb2b99a86dd61fcef3eb5d0e5e6151c452a90ef34ba47b6f8c7233e95/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/2c4a1a4e45ae57eb2052ef5719ae4869a3fadc7cec0009e66abbaee5114439bd/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/04e32b5cc7f98c8209535b8209e265b73755b60aaa1754d618161071ae4c198f/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/54b306ea7429a3bab483cdf7442b5c074723ca8d724c258acdbcd628f7282686/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c120df2778ad494ee8f448c658cc590a07dc8ffe91c986bd5e38fcf8bfc867d9/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/ac025d57631ee624a1d858f222197bdd22bc89ff59d4e06661d787377dd2c75a/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/2a61c1bb6e6357ad1fed1667edb89654dd910d492205c1221105d6797fcfa253/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/29aa5c93a0f9908e49926e42c31669edce74f09b2bb05e9df7cf60dfc355ad34/merged 2024-04-24T22:43:00.778 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/4e3fd7004970a7e3e8843641074168069c1af5a02f9afdd2f7929a9d1f65e9d0/merged 2024-04-24T22:43:00.779 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-04-24T22:43:00.779 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:43:00.779 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:43:00.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:00.779 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-04-24T22:43:00.832 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T22:43:00.832 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-24T22:43:00.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-04-24T22:43:00.892 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:00 smithi192 ceph-mon[33991]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T22:43:00.897 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T22:43:00.952 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-04-24T22:43:00.953 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T22:43:00.954 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/0 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/20236c6eb2b99a86dd61fcef3eb5d0e5e6151c452a90ef34ba47b6f8c7233e95/merged 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/2c4a1a4e45ae57eb2052ef5719ae4869a3fadc7cec0009e66abbaee5114439bd/merged 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/04e32b5cc7f98c8209535b8209e265b73755b60aaa1754d618161071ae4c198f/merged 2024-04-24T22:43:00.955 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/54b306ea7429a3bab483cdf7442b5c074723ca8d724c258acdbcd628f7282686/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/c120df2778ad494ee8f448c658cc590a07dc8ffe91c986bd5e38fcf8bfc867d9/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/ac025d57631ee624a1d858f222197bdd22bc89ff59d4e06661d787377dd2c75a/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/2a61c1bb6e6357ad1fed1667edb89654dd910d492205c1221105d6797fcfa253/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/29aa5c93a0f9908e49926e42c31669edce74f09b2bb05e9df7cf60dfc355ad34/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/containers/storage/overlay/4e3fd7004970a7e3e8843641074168069c1af5a02f9afdd2f7929a9d1f65e9d0/merged 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-04-24T22:43:00.956 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:43:00.957 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T22:43:00.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:00.957 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-04-24T22:43:00.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2024-04-24T22:43:00.971+0000 7f243f491300 -1 init, newargv = 0x55c79232ba40 newargc=15 2024-04-24T22:43:00.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[58691]: starting ceph client 2024-04-24T22:43:00.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[58691]: starting fuse 2024-04-24T22:43:00.989 INFO:teuthology.orchestra.run.smithi192.stdout:112 2024-04-24T22:43:00.989 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-24T22:43:00.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-04-24T22:43:00.989 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-04-24T22:43:00.989 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-04-24T22:43:00.989 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-04-24T22:43:00.989 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T22:43:00.990 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2024-04-24T22:43:00.991 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-04-24T22:43:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:00 smithi063 ceph-mon[26912]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T22:43:01.106 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.58691.asok 2024-04-24T22:43:01.109 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-24T22:43:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:43:01.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58691.asok 2024-04-24T22:43:01.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:01.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58691.asok status 2024-04-24T22:43:01.255 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-04-24T22:43:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-04-24T22:43:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "58691", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 24305, 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-04-24T22:43:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 24305 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 857152497 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 857152497 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.24305 192.168.0.1:0/857152497", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/857152497", 2024-04-24T22:43:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 14, 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 46, 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-04-24T22:43:01.258 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-04-24T22:43:01.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:01.264 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-04-24T22:43:01.264 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.285 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-04-24T22:43:01.285 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:01.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:01.359 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-04-24T22:43:01.359 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.374 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-04-24T22:43:01.374 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:43:01.375 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.440 INFO:teuthology.run_tasks:Running task print... 2024-04-24T22:43:01.450 INFO:teuthology.task.print:**** done client 2024-04-24T22:43:01.451 INFO:teuthology.run_tasks:Running task parallel... 2024-04-24T22:43:01.576 INFO:teuthology.task.parallel:starting parallel... 2024-04-24T22:43:01.577 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T22:43:01.577 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T22:43:01.578 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:43:01.578 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T22:43:01.578 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T22:43:01.579 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-24T22:43:01.582 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:01.582 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T22:43:01.582 INFO:tasks.workunit:timeout=3h 2024-04-24T22:43:01.582 INFO:tasks.workunit:cleanup=True 2024-04-24T22:43:01.583 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 86h/134d Inode: 1 Links: 2 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-24 22:42:27.631883616 +0000 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-24 22:43:01.355026643 +0000 2024-04-24T22:43:01.604 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-04-24T22:43:01.605 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T22:43:01.605 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T22:43:01.676 DEBUG:teuthology.orchestra.run.smithi192:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.692 INFO:teuthology.orchestra.run.smithi192.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-24 22:43:01.669986343 +0000 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-24 22:43:01.669986343 +0000 2024-04-24T22:43:01.693 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-24T22:43:01.693 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-24T22:43:01.693 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-24T22:43:01.761 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:01.762 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:01.778 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T22:43:01.818 INFO:tasks.workunit.client.1.smithi192.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-24T22:43:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:43:02.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:02 smithi063 ceph-mon[26912]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:03.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:02 smithi192 ceph-mon[33991]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:03.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 -- 172.21.15.63:0/2782394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 msgr2=0x7f37a00fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:03.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 --2- 172.21.15.63:0/2782394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a00fd3e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3790004940 tx=0x7f3790009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:03.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 -- 172.21.15.63:0/2782394504 shutdown_connections 2024-04-24T22:43:03.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 --2- 172.21.15.63:0/2782394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a00fd3e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 --2- 172.21.15.63:0/2782394504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 0x7f37a00fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.621+0000 7f37a73e1700 1 -- 172.21.15.63:0/2782394504 >> 172.21.15.63:0/2782394504 conn(0x7f37a00f7440 msgr2=0x7f37a00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 -- 172.21.15.63:0/2782394504 shutdown_connections 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 -- 172.21.15.63:0/2782394504 wait complete. 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 Processor -- start 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 -- start start 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 0x7f37a010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a01106d0 con 0x7f37a00fcf70 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a73e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0110840 con 0x7f37a00fbcc0 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.622+0000 7f37a497c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45396/0 (socket says 172.21.15.63:45396) 2024-04-24T22:43:03.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 -- 172.21.15.63:0/1478716684 learned_addr learned my addr 172.21.15.63:0/1478716684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:03.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a517d700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 0x7f37a010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 -- 172.21.15.63:0/1478716684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 msgr2=0x7f37a010fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 0x7f37a010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 -- 172.21.15.63:0/1478716684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3790005040 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a497c700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37900090d0 tx=0x7f3790009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379000e040 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f379000bbf0 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a0115240 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379001c3f0 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.623+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a0115800 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.624+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a0101900 con 0x7f37a00fcf70 2024-04-24T22:43:03.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.625+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f379000bd60 con 0x7f37a00fcf70 2024-04-24T22:43:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.625+0000 7f3795ffb700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 0x7f378c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.625+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3790088500 con 0x7f37a00fcf70 2024-04-24T22:43:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.625+0000 7f37a517d700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 0x7f378c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.626+0000 7f37a517d700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 0x7f378c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f379c004d60 tx=0x7f379c00b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:03.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.630+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f37900583c0 con 0x7f37a00fcf70 2024-04-24T22:43:03.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.827+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f37a01047b0 con 0x7f37a00fcf70 2024-04-24T22:43:03.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.828+0000 7f3795ffb700 1 -- 172.21.15.63:0/1478716684 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f3790057f50 con 0x7f37a00fcf70 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 msgr2=0x7f378c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 0x7f378c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f379c004d60 tx=0x7f379c00b040 comp rx=0 tx=0).stop 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 msgr2=0x7f37a01100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37900090d0 tx=0x7f3790009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 shutdown_connections 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f378c0692a0 0x7f378c06b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a00fbcc0 0x7f37a010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 --2- 172.21.15.63:0/1478716684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a00fcf70 0x7f37a01100d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 >> 172.21.15.63:0/1478716684 conn(0x7f37a00f7440 msgr2=0x7f37a00668a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 shutdown_connections 2024-04-24T22:43:03.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:03.830+0000 7f37a73e1700 1 -- 172.21.15.63:0/1478716684 wait complete. 2024-04-24T22:43:04.497 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T22:43:04.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:04 smithi063 ceph-mon[26912]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:04.803 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:43:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:04 smithi192 ceph-mon[33991]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:06.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 -- 172.21.15.63:0/1531928470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fce60 msgr2=0x7f5e200fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:06.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 --2- 172.21.15.63:0/1531928470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fce60 0x7f5e200fd270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c004940 tx=0x7f5e1c009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 -- 172.21.15.63:0/1531928470 shutdown_connections 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 --2- 172.21.15.63:0/1531928470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fdfd0 0x7f5e200fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 --2- 172.21.15.63:0/1531928470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fce60 0x7f5e200fd270 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 -- 172.21.15.63:0/1531928470 >> 172.21.15.63:0/1531928470 conn(0x7f5e200f8560 msgr2=0x7f5e200fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 -- 172.21.15.63:0/1531928470 shutdown_connections 2024-04-24T22:43:06.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.365+0000 7f5e27909700 1 -- 172.21.15.63:0/1531928470 wait complete. 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 Processor -- start 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 -- start start 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fce60 0x7f5e2010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2010e5b0 con 0x7f5e200fdfd0 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e27909700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2010eda0 con 0x7f5e200fce60 2024-04-24T22:43:06.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e24ea4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e24ea4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45414/0 (socket says 172.21.15.63:45414) 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e24ea4700 1 -- 172.21.15.63:0/3383617721 learned_addr learned my addr 172.21.15.63:0/3383617721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.366+0000 7f5e256a5700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fce60 0x7f5e2010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e24ea4700 1 -- 172.21.15.63:0/3383617721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fce60 msgr2=0x7f5e2010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e24ea4700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fce60 0x7f5e2010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e24ea4700 1 -- 172.21.15.63:0/3383617721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c005040 con 0x7f5e200fdfd0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e24ea4700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b790 tx=0x7f5e14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1400f770 con 0x7f5e200fdfd0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e1400fda0 con 0x7f5e200fdfd0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2010f080 con 0x7f5e200fdfd0 2024-04-24T22:43:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1400d310 con 0x7f5e200fdfd0 2024-04-24T22:43:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.367+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2010f640 con 0x7f5e200fdfd0 2024-04-24T22:43:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.368+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2005f5c0 con 0x7f5e200fdfd0 2024-04-24T22:43:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.368+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5e14010420 con 0x7f5e200fdfd0 2024-04-24T22:43:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.369+0000 7f5e11ffb700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 0x7f5e0c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:06.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.369+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5e14087100 con 0x7f5e200fdfd0 2024-04-24T22:43:06.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.369+0000 7f5e256a5700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 0x7f5e0c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:06.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.369+0000 7f5e256a5700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 0x7f5e0c06b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b3b0 tx=0x7f5e1c009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:06.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.373+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5e14056f10 con 0x7f5e200fdfd0 2024-04-24T22:43:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.570+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f5e2010f910 con 0x7f5e200fdfd0 2024-04-24T22:43:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.570+0000 7f5e11ffb700 1 -- 172.21.15.63:0/3383617721 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f5e14056aa0 con 0x7f5e200fdfd0 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 msgr2=0x7f5e0c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 0x7f5e0c06b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b3b0 tx=0x7f5e1c009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 msgr2=0x7f5e2010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b790 tx=0x7f5e14009250 comp rx=0 tx=0).stop 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 shutdown_connections 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5e0c0692b0 0x7f5e0c06b760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e200fce60 0x7f5e2010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 --2- 172.21.15.63:0/3383617721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e200fdfd0 0x7f5e2010df20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 >> 172.21.15.63:0/3383617721 conn(0x7f5e200f8560 msgr2=0x7f5e200f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 shutdown_connections 2024-04-24T22:43:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:06.572+0000 7f5e27909700 1 -- 172.21.15.63:0/3383617721 wait complete. 2024-04-24T22:43:06.885 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:06 smithi063 ceph-mon[26912]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:06 smithi192 ceph-mon[33991]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T22:43:07.227 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T22:43:07.495 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:43:08.795 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:08 smithi063 ceph-mon[26912]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-24T22:43:09.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 -- 172.21.15.63:0/3170292427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 msgr2=0x7ff2b80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/3170292427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80fe440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8004940 tx=0x7ff2a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:09.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 -- 172.21.15.63:0/3170292427 shutdown_connections 2024-04-24T22:43:09.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/3170292427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80fe440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/3170292427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 0x7ff2b80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 -- 172.21.15.63:0/3170292427 >> 172.21.15.63:0/3170292427 conn(0x7ff2b80f8560 msgr2=0x7ff2b80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 -- 172.21.15.63:0/3170292427 shutdown_connections 2024-04-24T22:43:09.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.041+0000 7ff2c00f0700 1 -- 172.21.15.63:0/3170292427 wait complete. 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 Processor -- start 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 -- start start 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 0x7ff2b8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80708f0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2c00f0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b8070a60 con 0x7ff2b80fce60 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2bd68b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2bd68b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49140/0 (socket says 172.21.15.63:49140) 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2bd68b700 1 -- 172.21.15.63:0/525270667 learned_addr learned my addr 172.21.15.63:0/525270667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.042+0000 7ff2bde8c700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 0x7ff2b8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2bd68b700 1 -- 172.21.15.63:0/525270667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 msgr2=0x7ff2b8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2bd68b700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 0x7ff2b8071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2bd68b700 1 -- 172.21.15.63:0/525270667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8005040 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2bd68b700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009150 tx=0x7ff2a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a800e040 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2a800bbf0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b8070ce0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a801c630 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.043+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b8115a10 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.044+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b805f5c0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.045+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff2a800bd60 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.046+0000 7ff2ae7fc700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 0x7ff2a406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.046+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff2a80881f0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.046+0000 7ff2bde8c700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 0x7ff2a406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.046+0000 7ff2bde8c700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 0x7ff2a406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8071b70 tx=0x7ff2b40091a0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.050+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff2a8054570 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:08 smithi192 ceph-mon[33991]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-24T22:43:09.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.248+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7ff2b804bed0 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.248+0000 7ff2ae7fc700 1 -- 172.21.15.63:0/525270667 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7ff2a8013e60 con 0x7ff2b80fdfd0 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 msgr2=0x7ff2a406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 0x7ff2a406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8071b70 tx=0x7ff2b40091a0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 msgr2=0x7ff2b80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009150 tx=0x7ff2a8009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 shutdown_connections 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7ff2a40692b0 0x7ff2a406b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b80fce60 0x7ff2b8071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 --2- 172.21.15.63:0/525270667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b80fdfd0 0x7ff2b80702b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 >> 172.21.15.63:0/525270667 conn(0x7ff2b80f8560 msgr2=0x7ff2b80f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 shutdown_connections 2024-04-24T22:43:09.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:09.250+0000 7ff2c00f0700 1 -- 172.21.15.63:0/525270667 wait complete. 2024-04-24T22:43:09.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-24T22:43:10.170 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:43:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:10 smithi063 ceph-mon[26912]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-24T22:43:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:10 smithi192 ceph-mon[33991]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-24T22:43:11.834 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 -- 172.21.15.63:0/2458753986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 msgr2=0x7f047c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 --2- 172.21.15.63:0/2458753986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c0fd3d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0470004940 tx=0x7f0470009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 -- 172.21.15.63:0/2458753986 shutdown_connections 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 --2- 172.21.15.63:0/2458753986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c0fd3d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 --2- 172.21.15.63:0/2458753986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 -- 172.21.15.63:0/2458753986 >> 172.21.15.63:0/2458753986 conn(0x7f047c0f7430 msgr2=0x7f047c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 -- 172.21.15.63:0/2458753986 shutdown_connections 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.858+0000 7f048336e700 1 -- 172.21.15.63:0/2458753986 wait complete. 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 Processor -- start 2024-04-24T22:43:11.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 -- start start 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c10dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c10e5f0 con 0x7f047c0fcf60 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048336e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c10edf0 con 0x7f047c0fbcb0 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49168/0 (socket says 172.21.15.63:49168) 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 -- 172.21.15.63:0/2601413821 learned_addr learned my addr 172.21.15.63:0/2601413821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048110a700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c10dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 -- 172.21.15.63:0/2601413821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 msgr2=0x7f047c10dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:11.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c10dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 -- 172.21.15.63:0/2601413821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470005040 con 0x7f047c0fcf60 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f048110a700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.859+0000 7f0480909700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0470009150 tx=0x7f0470009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.860+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047000e040 con 0x7f047c0fcf60 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.860+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047000bcc0 con 0x7f047c0fcf60 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.860+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c10f070 con 0x7f047c0fcf60 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.860+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047001c630 con 0x7f047c0fcf60 2024-04-24T22:43:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.860+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c10f5b0 con 0x7f047c0fcf60 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.861+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c101af0 con 0x7f047c0fcf60 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.861+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0470013420 con 0x7f047c0fcf60 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.862+0000 7f0479ffb700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 0x7f046406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.862+0000 7f048110a700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 0x7f046406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.862+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0470087ff0 con 0x7f047c0fcf60 2024-04-24T22:43:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.862+0000 7f048110a700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 0x7f046406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f046c004550 tx=0x7f046c009040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:11.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:11.867+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0470057d60 con 0x7f047c0fcf60 2024-04-24T22:43:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:12.078+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f047c10f7d0 con 0x7f04640692a0 2024-04-24T22:43:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:13.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:12 smithi063 ceph-mon[26912]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-24T22:43:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:12 smithi192 ceph-mon[33991]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-24T22:43:14.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:13 smithi063 ceph-mon[26912]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:14.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:13 smithi192 ceph-mon[33991]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:15.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:14 smithi063 ceph-mon[26912]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-24T22:43:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:14 smithi192 ceph-mon[33991]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-24T22:43:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:16 smithi192 ceph-mon[33991]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T22:43:17.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:16 smithi063 ceph-mon[26912]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T22:43:19.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:18 smithi063 ceph-mon[26912]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T22:43:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:18 smithi192 ceph-mon[33991]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T22:43:21.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:20 smithi063 ceph-mon[26912]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T22:43:21.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:20 smithi192 ceph-mon[33991]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T22:43:23.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:22 smithi063 ceph-mon[26912]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-24T22:43:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:22 smithi192 ceph-mon[33991]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-24T22:43:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:24 smithi063 ceph-mon[26912]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T22:43:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:24 smithi192 ceph-mon[33991]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T22:43:26.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:26 smithi192 ceph-mon[33991]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T22:43:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:26 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:27.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:26 smithi063 ceph-mon[26912]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T22:43:27.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:26 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:29.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:28 smithi063 ceph-mon[26912]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:28 smithi192 ceph-mon[33991]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:30 smithi063 ceph-mon[26912]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:30 smithi192 ceph-mon[33991]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:33.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:32 smithi063 ceph-mon[26912]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:32 smithi192 ceph-mon[33991]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:35.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:34 smithi063 ceph-mon[26912]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:34 smithi192 ceph-mon[33991]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:36 smithi192 ceph-mon[33991]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:36 smithi063 ceph-mon[26912]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:39.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:38 smithi063 ceph-mon[26912]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:39.174 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:38 smithi192 ceph-mon[33991]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:40 smithi063 ceph-mon[26912]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:40 smithi192 ceph-mon[33991]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:43.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:42 smithi063 ceph-mon[26912]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:42 smithi192 ceph-mon[33991]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:45.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:44 smithi063 ceph-mon[26912]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:44 smithi192 ceph-mon[33991]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.394+0000 7f0479ffb700 1 -- 172.21.15.63:0/2601413821 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f047c10f7d0 con 0x7f04640692a0 2024-04-24T22:43:46.397 INFO:teuthology.orchestra.run.smithi063.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.397+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 msgr2=0x7f046406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.397+0000 7f048336e700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 0x7f046406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f046c004550 tx=0x7f046c009040 comp rx=0 tx=0).stop 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 msgr2=0x7f047c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0470009150 tx=0x7f0470009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 shutdown_connections 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f04640692a0 0x7f046406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f047c0fbcb0 0x7f047c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 --2- 172.21.15.63:0/2601413821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c0fcf60 0x7f047c10dff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 >> 172.21.15.63:0/2601413821 conn(0x7f047c0f7430 msgr2=0x7f047c0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 shutdown_connections 2024-04-24T22:43:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:46.398+0000 7f048336e700 1 -- 172.21.15.63:0/2601413821 wait complete. 2024-04-24T22:43:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:46 smithi192 ceph-mon[33991]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:46 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:47.047 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:46 smithi063 ceph-mon[26912]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:47.047 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:47.048 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:46 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:47.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-24T22:43:47.449 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:43:47.833 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:47 smithi063 ceph-mon[26912]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:47.834 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:43:47.834 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:43:47.834 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:47 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:47 smithi192 ceph-mon[33991]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:48.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:43:48.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:43:48.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:47 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:48 smithi192 ceph-mon[33991]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:48 smithi192 ceph-mon[33991]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:48.941 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:48 smithi063 ceph-mon[26912]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:43:48.941 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:48 smithi063 ceph-mon[26912]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3576539146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 msgr2=0x7f5cc8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 --2- 172.21.15.63:0/3576539146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8104720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8004940 tx=0x7f5cb8009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3576539146 shutdown_connections 2024-04-24T22:43:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 --2- 172.21.15.63:0/3576539146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8104720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 --2- 172.21.15.63:0/3576539146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.411+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3576539146 >> 172.21.15.63:0/3576539146 conn(0x7f5cc80f7430 msgr2=0x7f5cc80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3576539146 shutdown_connections 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3576539146 wait complete. 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 Processor -- start 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 -- start start 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8114510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8114b10 con 0x7f5cc8102340 2024-04-24T22:43:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.412+0000 7f5ccf5df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8115310 con 0x7f5cc8066260 2024-04-24T22:43:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5cccb7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5ccd37b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5cccb7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8114510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37366/0 (socket says 172.21.15.63:37366) 2024-04-24T22:43:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5cccb7a700 1 -- 172.21.15.63:0/3677774573 learned_addr learned my addr 172.21.15.63:0/3677774573 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5ccd37b700 1 -- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 msgr2=0x7f5cc8114510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5ccd37b700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8114510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5ccd37b700 1 -- 172.21.15.63:0/3677774573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8005040 con 0x7f5cc8066260 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.413+0000 7f5ccd37b700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8113fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8065fa0 tx=0x7f5cc400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.414+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc4004e90 con 0x7f5cc8066260 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.414+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cc4004e90 con 0x7f5cc8066260 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.414+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3677774573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8115590 con 0x7f5cc8066260 2024-04-24T22:43:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.414+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc400c6e0 con 0x7f5cc8066260 2024-04-24T22:43:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.414+0000 7f5ccf5df700 1 -- 172.21.15.63:0/3677774573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8115ad0 con 0x7f5cc8066260 2024-04-24T22:43:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.415+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc805f5c0 con 0x7f5cc8066260 2024-04-24T22:43:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.417+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5cc4007cc0 con 0x7f5cc8066260 2024-04-24T22:43:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.417+0000 7f5cbdffb700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 0x7f5cb406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.417+0000 7f5cccb7a700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 0x7f5cb406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.418+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5cc4052e90 con 0x7f5cc8066260 2024-04-24T22:43:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.418+0000 7f5cccb7a700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 0x7f5cb406b7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8013740 tx=0x7f5cb8009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:49.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.423+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5cc4056550 con 0x7f5cc8066260 2024-04-24T22:43:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.807+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cc804bed0 con 0x7f5cb4069300 2024-04-24T22:43:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.808+0000 7f5cbdffb700 1 -- 172.21.15.63:0/3677774573 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f5cc804bed0 con 0x7f5cb4069300 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 msgr2=0x7f5cb406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 0x7f5cb406b7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8013740 tx=0x7f5cb8009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 msgr2=0x7f5cc8113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8113fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8065fa0 tx=0x7f5cc400b040 comp rx=0 tx=0).stop 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 shutdown_connections 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f5cb4069300 0x7f5cb406b7b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cc8066260 0x7f5cc8113fd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 --2- 172.21.15.63:0/3677774573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cc8102340 0x7f5cc8114510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 >> 172.21.15.63:0/3677774573 conn(0x7f5cc80f7430 msgr2=0x7f5cc80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 shutdown_connections 2024-04-24T22:43:49.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.812+0000 7f5cb37fe700 1 -- 172.21.15.63:0/3677774573 wait complete. 2024-04-24T22:43:49.832 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:43:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 -- 172.21.15.63:0/4107814955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224066e70 msgr2=0x7f8224101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 --2- 172.21.15.63:0/4107814955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224066e70 0x7f8224101e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8220004940 tx=0x7f8220009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 -- 172.21.15.63:0/4107814955 shutdown_connections 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 --2- 172.21.15.63:0/4107814955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224102370 0x7f8224104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 --2- 172.21.15.63:0/4107814955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224066e70 0x7f8224101e30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 -- 172.21.15.63:0/4107814955 >> 172.21.15.63:0/4107814955 conn(0x7f82240f7420 msgr2=0x7f82240f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 -- 172.21.15.63:0/4107814955 shutdown_connections 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.977+0000 7f822b3f6700 1 -- 172.21.15.63:0/4107814955 wait complete. 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 Processor -- start 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 -- start start 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 0x7f8224113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 0x7f82241143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82241149c0 con 0x7f8224102370 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f822b3f6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224114b30 con 0x7f8224066e70 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f8229192700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 0x7f8224113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f8229192700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 0x7f8224113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:34966/0 (socket says 172.21.15.63:34966) 2024-04-24T22:43:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f8229192700 1 -- 172.21.15.63:0/3270804542 learned_addr learned my addr 172.21.15.63:0/3270804542 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.978+0000 7f8228991700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 0x7f82241143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8228991700 1 -- 172.21.15.63:0/3270804542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 msgr2=0x7f8224113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8228991700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 0x7f8224113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8228991700 1 -- 172.21.15.63:0/3270804542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220005040 con 0x7f8224102370 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8228991700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 0x7f82241143c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f821400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8214004e90 con 0x7f8224102370 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8224119530 con 0x7f8224102370 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f821400f040 con 0x7f8224102370 2024-04-24T22:43:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.979+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f821400ca50 con 0x7f8224102370 2024-04-24T22:43:49.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.980+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224119a70 con 0x7f8224102370 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.981+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822404c710 con 0x7f8224102370 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.982+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8214007cc0 con 0x7f8224102370 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.982+0000 7f8219ffb700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 0x7f821006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.982+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8214086bb0 con 0x7f8224102370 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.982+0000 7f8229192700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 0x7f821006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:49.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.983+0000 7f8229192700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 0x7f821006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82200136a0 tx=0x7f8220009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:49.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:49.990+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f821404e250 con 0x7f8224102370 2024-04-24T22:43:50.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.323+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82240f8f80 con 0x7f82100692a0 2024-04-24T22:43:50.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.325+0000 7f8219ffb700 1 -- 172.21.15.63:0/3270804542 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f82240f8f80 con 0x7f82100692a0 2024-04-24T22:43:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 msgr2=0x7f821006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 0x7f821006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82200136a0 tx=0x7f8220009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 msgr2=0x7f82241143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 0x7f82241143c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f821400b040 comp rx=0 tx=0).stop 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 shutdown_connections 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f82100692a0 0x7f821006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8224066e70 0x7f8224113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 --2- 172.21.15.63:0/3270804542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8224102370 0x7f82241143c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 >> 172.21.15.63:0/3270804542 conn(0x7f82240f7420 msgr2=0x7f82240f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.327+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 shutdown_connections 2024-04-24T22:43:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.328+0000 7f822b3f6700 1 -- 172.21.15.63:0/3270804542 wait complete. 2024-04-24T22:43:50.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 -- 172.21.15.63:0/3716626244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc066240 msgr2=0x7f62cc0ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/3716626244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc066240 0x7f62cc0ffa20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004940 tx=0x7f62bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:50.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 -- 172.21.15.63:0/3716626244 shutdown_connections 2024-04-24T22:43:50.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/3716626244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc0fff60 0x7f62cc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/3716626244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc066240 0x7f62cc0ffa20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 -- 172.21.15.63:0/3716626244 >> 172.21.15.63:0/3716626244 conn(0x7f62cc0f7430 msgr2=0x7f62cc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:50.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.492+0000 7f62d1ec1700 1 -- 172.21.15.63:0/3716626244 shutdown_connections 2024-04-24T22:43:50.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.493+0000 7f62d1ec1700 1 -- 172.21.15.63:0/3716626244 wait complete. 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.493+0000 7f62d1ec1700 1 Processor -- start 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.493+0000 7f62d1ec1700 1 -- start start 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62d1ec1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc066240 0x7f62cc10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62d1ec1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62d1ec1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc10e630 con 0x7f62cc0fff60 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62d1ec1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc10ee20 con 0x7f62cc066240 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62cb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc066240 0x7f62cc10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37396/0 (socket says 172.21.15.63:37396) 2024-04-24T22:43:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 -- 172.21.15.63:0/1092968695 learned_addr learned my addr 172.21.15.63:0/1092968695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 -- 172.21.15.63:0/1092968695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc066240 msgr2=0x7f62cc10da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc066240 0x7f62cc10da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.494+0000 7f62caffd700 1 -- 172.21.15.63:0/1092968695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc005040 con 0x7f62cc0fff60 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62caffd700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b790 tx=0x7f62c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c000f6f0 con 0x7f62cc0fff60 2024-04-24T22:43:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc10f100 con 0x7f62cc0fff60 2024-04-24T22:43:50.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc10f660 con 0x7f62cc0fff60 2024-04-24T22:43:50.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c000fd20 con 0x7f62cc0fff60 2024-04-24T22:43:50.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.495+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c0011400 con 0x7f62cc0fff60 2024-04-24T22:43:50.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.496+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b80052e0 con 0x7f62cc0fff60 2024-04-24T22:43:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.497+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f62c001a400 con 0x7f62cc0fff60 2024-04-24T22:43:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.498+0000 7f62b3fff700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 0x7f62b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.498+0000 7f62cb7fe700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 0x7f62b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.498+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f62c0086e70 con 0x7f62cc0fff60 2024-04-24T22:43:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.499+0000 7f62cb7fe700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 0x7f62b406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004890 tx=0x7f62bc009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:50.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.507+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f62c0056c80 con 0x7f62cc0fff60 2024-04-24T22:43:50.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:50 smithi063 ceph-mon[26912]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:50.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:50 smithi063 ceph-mon[26912]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:50.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:50.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:50.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:50 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:50.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.902+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b8000bf0 con 0x7f62b40692a0 2024-04-24T22:43:50.924 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 58s ago 6m 24.5M - 0.25.0 c8568f914cd2 f1e9967ae2bb 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (7m) 58s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (5m) 63s ago 5m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 58s ago 6m 85.4M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (77s) 58s ago 76s 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (84s) 58s ago 83s 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (81s) 63s ago 81s 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (74s) 63s ago 74s 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (8m) 58s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (5m) 63s ago 5m 431M - 17.2.7-902-gc61472c8 0de407c1b3af afb9d2db7487 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 58s ago 8m 46.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:43:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (5m) 63s ago 5m 39.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 58s ago 7m 15.4M - 1.5.0 0da6a335fe13 b97c5a0b7f9e 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 63s ago 5m 15.4M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 58s ago 5m 48.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 58s ago 4m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 58s ago 4m 44.4M 4288M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (3m) 63s ago 3m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (3m) 63s ago 3m 41.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (2m) 63s ago 2m 42.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 58s ago 6m 53.9M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:43:50.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.921+0000 7f62b3fff700 1 -- 172.21.15.63:0/1092968695 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f62b8000bf0 con 0x7f62b40692a0 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.924+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 msgr2=0x7f62b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.924+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 0x7f62b406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004890 tx=0x7f62bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.924+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 msgr2=0x7f62cc10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.924+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b790 tx=0x7f62c0009250 comp rx=0 tx=0).stop 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.924+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 shutdown_connections 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.925+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f62b40692a0 0x7f62b406b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.925+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62cc066240 0x7f62cc10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.925+0000 7f62d1ec1700 1 --2- 172.21.15.63:0/1092968695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0fff60 0x7f62cc10dfa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:50.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.925+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 >> 172.21.15.63:0/1092968695 conn(0x7f62cc0f7430 msgr2=0x7f62cc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:50.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.925+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 shutdown_connections 2024-04-24T22:43:50.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:50.926+0000 7f62d1ec1700 1 -- 172.21.15.63:0/1092968695 wait complete. 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.082+0000 7fda65230700 1 -- 172.21.15.63:0/568060429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda600fff40 msgr2=0x7fda60102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.082+0000 7fda65230700 1 --2- 172.21.15.63:0/568060429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda600fff40 0x7fda60102320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda54004940 tx=0x7fda54009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 -- 172.21.15.63:0/568060429 shutdown_connections 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 --2- 172.21.15.63:0/568060429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda600fff40 0x7fda60102320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 --2- 172.21.15.63:0/568060429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda60066260 0x7fda600ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 -- 172.21.15.63:0/568060429 >> 172.21.15.63:0/568060429 conn(0x7fda600f7430 msgr2=0x7fda600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 -- 172.21.15.63:0/568060429 shutdown_connections 2024-04-24T22:43:51.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 -- 172.21.15.63:0/568060429 wait complete. 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.083+0000 7fda65230700 1 Processor -- start 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda65230700 1 -- start start 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda65230700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda65230700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 0x7fda6010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda65230700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6010e4f0 con 0x7fda60066260 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda65230700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6010ecf0 con 0x7fda600fff40 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37406/0 (socket says 172.21.15.63:37406) 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 0x7fda6010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 -- 172.21.15.63:0/4229647614 learned_addr learned my addr 172.21.15.63:0/4229647614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 -- 172.21.15.63:0/4229647614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 msgr2=0x7fda6010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 0x7fda6010def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5ed9d700 1 -- 172.21.15.63:0/4229647614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54005040 con 0x7fda60066260 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.084+0000 7fda5e59c700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 0x7fda6010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda5ed9d700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fda50002a80 tx=0x7fda500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda50007cc0 con 0x7fda60066260 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6010efd0 con 0x7fda60066260 2024-04-24T22:43:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda5000f040 con 0x7fda60066260 2024-04-24T22:43:51.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6010f510 con 0x7fda60066260 2024-04-24T22:43:51.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.085+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda5000db30 con 0x7fda60066260 2024-04-24T22:43:51.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.087+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6005f5c0 con 0x7fda60066260 2024-04-24T22:43:51.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.087+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fda5000dc90 con 0x7fda60066260 2024-04-24T22:43:51.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.088+0000 7fda477fe700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 0x7fda4806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.088+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fda50053000 con 0x7fda60066260 2024-04-24T22:43:51.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.088+0000 7fda5e59c700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 0x7fda4806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.089+0000 7fda5e59c700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 0x7fda4806b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda5400b3b0 tx=0x7fda54009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:51.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.097+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fda5004d250 con 0x7fda60066260 2024-04-24T22:43:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:50 smithi192 ceph-mon[33991]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:50 smithi192 ceph-mon[33991]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:43:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:50 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:43:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.401+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda6010f7f0 con 0x7fda60066260 2024-04-24T22:43:51.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.402+0000 7fda477fe700 1 -- 172.21.15.63:0/4229647614 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fda500566c0 con 0x7fda60066260 2024-04-24T22:43:51.406 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:43:51.406 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:43:51.406 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:43:51.407 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 msgr2=0x7fda4806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 0x7fda4806b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda5400b3b0 tx=0x7fda54009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 msgr2=0x7fda6010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fda50002a80 tx=0x7fda500046b0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 shutdown_connections 2024-04-24T22:43:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fda480692b0 0x7fda4806b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda60066260 0x7fda6010d9b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 --2- 172.21.15.63:0/4229647614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda600fff40 0x7fda6010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.405+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 >> 172.21.15.63:0/4229647614 conn(0x7fda600f7430 msgr2=0x7fda600f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.406+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 shutdown_connections 2024-04-24T22:43:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.406+0000 7fda65230700 1 -- 172.21.15.63:0/4229647614 wait complete. 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 -- 172.21.15.63:0/2616131917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fce70 msgr2=0x7fe10c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 --2- 172.21.15.63:0/2616131917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fce70 0x7fe10c0fd280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc004940 tx=0x7fe0fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 -- 172.21.15.63:0/2616131917 shutdown_connections 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 --2- 172.21.15.63:0/2616131917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fdfe0 0x7fe10c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 --2- 172.21.15.63:0/2616131917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fce70 0x7fe10c0fd280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 -- 172.21.15.63:0/2616131917 >> 172.21.15.63:0/2616131917 conn(0x7fe10c0f8550 msgr2=0x7fe10c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:51.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 -- 172.21.15.63:0/2616131917 shutdown_connections 2024-04-24T22:43:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.568+0000 7fe11137c700 1 -- 172.21.15.63:0/2616131917 wait complete. 2024-04-24T22:43:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 Processor -- start 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 -- start start 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fdfe0 0x7fe10c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c072200 con 0x7fe10c0fdfe0 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe11137c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c1115c0 con 0x7fe10c0fce70 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe10affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe10affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:35024/0 (socket says 172.21.15.63:35024) 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe10affd700 1 -- 172.21.15.63:0/2717539134 learned_addr learned my addr 172.21.15.63:0/2717539134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.569+0000 7fe10a7fc700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fdfe0 0x7fe10c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe10affd700 1 -- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fdfe0 msgr2=0x7fe10c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe10affd700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fdfe0 0x7fe10c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe10affd700 1 -- 172.21.15.63:0/2717539134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc005040 con 0x7fe10c0fce70 2024-04-24T22:43:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe10affd700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc013870 tx=0x7fe0fc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0fc021070 con 0x7fe10c0fce70 2024-04-24T22:43:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0fc009430 con 0x7fe10c0fce70 2024-04-24T22:43:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c111810 con 0x7fe10c0fce70 2024-04-24T22:43:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.570+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0fc00e070 con 0x7fe10c0fce70 2024-04-24T22:43:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.571+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c070ba0 con 0x7fe10c0fce70 2024-04-24T22:43:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.572+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c04c710 con 0x7fe10c0fce70 2024-04-24T22:43:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.572+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe0fc01c750 con 0x7fe10c0fce70 2024-04-24T22:43:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.573+0000 7fe0f37fe700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 0x7fe0f406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:51.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.573+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe0fc0882a0 con 0x7fe10c0fce70 2024-04-24T22:43:51.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.573+0000 7fe10a7fc700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 0x7fe0f406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:51.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.573+0000 7fe10a7fc700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 0x7fe0f406b6f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b80 tx=0x7fe10000b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:51.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.581+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe0fc058010 con 0x7fe10c0fce70 2024-04-24T22:43:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.817+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe10c111810 con 0x7fe0f4069240 2024-04-24T22:43:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.818+0000 7fe0f37fe700 1 -- 172.21.15.63:0/2717539134 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fe10c111810 con 0x7fe0f4069240 2024-04-24T22:43:51.820 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:43:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:43:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 image on host smithi192", 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:43:51.821 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 msgr2=0x7fe0f406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 0x7fe0f406b6f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe100009b80 tx=0x7fe10000b040 comp rx=0 tx=0).stop 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 msgr2=0x7fe10c071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc013870 tx=0x7fe0fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 shutdown_connections 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fe0f4069240 0x7fe0f406b6f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe10c0fce70 0x7fe10c071c00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 --2- 172.21.15.63:0/2717539134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10c0fdfe0 0x7fe10c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.820+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 >> 172.21.15.63:0/2717539134 conn(0x7fe10c0f8550 msgr2=0x7fe10c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:51.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.821+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 shutdown_connections 2024-04-24T22:43:51.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:43:51.821+0000 7fe11137c700 1 -- 172.21.15.63:0/2717539134 wait complete. 2024-04-24T22:43:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-24T22:43:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T22:43:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: Upgrade: Need to upgrade myself (mgr.smithi063.upcnvd) 2024-04-24T22:43:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:51 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4229647614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:43:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-24T22:43:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T22:43:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: Upgrade: Need to upgrade myself (mgr.smithi063.upcnvd) 2024-04-24T22:43:52.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:52.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:51 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4229647614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:43:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:52 smithi063 ceph-mon[26912]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi192 2024-04-24T22:43:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:52 smithi063 ceph-mon[26912]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:52 smithi063 ceph-mon[26912]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:52 smithi192 ceph-mon[33991]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi192 2024-04-24T22:43:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:52 smithi192 ceph-mon[33991]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:52 smithi192 ceph-mon[33991]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:43:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:54 smithi063 ceph-mon[26912]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:54 smithi192 ceph-mon[33991]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:56 smithi192 ceph-mon[33991]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:56 smithi063 ceph-mon[26912]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:43:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:43:58 smithi063 ceph-mon[26912]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:43:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:43:58 smithi192 ceph-mon[33991]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:00 smithi063 ceph-mon[26912]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:00 smithi192 ceph-mon[33991]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:03.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:02 smithi063 ceph-mon[26912]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:02 smithi192 ceph-mon[33991]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:05.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:04 smithi063 ceph-mon[26912]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:04 smithi192 ceph-mon[33991]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:06 smithi192 ceph-mon[33991]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:06 smithi063 ceph-mon[26912]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:08 smithi063 ceph-mon[26912]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:09.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:08 smithi192 ceph-mon[33991]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:11.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:10 smithi063 ceph-mon[26912]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:11.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:10 smithi192 ceph-mon[33991]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:11 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:11 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:13.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:12 smithi063 ceph-mon[26912]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:12 smithi192 ceph-mon[33991]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:15.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:14 smithi063 ceph-mon[26912]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:14 smithi192 ceph-mon[33991]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:15.338 INFO:tasks.workunit.client.1.smithi192.stderr:Updating files: 95% (11872/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T22:44:15.728 INFO:tasks.workunit.client.0.smithi063.stderr:Updating files: 96% (12078/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T22:44:15.799 INFO:tasks.workunit.client.1.smithi192.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T22:44:15.799 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.799 INFO:tasks.workunit.client.1.smithi192.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: git switch -c 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:Or undo this operation with: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: git switch - 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-24T22:44:15.800 INFO:tasks.workunit.client.1.smithi192.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T22:44:15.804 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-24T22:44:15.866 INFO:tasks.workunit.client.1.smithi192.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T22:44:15.869 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T22:44:15.869 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T22:44:16.468 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T22:44:16.523 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-04-24T22:44:16.528 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.529 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-04-24T22:44:16.529 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.529 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T22:44:16.529 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-04-24T22:44:16.529 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T22:44:16.533 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T22:44:16.563 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T22:44:16.564 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T22:44:16.564 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T22:44:16.606 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T22:44:16.609 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-24T22:44:16.609 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-24T22:44:16.665 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-24T22:44:16.667 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T22:44:16.667 DEBUG:teuthology.orchestra.run.smithi192:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-24T22:44:16.723 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T22:44:16.726 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:44:16.726 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T22:44:16.735 INFO:tasks.workunit.client.1.smithi192.stderr:+ mkdir -p fsstress 2024-04-24T22:44:16.737 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd fsstress 2024-04-24T22:44:16.739 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T22:44:16.740 INFO:tasks.workunit.client.1.smithi192.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T22:44:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:16 smithi192 ceph-mon[33991]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:17.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:16 smithi063 ceph-mon[26912]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:17.425 INFO:tasks.workunit.client.1.smithi192.stderr:+ tar xzf ltp-full.tgz 2024-04-24T22:44:19.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:18 smithi192 ceph-mon[33991]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:18 smithi063 ceph-mon[26912]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:44:20.907 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T22:44:20.970 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T22:44:21.013 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:44:21.015 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:44:21.015 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T22:44:21.055 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:44:21.059 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:44:21.059 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T22:44:21.116 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-24T22:44:21.119 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T22:44:21.119 DEBUG:teuthology.orchestra.run.smithi063:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-24T22:44:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:20 smithi192 ceph-mon[33991]: pgmap v233: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 559 KiB/s wr, 4 op/s 2024-04-24T22:44:21.182 INFO:tasks.workunit.client.0.smithi063.stderr:+ mkdir -p fsstress 2024-04-24T22:44:21.184 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd fsstress 2024-04-24T22:44:21.185 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T22:44:21.185 INFO:tasks.workunit.client.0.smithi063.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T22:44:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:20 smithi063 ceph-mon[26912]: pgmap v233: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 559 KiB/s wr, 4 op/s 2024-04-24T22:44:21.880 INFO:tasks.workunit.client.0.smithi063.stderr:+ tar xzf ltp-full.tgz 2024-04-24T22:44:21.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 -- 172.21.15.63:0/4191993156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2724066200 msgr2=0x7f27240ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 --2- 172.21.15.63:0/4191993156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2724066200 0x7f27240ffa60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2720004940 tx=0x7f2720009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 -- 172.21.15.63:0/4191993156 shutdown_connections 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 --2- 172.21.15.63:0/4191993156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27240fffa0 0x7f2724102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 --2- 172.21.15.63:0/4191993156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2724066200 0x7f27240ffa60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 -- 172.21.15.63:0/4191993156 >> 172.21.15.63:0/4191993156 conn(0x7f27240f7430 msgr2=0x7f27240f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:21.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 -- 172.21.15.63:0/4191993156 shutdown_connections 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.957+0000 7f272bf8d700 1 -- 172.21.15.63:0/4191993156 wait complete. 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 Processor -- start 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 -- start start 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 0x7f272410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724110740 con 0x7f27240fffa0 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f272bf8d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724110f90 con 0x7f2724066200 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f2729d29700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 0x7f272410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f2729528700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f2729528700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40986/0 (socket says 172.21.15.63:40986) 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f2729d29700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 0x7f272410fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:50684/0 (socket says 172.21.15.63:50684) 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.958+0000 7f2729528700 1 -- 172.21.15.63:0/406022654 learned_addr learned my addr 172.21.15.63:0/406022654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:21.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f2729528700 1 -- 172.21.15.63:0/406022654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 msgr2=0x7f272410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:21.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f2729528700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 0x7f272410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:21.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f2729528700 1 -- 172.21.15.63:0/406022654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720005040 con 0x7f27240fffa0 2024-04-24T22:44:21.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f2729528700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f271400eb40 tx=0x7f271400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2714007cc0 con 0x7f27240fffa0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2724111210 con 0x7f27240fffa0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2724111750 con 0x7f27240fffa0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f271400f040 con 0x7f27240fffa0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.959+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271400c9d0 con 0x7f27240fffa0 2024-04-24T22:44:21.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.960+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27080052e0 con 0x7f27240fffa0 2024-04-24T22:44:21.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.961+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f271400cc40 con 0x7f27240fffa0 2024-04-24T22:44:21.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.962+0000 7f271a7fc700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 0x7f271006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:21.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.962+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2714086b90 con 0x7f27240fffa0 2024-04-24T22:44:21.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.962+0000 7f2729d29700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 0x7f271006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:21.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.962+0000 7f2729d29700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 0x7f271006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2720004890 tx=0x7f2720009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:21.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:21.967+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f271404e250 con 0x7f27240fffa0 2024-04-24T22:44:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.224+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2708000bf0 con 0x7f27100692a0 2024-04-24T22:44:22.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.225+0000 7f271a7fc700 1 -- 172.21.15.63:0/406022654 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f2708000bf0 con 0x7f27100692a0 2024-04-24T22:44:22.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 msgr2=0x7f271006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 0x7f271006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2720004890 tx=0x7f2720009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:22.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 msgr2=0x7f2724110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f271400eb40 tx=0x7f271400b040 comp rx=0 tx=0).stop 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 shutdown_connections 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f27100692a0 0x7f271006b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2724066200 0x7f272410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 --2- 172.21.15.63:0/406022654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27240fffa0 0x7f2724110140 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.227+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 >> 172.21.15.63:0/406022654 conn(0x7f27240f7430 msgr2=0x7f27240fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.228+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 shutdown_connections 2024-04-24T22:44:22.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.228+0000 7f272bf8d700 1 -- 172.21.15.63:0/406022654 wait complete. 2024-04-24T22:44:22.241 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:44:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 -- 172.21.15.63:0/894468866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fbcb0 msgr2=0x7f1fd40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/894468866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40fc0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0004940 tx=0x7f1fd0009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 -- 172.21.15.63:0/894468866 shutdown_connections 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/894468866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/894468866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40fc0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 -- 172.21.15.63:0/894468866 >> 172.21.15.63:0/894468866 conn(0x7f1fd40f7430 msgr2=0x7f1fd40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 -- 172.21.15.63:0/894468866 shutdown_connections 2024-04-24T22:44:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 -- 172.21.15.63:0/894468866 wait complete. 2024-04-24T22:44:22.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.344+0000 7f1fdb81c700 1 Processor -- start 2024-04-24T22:44:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fdb81c700 1 -- start start 2024-04-24T22:44:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fdb81c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fdb81c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fdb81c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40fe830 con 0x7f1fd40fcf60 2024-04-24T22:44:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fdb81c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40fe970 con 0x7f1fd40fbcb0 2024-04-24T22:44:22.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41004/0 (socket says 172.21.15.63:41004) 2024-04-24T22:44:22.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 -- 172.21.15.63:0/2477603576 learned_addr learned my addr 172.21.15.63:0/2477603576 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd95b8700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 -- 172.21.15.63:0/2477603576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fbcb0 msgr2=0x7f1fd40ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 -- 172.21.15.63:0/2477603576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0005040 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.345+0000 7f1fd8db7700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4002a50 tx=0x7f1fc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.346+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc4007cc0 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.346+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd40fec50 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.346+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fc400f040 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.346+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc400da30 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.346+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd4115ab0 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.347+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd4101af0 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.348+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1fc400db90 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.348+0000 7f1fc9ffb700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 0x7f1fc006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.348+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1fc4086fa0 con 0x7f1fd40fcf60 2024-04-24T22:44:22.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.348+0000 7f1fd95b8700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 0x7f1fc006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.349+0000 7f1fd95b8700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 0x7f1fc006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0004890 tx=0x7f1fd0009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:22.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.353+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1fc4053200 con 0x7f1fd40fcf60 2024-04-24T22:44:22.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.607+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fd405e3a0 con 0x7f1fc00692a0 2024-04-24T22:44:22.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.608+0000 7f1fc9ffb700 1 -- 172.21.15.63:0/2477603576 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f1fd405e3a0 con 0x7f1fc00692a0 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 msgr2=0x7f1fc006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 0x7f1fc006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0004890 tx=0x7f1fd0009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 msgr2=0x7f1fd40fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4002a50 tx=0x7f1fc40046b0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 shutdown_connections 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f1fc00692a0 0x7f1fc006b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd40fbcb0 0x7f1fd40ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 --2- 172.21.15.63:0/2477603576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fd40fcf60 0x7f1fd40fe2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 >> 172.21.15.63:0/2477603576 conn(0x7f1fd40f7430 msgr2=0x7f1fd4065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:22.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 shutdown_connections 2024-04-24T22:44:22.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.611+0000 7f1fdb81c700 1 -- 172.21.15.63:0/2477603576 wait complete. 2024-04-24T22:44:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- 172.21.15.63:0/4171464088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fd9c0 msgr2=0x7f70a80ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 --2- 172.21.15.63:0/4171464088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fd9c0 0x7f70a80ffda0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004940 tx=0x7f70a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- 172.21.15.63:0/4171464088 shutdown_connections 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 --2- 172.21.15.63:0/4171464088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a81002e0 0x7f70a81026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 --2- 172.21.15.63:0/4171464088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fd9c0 0x7f70a80ffda0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- 172.21.15.63:0/4171464088 >> 172.21.15.63:0/4171464088 conn(0x7f70a80f7440 msgr2=0x7f70a80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- 172.21.15.63:0/4171464088 shutdown_connections 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- 172.21.15.63:0/4171464088 wait complete. 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 Processor -- start 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.726+0000 7f70b07b7700 1 -- start start 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70b07b7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a80fd9c0 0x7f70a810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70b07b7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70b07b7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a810e550 con 0x7f70a81002e0 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70b07b7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a810ed50 con 0x7f70a80fd9c0 2024-04-24T22:44:22.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41032/0 (socket says 172.21.15.63:41032) 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 -- 172.21.15.63:0/2466525168 learned_addr learned my addr 172.21.15.63:0/2466525168 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70ae553700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a80fd9c0 0x7f70a810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 -- 172.21.15.63:0/2466525168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a80fd9c0 msgr2=0x7f70a810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a80fd9c0 0x7f70a810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 -- 172.21.15.63:0/2466525168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4005040 con 0x7f70a81002e0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f70add52700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f709800eb40 tx=0x7f709800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.727+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7098007cc0 con 0x7f70a81002e0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.728+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f709800f040 con 0x7f70a81002e0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.728+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a810efd0 con 0x7f70a81002e0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.728+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709800ca10 con 0x7f70a81002e0 2024-04-24T22:44:22.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.728+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a810f510 con 0x7f70a81002e0 2024-04-24T22:44:22.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.729+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a805f5c0 con 0x7f70a81002e0 2024-04-24T22:44:22.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.729+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f709800cb70 con 0x7f70a81002e0 2024-04-24T22:44:22.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.730+0000 7f709effd700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 0x7f709406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:22.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.730+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7098086d30 con 0x7f70a81002e0 2024-04-24T22:44:22.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.730+0000 7f70ae553700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 0x7f709406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:22.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.730+0000 7f70ae553700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 0x7f709406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004890 tx=0x7f70a4009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:22.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.734+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f709804e250 con 0x7f70a81002e0 2024-04-24T22:44:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.970+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70a810f840 con 0x7f70940692a0 2024-04-24T22:44:22.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:22 smithi063 ceph-mon[26912]: pgmap v234: 65 pgs: 65 active+clean; 9.0 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 724 KiB/s wr, 10 op/s 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.984+0000 7f709effd700 1 -- 172.21.15.63:0/2466525168 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f70a810f840 con 0x7f70940692a0 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 90s ago 7m 24.5M - 0.25.0 c8568f914cd2 f1e9967ae2bb 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (7m) 90s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (6m) 95s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 90s ago 7m 85.4M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (109s) 90s ago 108s 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (116s) 90s ago 115s 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:44:22.986 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (113s) 95s ago 113s 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (106s) 95s ago 106s 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (8m) 90s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (6m) 95s ago 6m 431M - 17.2.7-902-gc61472c8 0de407c1b3af afb9d2db7487 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 90s ago 9m 46.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (6m) 95s ago 6m 39.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 90s ago 7m 15.4M - 1.5.0 0da6a335fe13 b97c5a0b7f9e 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 95s ago 6m 15.4M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 90s ago 5m 48.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 90s ago 5m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 90s ago 4m 44.4M 4288M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (4m) 95s ago 4m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (3m) 95s ago 3m 41.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (3m) 95s ago 3m 42.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:44:22.987 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 90s ago 6m 53.9M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:44:22.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 msgr2=0x7f709406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 0x7f709406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004890 tx=0x7f70a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 msgr2=0x7f70a810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f709800eb40 tx=0x7f709800b040 comp rx=0 tx=0).stop 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 shutdown_connections 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f70940692a0 0x7f709406b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70a80fd9c0 0x7f70a810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 --2- 172.21.15.63:0/2466525168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81002e0 0x7f70a810df50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.986+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 >> 172.21.15.63:0/2466525168 conn(0x7f70a80f7440 msgr2=0x7f70a80f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.987+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 shutdown_connections 2024-04-24T22:44:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:22.987+0000 7f70b07b7700 1 -- 172.21.15.63:0/2466525168 wait complete. 2024-04-24T22:44:23.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 -- 172.21.15.63:0/1818479454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 msgr2=0x7f3f9c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/1818479454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c0fc0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3f98004940 tx=0x7f3f98009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 -- 172.21.15.63:0/1818479454 shutdown_connections 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/1818479454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 0x7f3f9c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/1818479454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c0fc0c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.100+0000 7f3fa41a9700 1 -- 172.21.15.63:0/1818479454 >> 172.21.15.63:0/1818479454 conn(0x7f3f9c0f7430 msgr2=0x7f3f9c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 -- 172.21.15.63:0/1818479454 shutdown_connections 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 -- 172.21.15.63:0/1818479454 wait complete. 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 Processor -- start 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 -- start start 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 0x7f3f9c112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c066390 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa41a9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c0664d0 con 0x7f3f9c0fcf60 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa1f45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa1f45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41048/0 (socket says 172.21.15.63:41048) 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.101+0000 7f3fa1f45700 1 -- 172.21.15.63:0/2504489056 learned_addr learned my addr 172.21.15.63:0/2504489056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa1744700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 0x7f3f9c112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa1f45700 1 -- 172.21.15.63:0/2504489056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 msgr2=0x7f3f9c112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa1f45700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 0x7f3f9c112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa1f45700 1 -- 172.21.15.63:0/2504489056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f98005040 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa1f45700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b3b0 tx=0x7f3f98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9800e040 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9800bbf0 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9801c630 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c113130 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.102+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1136f0 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.103+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9c05f5c0 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.104+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3f9800bd60 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.104+0000 7f3f927fc700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 0x7f3f8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.104+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3f98087ea0 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.104+0000 7f3fa1744700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 0x7f3f8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.105+0000 7f3fa1744700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 0x7f3f8806b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fd0 tx=0x7f3f8c00b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:23.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.109+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3f98054180 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:22 smithi192 ceph-mon[33991]: pgmap v234: 65 pgs: 65 active+clean; 9.0 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 724 KiB/s wr, 10 op/s 2024-04-24T22:44:23.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.425+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f9c113a70 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.426+0000 7f3f927fc700 1 -- 172.21.15.63:0/2504489056 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f3f980577a0 con 0x7f3f9c0fbcb0 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:44:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:23.429 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:44:23.429 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-24T22:44:23.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:44:23.429 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 msgr2=0x7f3f8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 0x7f3f8806b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fd0 tx=0x7f3f8c00b040 comp rx=0 tx=0).stop 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 msgr2=0x7f3f9c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b3b0 tx=0x7f3f98009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 shutdown_connections 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f3f880692a0 0x7f3f8806b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f9c0fbcb0 0x7f3f9c110640 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 --2- 172.21.15.63:0/2504489056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c0fcf60 0x7f3f9c112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 >> 172.21.15.63:0/2504489056 conn(0x7f3f9c0f7430 msgr2=0x7f3f9c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:23.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 shutdown_connections 2024-04-24T22:44:23.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.428+0000 7f3fa41a9700 1 -- 172.21.15.63:0/2504489056 wait complete. 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 -- 172.21.15.63:0/440538207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab8066260 msgr2=0x7f6ab80ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 --2- 172.21.15.63:0/440538207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab8066260 0x7f6ab80ffa00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8004940 tx=0x7f6aa8009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 -- 172.21.15.63:0/440538207 shutdown_connections 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 --2- 172.21.15.63:0/440538207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab80fff40 0x7f6ab8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 --2- 172.21.15.63:0/440538207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab8066260 0x7f6ab80ffa00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 -- 172.21.15.63:0/440538207 >> 172.21.15.63:0/440538207 conn(0x7f6ab80f7430 msgr2=0x7f6ab80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 -- 172.21.15.63:0/440538207 shutdown_connections 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 -- 172.21.15.63:0/440538207 wait complete. 2024-04-24T22:44:23.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.550+0000 7f6abdec0700 1 Processor -- start 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6abdec0700 1 -- start start 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6abdec0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab8066260 0x7f6ab80fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6abdec0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6abdec0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80fbe90 con 0x7f6ab80fff40 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6abdec0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80fbfd0 con 0x7f6ab8066260 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41066/0 (socket says 172.21.15.63:41066) 2024-04-24T22:44:23.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab8066260 0x7f6ab80fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 -- 172.21.15.63:0/3700104045 learned_addr learned my addr 172.21.15.63:0/3700104045 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 -- 172.21.15.63:0/3700104045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab8066260 msgr2=0x7f6ab80fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab8066260 0x7f6ab80fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 -- 172.21.15.63:0/3700104045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8005040 con 0x7f6ab80fff40 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.551+0000 7f6ab6ffd700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00eb40 tx=0x7f6aac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.552+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac007cc0 con 0x7f6ab80fff40 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.552+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aac00f040 con 0x7f6ab80fff40 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.552+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab80fc250 con 0x7f6ab80fff40 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.552+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac00ca10 con 0x7f6ab80fff40 2024-04-24T22:44:23.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.552+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab8115a30 con 0x7f6ab80fff40 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.553+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab805f5c0 con 0x7f6ab80fff40 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.553+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6aac00cc80 con 0x7f6ab80fff40 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.554+0000 7f6a9ffff700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 0x7f6aa006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.554+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6aac086de0 con 0x7f6ab80fff40 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.554+0000 7f6ab77fe700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 0x7f6aa006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:23.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.555+0000 7f6ab77fe700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 0x7f6aa006b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8013740 tx=0x7f6aa8009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:23.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.559+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6aac04e250 con 0x7f6ab80fff40 2024-04-24T22:44:23.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.836+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab80f8f20 con 0x7f6aa00692b0 2024-04-24T22:44:23.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.837+0000 7f6a9ffff700 1 -- 172.21.15.63:0/3700104045 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6ab80f8f20 con 0x7f6aa00692b0 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 image on host smithi192", 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:44:23.839 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:44:23.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 msgr2=0x7f6aa006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 0x7f6aa006b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8013740 tx=0x7f6aa8009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 msgr2=0x7f6ab80fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00eb40 tx=0x7f6aac00b040 comp rx=0 tx=0).stop 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 shutdown_connections 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f6aa00692b0 0x7f6aa006b760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ab8066260 0x7f6ab80fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 --2- 172.21.15.63:0/3700104045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ab80fff40 0x7f6ab80fb900 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.839+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 >> 172.21.15.63:0/3700104045 conn(0x7f6ab80f7430 msgr2=0x7f6ab80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.840+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 shutdown_connections 2024-04-24T22:44:23.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:23.840+0000 7f6abdec0700 1 -- 172.21.15.63:0/3700104045 wait complete. 2024-04-24T22:44:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:23 smithi063 ceph-mon[26912]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:23 smithi063 ceph-mon[26912]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:23 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2504489056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:44:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:23 smithi192 ceph-mon[33991]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:23 smithi192 ceph-mon[33991]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:23 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2504489056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:44:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:24 smithi192 ceph-mon[33991]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:24 smithi192 ceph-mon[33991]: pgmap v235: 65 pgs: 65 active+clean; 15 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-24T22:44:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:24 smithi192 ceph-mon[33991]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:24 smithi063 ceph-mon[26912]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:24 smithi063 ceph-mon[26912]: pgmap v235: 65 pgs: 65 active+clean; 15 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-24T22:44:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:24 smithi063 ceph-mon[26912]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:26.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:26 smithi192 ceph-mon[33991]: pgmap v236: 65 pgs: 65 active+clean; 16 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 54 op/s 2024-04-24T22:44:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:26 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:26 smithi063 ceph-mon[26912]: pgmap v236: 65 pgs: 65 active+clean; 16 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 54 op/s 2024-04-24T22:44:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:26 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:29.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:28 smithi192 ceph-mon[33991]: pgmap v237: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 113 op/s 2024-04-24T22:44:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:28 smithi063 ceph-mon[26912]: pgmap v237: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 113 op/s 2024-04-24T22:44:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:30 smithi192 ceph-mon[33991]: pgmap v238: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 128 op/s 2024-04-24T22:44:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:30 smithi063 ceph-mon[26912]: pgmap v238: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 128 op/s 2024-04-24T22:44:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:32 smithi192 ceph-mon[33991]: pgmap v239: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 135 op/s 2024-04-24T22:44:33.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:32 smithi063 ceph-mon[26912]: pgmap v239: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 135 op/s 2024-04-24T22:44:35.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:34 smithi192 ceph-mon[33991]: pgmap v240: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 190 op/s 2024-04-24T22:44:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:34 smithi063 ceph-mon[26912]: pgmap v240: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 190 op/s 2024-04-24T22:44:36.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: Upgrade: Updating mgr.smithi192.ctmwwc 2024-04-24T22:44:36.786 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:36.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:44:36.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:44:36.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:44:36.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:44:36.787 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:36 smithi192 ceph-mon[33991]: pgmap v241: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 171 op/s 2024-04-24T22:44:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: Upgrade: Updating mgr.smithi192.ctmwwc 2024-04-24T22:44:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:44:36.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:44:36.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:44:36.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:44:36.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:36 smithi063 ceph-mon[26912]: pgmap v241: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 171 op/s 2024-04-24T22:44:38.994 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:38 smithi192 ceph-mon[33991]: pgmap v242: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.8 MiB/s wr, 214 op/s 2024-04-24T22:44:39.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:38 smithi063 ceph-mon[26912]: pgmap v242: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.8 MiB/s wr, 214 op/s 2024-04-24T22:44:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:40 smithi063 ceph-mon[26912]: pgmap v243: 65 pgs: 65 active+clean; 88 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 176 op/s 2024-04-24T22:44:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:40 smithi192 ceph-mon[33991]: pgmap v243: 65 pgs: 65 active+clean; 88 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 176 op/s 2024-04-24T22:44:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:41 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:42.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:41 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:42.954 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:42 smithi063 ceph-mon[26912]: pgmap v244: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 182 op/s 2024-04-24T22:44:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:42 smithi192 ceph-mon[33991]: pgmap v244: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 182 op/s 2024-04-24T22:44:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:44 smithi063 ceph-mon[26912]: pgmap v245: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.8 MiB/s wr, 208 op/s 2024-04-24T22:44:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:44 smithi192 ceph-mon[33991]: pgmap v245: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.8 MiB/s wr, 208 op/s 2024-04-24T22:44:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:46 smithi192 ceph-mon[33991]: pgmap v246: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 157 op/s 2024-04-24T22:44:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:46 smithi063 ceph-mon[26912]: pgmap v246: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 157 op/s 2024-04-24T22:44:49.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:48 smithi063 ceph-mon[26912]: pgmap v247: 65 pgs: 65 active+clean; 119 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 201 op/s 2024-04-24T22:44:49.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:48 smithi192 ceph-mon[33991]: pgmap v247: 65 pgs: 65 active+clean; 119 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 201 op/s 2024-04-24T22:44:50.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:50 smithi192 ceph-mon[33991]: pgmap v248: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 172 op/s 2024-04-24T22:44:51.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:50 smithi063 ceph-mon[26912]: pgmap v248: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 172 op/s 2024-04-24T22:44:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:52 smithi192 ceph-mon[33991]: pgmap v249: 65 pgs: 65 active+clean; 126 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 166 op/s 2024-04-24T22:44:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:52 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:44:53.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:52 smithi063 ceph-mon[26912]: pgmap v249: 65 pgs: 65 active+clean; 126 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 166 op/s 2024-04-24T22:44:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:44:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:52 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:44:53.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- 172.21.15.63:0/478388700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 msgr2=0x7f22d80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 --2- 172.21.15.63:0/478388700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d80fd270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f22cc004940 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- 172.21.15.63:0/478388700 shutdown_connections 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 --2- 172.21.15.63:0/478388700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80fdfd0 0x7f22d80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 --2- 172.21.15.63:0/478388700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d80fd270 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- 172.21.15.63:0/478388700 >> 172.21.15.63:0/478388700 conn(0x7f22d80f8540 msgr2=0x7f22d80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- 172.21.15.63:0/478388700 shutdown_connections 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- 172.21.15.63:0/478388700 wait complete. 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 Processor -- start 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.956+0000 7f22ded4a700 1 -- start start 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22ded4a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22ded4a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80fdfd0 0x7f22d80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22ded4a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80707f0 con 0x7f22d80fce60 2024-04-24T22:44:53.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22ded4a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8070960 con 0x7f22d80fdfd0 2024-04-24T22:44:53.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:53.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39180/0 (socket says 172.21.15.63:39180) 2024-04-24T22:44:53.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 -- 172.21.15.63:0/2940861602 learned_addr learned my addr 172.21.15.63:0/2940861602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 -- 172.21.15.63:0/2940861602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80fdfd0 msgr2=0x7f22d80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80fdfd0 0x7f22d80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 -- 172.21.15.63:0/2940861602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc005040 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22dcae6700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b3b0 tx=0x7f22cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.957+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc00e040 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.958+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d8070be0 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.958+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22cc00bcc0 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.958+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc01c630 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.958+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8115a40 con 0x7f22d80fce60 2024-04-24T22:44:53.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.959+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d804c710 con 0x7f22d80fce60 2024-04-24T22:44:53.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.959+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f22cc013980 con 0x7f22d80fce60 2024-04-24T22:44:53.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.960+0000 7f22d57fa700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 0x7f22c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:53.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.960+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f22cc088010 con 0x7f22d80fce60 2024-04-24T22:44:53.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.960+0000 7f22d7fff700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 0x7f22c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:53.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.960+0000 7f22d7fff700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 0x7f22c806b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006db0 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:53.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:53.964+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f22cc057d00 con 0x7f22d80fce60 2024-04-24T22:44:54.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.186+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22d8102270 con 0x7f22c80692a0 2024-04-24T22:44:54.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.187+0000 7f22d57fa700 1 -- 172.21.15.63:0/2940861602 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f22d8102270 con 0x7f22c80692a0 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.189+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 msgr2=0x7f22c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.189+0000 7f22ded4a700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 0x7f22c806b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006db0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.189+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 msgr2=0x7f22d8071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.189+0000 7f22ded4a700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b3b0 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 shutdown_connections 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f22c80692a0 0x7f22c806b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d80fce60 0x7f22d8071c00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 --2- 172.21.15.63:0/2940861602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80fdfd0 0x7f22d80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 >> 172.21.15.63:0/2940861602 conn(0x7f22d80f8540 msgr2=0x7f22d80f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:54.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 shutdown_connections 2024-04-24T22:44:54.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.190+0000 7f22ded4a700 1 -- 172.21.15.63:0/2940861602 wait complete. 2024-04-24T22:44:54.202 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:44:54.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 -- 172.21.15.63:0/3204225057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 msgr2=0x7f8b680ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 --2- 172.21.15.63:0/3204225057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b680ffa30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b58004940 tx=0x7f8b58009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 -- 172.21.15.63:0/3204225057 shutdown_connections 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 --2- 172.21.15.63:0/3204225057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 0x7f8b68102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 --2- 172.21.15.63:0/3204225057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b680ffa30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 -- 172.21.15.63:0/3204225057 >> 172.21.15.63:0/3204225057 conn(0x7f8b680f7420 msgr2=0x7f8b680f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 -- 172.21.15.63:0/3204225057 shutdown_connections 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.306+0000 7f8b6d702700 1 -- 172.21.15.63:0/3204225057 wait complete. 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 Processor -- start 2024-04-24T22:44:54.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 -- start start 2024-04-24T22:44:54.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 0x7f8b6810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6810e530 con 0x7f8b68066ad0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b6d702700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6810ed30 con 0x7f8b680fff70 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b66ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b66ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39198/0 (socket says 172.21.15.63:39198) 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b66ffd700 1 -- 172.21.15.63:0/1197141442 learned_addr learned my addr 172.21.15.63:0/1197141442 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.307+0000 7f8b667fc700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 0x7f8b6810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b66ffd700 1 -- 172.21.15.63:0/1197141442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 msgr2=0x7f8b6810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b66ffd700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 0x7f8b6810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b66ffd700 1 -- 172.21.15.63:0/1197141442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58005040 con 0x7f8b68066ad0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b66ffd700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5800e040 con 0x7f8b68066ad0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b5800bbf0 con 0x7f8b68066ad0 2024-04-24T22:44:54.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6810efb0 con 0x7f8b68066ad0 2024-04-24T22:44:54.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5801c630 con 0x7f8b68066ad0 2024-04-24T22:44:54.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.308+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6810f4f0 con 0x7f8b68066ad0 2024-04-24T22:44:54.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.309+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6805f5c0 con 0x7f8b68066ad0 2024-04-24T22:44:54.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.310+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8b5800bd60 con 0x7f8b68066ad0 2024-04-24T22:44:54.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.310+0000 7f8b4f7fe700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.310+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8b580884e0 con 0x7f8b68066ad0 2024-04-24T22:44:54.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.310+0000 7f8b667fc700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.311+0000 7f8b667fc700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 0x7f8b5006b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009fd0 tx=0x7f8b5c00b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:54.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.314+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8b58058250 con 0x7f8b68066ad0 2024-04-24T22:44:54.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.573+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b6810f870 con 0x7f8b500692a0 2024-04-24T22:44:54.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.574+0000 7f8b4f7fe700 1 -- 172.21.15.63:0/1197141442 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f8b6810f870 con 0x7f8b500692a0 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 msgr2=0x7f8b5006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 0x7f8b5006b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009fd0 tx=0x7f8b5c00b040 comp rx=0 tx=0).stop 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 msgr2=0x7f8b6810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b58009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 shutdown_connections 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b68066ad0 0x7f8b6810d9f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 --2- 172.21.15.63:0/1197141442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b680fff70 0x7f8b6810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 >> 172.21.15.63:0/1197141442 conn(0x7f8b680f7420 msgr2=0x7f8b680f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 shutdown_connections 2024-04-24T22:44:54.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.577+0000 7f8b6d702700 1 -- 172.21.15.63:0/1197141442 wait complete. 2024-04-24T22:44:54.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 -- 172.21.15.63:0/1667052627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 msgr2=0x7f65880fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 --2- 172.21.15.63:0/1667052627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65880fd3d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6578004940 tx=0x7f6578009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 -- 172.21.15.63:0/1667052627 shutdown_connections 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 --2- 172.21.15.63:0/1667052627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65880fd3d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 --2- 172.21.15.63:0/1667052627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 0x7f65880fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 -- 172.21.15.63:0/1667052627 >> 172.21.15.63:0/1667052627 conn(0x7f65880f7430 msgr2=0x7f65880f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 -- 172.21.15.63:0/1667052627 shutdown_connections 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.685+0000 7f658fd7b700 1 -- 172.21.15.63:0/1667052627 wait complete. 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 Processor -- start 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 -- start start 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 0x7f6588112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6588066220 con 0x7f65880fcf60 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658fd7b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6588066360 con 0x7f65880fbcb0 2024-04-24T22:44:54.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39212/0 (socket says 172.21.15.63:39212) 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 -- 172.21.15.63:0/3831880335 learned_addr learned my addr 172.21.15.63:0/3831880335 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658db17700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 0x7f6588112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 -- 172.21.15.63:0/3831880335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 msgr2=0x7f6588112b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 0x7f6588112b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.686+0000 7f658d316700 1 -- 172.21.15.63:0/3831880335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6578005040 con 0x7f65880fcf60 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f658d316700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f657800b3b0 tx=0x7f6578009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657800e040 con 0x7f65880fcf60 2024-04-24T22:44:54.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65881155e0 con 0x7f65880fcf60 2024-04-24T22:44:54.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657800bbf0 con 0x7f65880fcf60 2024-04-24T22:44:54.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657801c630 con 0x7f65880fcf60 2024-04-24T22:44:54.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.687+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881159b0 con 0x7f65880fcf60 2024-04-24T22:44:54.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.688+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658805f5c0 con 0x7f65880fcf60 2024-04-24T22:44:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.689+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f657800bd60 con 0x7f65880fcf60 2024-04-24T22:44:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.689+0000 7f657e7fc700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 0x7f657406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.689+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6578087ee0 con 0x7f65880fcf60 2024-04-24T22:44:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.689+0000 7f658db17700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 0x7f657406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.690+0000 7f658db17700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 0x7f657406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6588065fb0 tx=0x7f658400b040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:54.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.693+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f65780541c0 con 0x7f65880fcf60 2024-04-24T22:44:54.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.926+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f65881012c0 con 0x7f65740692a0 2024-04-24T22:44:54.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.936+0000 7f657e7fc700 1 -- 172.21.15.63:0/3831880335 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7f65881012c0 con 0x7f65740692a0 2024-04-24T22:44:54.938 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:44:54.938 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 2m ago 7m 24.5M - 0.25.0 c8568f914cd2 f1e9967ae2bb 2024-04-24T22:44:54.938 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 2m ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (7m) 2m ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 2m ago 7m 85.4M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (2m) 2m ago 2m 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (2m) 2m ago 2m 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (2m) 2m ago 2m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (2m) 2m ago 2m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 starting - - - - 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (9m) 2m ago 9m 46.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (6m) 2m ago 6m 39.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 2m ago 8m 15.4M - 1.5.0 0da6a335fe13 b97c5a0b7f9e 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 2m ago 6m 15.4M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 6m 48.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:44:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 2m ago 5m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:44:54.940 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 2m ago 5m 44.4M 4288M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:44:54.940 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (4m) 2m ago 4m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:44:54.940 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (4m) 2m ago 4m 41.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:44:54.940 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (3m) 2m ago 3m 42.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:44:54.940 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 2m ago 7m 53.9M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 msgr2=0x7f657406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 0x7f657406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6588065fb0 tx=0x7f658400b040 comp rx=0 tx=0).stop 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 msgr2=0x7f65881150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f657800b3b0 tx=0x7f6578009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 shutdown_connections 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f65740692a0 0x7f657406b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbcb0 0x7f6588112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 --2- 172.21.15.63:0/3831880335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcf60 0x7f65881150a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 >> 172.21.15.63:0/3831880335 conn(0x7f65880f7430 msgr2=0x7f65880f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 shutdown_connections 2024-04-24T22:44:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:54.939+0000 7f658fd7b700 1 -- 172.21.15.63:0/3831880335 wait complete. 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 -- 172.21.15.63:0/552164736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fdfd0 msgr2=0x7fdf940fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 --2- 172.21.15.63:0/552164736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fdfd0 0x7fdf940fe440 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c004940 tx=0x7fdf7c009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 -- 172.21.15.63:0/552164736 shutdown_connections 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 --2- 172.21.15.63:0/552164736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fdfd0 0x7fdf940fe440 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 --2- 172.21.15.63:0/552164736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fce60 0x7fdf940fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 -- 172.21.15.63:0/552164736 >> 172.21.15.63:0/552164736 conn(0x7fdf940f8540 msgr2=0x7fdf940fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 -- 172.21.15.63:0/552164736 shutdown_connections 2024-04-24T22:44:55.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.045+0000 7fdf9976e700 1 -- 172.21.15.63:0/552164736 wait complete. 2024-04-24T22:44:55.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 Processor -- start 2024-04-24T22:44:55.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 -- start start 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fdfd0 0x7fdf940702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94072200 con 0x7fdf940fce60 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf9976e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf940707f0 con 0x7fdf940fdfd0 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf927fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fdfd0 0x7fdf940702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39228/0 (socket says 172.21.15.63:39228) 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 -- 172.21.15.63:0/430000543 learned_addr learned my addr 172.21.15.63:0/430000543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 -- 172.21.15.63:0/430000543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fdfd0 msgr2=0x7fdf940702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fdfd0 0x7fdf940702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.046+0000 7fdf92ffd700 1 -- 172.21.15.63:0/430000543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c005040 con 0x7fdf940fce60 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf92ffd700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf84001d80 tx=0x7fdf84009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf84007d10 con 0x7fdf940fce60 2024-04-24T22:44:55.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf84001e80 con 0x7fdf940fce60 2024-04-24T22:44:55.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf94070aa0 con 0x7fdf940fce60 2024-04-24T22:44:55.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf84014640 con 0x7fdf940fce60 2024-04-24T22:44:55.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.047+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf94071060 con 0x7fdf940fce60 2024-04-24T22:44:55.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.048+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9404c710 con 0x7fdf940fce60 2024-04-24T22:44:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.048+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fdf8400e960 con 0x7fdf940fce60 2024-04-24T22:44:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.049+0000 7fdf8b7fe700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 0x7fdf8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.049+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdf84086eb0 con 0x7fdf940fce60 2024-04-24T22:44:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.049+0000 7fdf927fc700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 0x7fdf8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.049+0000 7fdf927fc700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 0x7fdf8006b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00b3b0 tx=0x7fdf7c009e30 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:55.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.053+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdf8404e250 con 0x7fdf940fce60 2024-04-24T22:44:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:54 smithi192 ceph-mon[33991]: pgmap v250: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 203 op/s 2024-04-24T22:44:55.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:54 smithi063 ceph-mon[26912]: pgmap v250: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 203 op/s 2024-04-24T22:44:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.395+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf94115cc0 con 0x7fdf940fce60 2024-04-24T22:44:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.396+0000 7fdf8b7fe700 1 -- 172.21.15.63:0/430000543 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fdf84056cd0 con 0x7fdf940fce60 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:55.398 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:44:55.399 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:44:55.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 msgr2=0x7fdf8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 0x7fdf8006b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00b3b0 tx=0x7fdf7c009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 msgr2=0x7fdf94071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf84001d80 tx=0x7fdf84009ec0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 shutdown_connections 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7fdf800692a0 0x7fdf8006b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf940fce60 0x7fdf94071c00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 --2- 172.21.15.63:0/430000543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf940fdfd0 0x7fdf940702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 >> 172.21.15.63:0/430000543 conn(0x7fdf940f8540 msgr2=0x7fdf940f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.398+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 shutdown_connections 2024-04-24T22:44:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.399+0000 7fdf9976e700 1 -- 172.21.15.63:0/430000543 wait complete. 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/1724701107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 msgr2=0x7f0c180fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/1724701107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180fd6d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c08004940 tx=0x7f0c08009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/1724701107 shutdown_connections 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/1724701107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fdc10 0x7f0c180fc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/1724701107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180fd6d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/1724701107 >> 172.21.15.63:0/1724701107 conn(0x7f0c180f6bb0 msgr2=0x7f0c180f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/1724701107 shutdown_connections 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.528+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/1724701107 wait complete. 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 Processor -- start 2024-04-24T22:44:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 -- start start 2024-04-24T22:44:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fe320 0x7f0c180fe790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181001f0 con 0x7f0c180fd2c0 2024-04-24T22:44:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c1e7b4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180fecd0 con 0x7f0c180fe320 2024-04-24T22:44:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.529+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39252/0 (socket says 172.21.15.63:39252) 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c17fff700 1 -- 172.21.15.63:0/877739721 learned_addr learned my addr 172.21.15.63:0/877739721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c177fe700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fe320 0x7f0c180fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c17fff700 1 -- 172.21.15.63:0/877739721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fe320 msgr2=0x7f0c180fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c17fff700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fe320 0x7f0c180fe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c17fff700 1 -- 172.21.15.63:0/877739721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08005040 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c17fff700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fd0 tx=0x7f0c08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0800e040 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c180fef50 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c0800bcc0 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0801c630 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.530+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18115dd0 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.531+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18101b70 con 0x7f0c180fd2c0 2024-04-24T22:44:55.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.532+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0c08013420 con 0x7f0c180fd2c0 2024-04-24T22:44:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.532+0000 7f0c14ff9700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 0x7f0c0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.532+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0c080884e0 con 0x7f0c180fd2c0 2024-04-24T22:44:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.533+0000 7f0c177fe700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 0x7f0c0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.533+0000 7f0c177fe700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 0x7f0c0006b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c006fd0 tx=0x7f0c0c008040 comp rx=0 tx=0).ready entity=mgr.14216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:55.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.538+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c08058250 con 0x7f0c180fd2c0 2024-04-24T22:44:55.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.782+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 --> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c180fdc80 con 0x7f0c000692a0 2024-04-24T22:44:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.784+0000 7f0c14ff9700 1 -- 172.21.15.63:0/877739721 <== mgr.14216 v2:172.21.15.63:6800/622073725 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f0c180fdc80 con 0x7f0c000692a0 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:44:55.786 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 msgr2=0x7f0c0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 0x7f0c0006b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c006fd0 tx=0x7f0c0c008040 comp rx=0 tx=0).stop 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 msgr2=0x7f0c180ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fd0 tx=0x7f0c08009e30 comp rx=0 tx=0).stop 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 shutdown_connections 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:6800/622073725,v1:172.21.15.63:6801/622073725] conn(0x7f0c000692a0 0x7f0c0006b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c180fd2c0 0x7f0c180ffbd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 --2- 172.21.15.63:0/877739721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c180fe320 0x7f0c180fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 >> 172.21.15.63:0/877739721 conn(0x7f0c180f6bb0 msgr2=0x7f0c180fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 shutdown_connections 2024-04-24T22:44:55.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:44:55.786+0000 7f0c1e7b4700 1 -- 172.21.15.63:0/877739721 wait complete. 2024-04-24T22:44:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:55 smithi063 ceph-mon[26912]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:55 smithi063 ceph-mon[26912]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:55 smithi063 ceph-mon[26912]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:55 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/430000543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:44:56.146 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:55 smithi192 ceph-mon[33991]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:55 smithi192 ceph-mon[33991]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:55 smithi192 ceph-mon[33991]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:56.147 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:55 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/430000543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:44:57.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:56 smithi192 ceph-mon[33991]: pgmap v251: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 179 op/s 2024-04-24T22:44:57.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:56 smithi192 ceph-mon[33991]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:57.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:56 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:57.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:56 smithi063 ceph-mon[26912]: pgmap v251: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 179 op/s 2024-04-24T22:44:57.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:56 smithi063 ceph-mon[26912]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:44:57.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:56 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:44:59.144 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:58 smithi192 ceph-mon[33991]: pgmap v252: 65 pgs: 65 active+clean; 156 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 256 op/s 2024-04-24T22:44:59.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:58 smithi063 ceph-mon[26912]: pgmap v252: 65 pgs: 65 active+clean; 156 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 256 op/s 2024-04-24T22:45:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:45:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:45:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:45:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:45:00.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:45:00.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:44:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:45:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:45:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:45:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:45:00.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:45:00.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:45:00.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:44:59 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:45:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:00 smithi192 ceph-mon[33991]: pgmap v253: 65 pgs: 65 active+clean; 160 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 238 op/s 2024-04-24T22:45:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:00 smithi192 ceph-mon[33991]: mgrmap e20: smithi063.upcnvd(active, since 7m), standbys: smithi192.ctmwwc 2024-04-24T22:45:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:00 smithi063 ceph-mon[26912]: pgmap v253: 65 pgs: 65 active+clean; 160 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 238 op/s 2024-04-24T22:45:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:00 smithi063 ceph-mon[26912]: mgrmap e20: smithi063.upcnvd(active, since 7m), standbys: smithi192.ctmwwc 2024-04-24T22:45:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:02 smithi192 ceph-mon[33991]: pgmap v254: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-24T22:45:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:02 smithi063 ceph-mon[26912]: pgmap v254: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-24T22:45:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:04 smithi192 ceph-mon[33991]: pgmap v255: 65 pgs: 65 active+clean; 168 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 282 op/s 2024-04-24T22:45:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:04 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:04 smithi063 ceph-mon[26912]: pgmap v255: 65 pgs: 65 active+clean; 168 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 282 op/s 2024-04-24T22:45:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:04 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:06.646 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:06.647 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:06 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:06.647 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:06 smithi192 ceph-mon[33991]: pgmap v256: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-24T22:45:06.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:06.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:06 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:06.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:06 smithi063 ceph-mon[26912]: pgmap v256: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-24T22:45:09.048 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T22:45:09.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:08 smithi063 ceph-mon[26912]: pgmap v257: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 367 op/s 2024-04-24T22:45:09.054 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T22:45:09.054 INFO:tasks.workunit.client.1.smithi192.stderr:+ make 2024-04-24T22:45:09.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:08 smithi192 ceph-mon[33991]: pgmap v257: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 367 op/s 2024-04-24T22:45:09.618 INFO:tasks.workunit.client.1.smithi192.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T22:45:10.077 INFO:tasks.workunit.client.1.smithi192.stderr:++ readlink -f fsstress 2024-04-24T22:45:10.079 INFO:tasks.workunit.client.1.smithi192.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T22:45:10.079 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-04-24T22:45:10.080 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T22:45:10.080 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-04-24T22:45:10.080 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-24T22:45:10.081 INFO:tasks.workunit.client.1.smithi192.stderr:++ mktemp -d -p . 2024-04-24T22:45:10.084 INFO:tasks.workunit.client.1.smithi192.stderr:+ T=./tmp.gixOwSWXYz 2024-04-24T22:45:10.084 INFO:tasks.workunit.client.1.smithi192.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.gixOwSWXYz -l 1 -n 1000 -p 10 -v 2024-04-24T22:45:10.089 INFO:tasks.workunit.client.1.smithi192.stdout:seed = 1713952638 2024-04-24T22:45:10.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/0: rename - no filename 2024-04-24T22:45:10.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/1: stat - no entries 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/0: dread - no filename 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/1: write - no filename 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/2: write - no filename 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/3: dread - no filename 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/4: rmdir - no directory 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/5: dwrite - no filename 2024-04-24T22:45:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/6: link - no file 2024-04-24T22:45:10.102 INFO:tasks.workunit.client.1.smithi192.stdout:0/0: symlink l0 0 2024-04-24T22:45:10.102 INFO:tasks.workunit.client.1.smithi192.stdout:0/1: dwrite - no filename 2024-04-24T22:45:10.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/0: write - no filename 2024-04-24T22:45:10.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/1: dread - no filename 2024-04-24T22:45:10.104 INFO:tasks.workunit.client.1.smithi192.stdout:1/2: creat f0 x:0 0 0 2024-04-24T22:45:10.106 INFO:tasks.workunit.client.1.smithi192.stdout:0/2: creat f1 x:0 0 0 2024-04-24T22:45:10.106 INFO:tasks.workunit.client.1.smithi192.stdout:0/3: chown f1 0 1 2024-04-24T22:45:10.107 INFO:tasks.workunit.client.1.smithi192.stdout:0/4: write f1 [357472,103244] 0 2024-04-24T22:45:10.109 INFO:tasks.workunit.client.1.smithi192.stdout:2/7: creat f0 x:0 0 0 2024-04-24T22:45:10.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/5: creat f2 x:0 0 0 2024-04-24T22:45:10.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/6: write f2 [642009,12120] 0 2024-04-24T22:45:10.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/7: chown f2 33690511 1 2024-04-24T22:45:10.118 INFO:tasks.workunit.client.1.smithi192.stdout:3/2: mkdir d0 0 2024-04-24T22:45:10.118 INFO:tasks.workunit.client.1.smithi192.stdout:3/3: write - no filename 2024-04-24T22:45:10.122 INFO:tasks.workunit.client.1.smithi192.stdout:0/8: mkdir d3 0 2024-04-24T22:45:10.125 INFO:tasks.workunit.client.1.smithi192.stdout:7/0: fdatasync - no filename 2024-04-24T22:45:10.125 INFO:tasks.workunit.client.1.smithi192.stdout:7/1: write - no filename 2024-04-24T22:45:10.125 INFO:tasks.workunit.client.1.smithi192.stdout:7/2: stat - no entries 2024-04-24T22:45:10.128 INFO:tasks.workunit.client.1.smithi192.stdout:4/0: getdents . 0 2024-04-24T22:45:10.128 INFO:tasks.workunit.client.1.smithi192.stdout:4/1: write - no filename 2024-04-24T22:45:10.135 INFO:tasks.workunit.client.1.smithi192.stdout:8/0: write - no filename 2024-04-24T22:45:10.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/0: mkdir d0 0 2024-04-24T22:45:10.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/1: fsync - no filename 2024-04-24T22:45:10.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/2: truncate - no filename 2024-04-24T22:45:10.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/3: write - no filename 2024-04-24T22:45:10.140 INFO:tasks.workunit.client.1.smithi192.stdout:6/4: stat d0 0 2024-04-24T22:45:10.144 INFO:tasks.workunit.client.1.smithi192.stdout:3/4: mkdir d0/d1 0 2024-04-24T22:45:10.144 INFO:tasks.workunit.client.1.smithi192.stdout:3/5: dwrite - no filename 2024-04-24T22:45:10.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/6: chown d0 0 1 2024-04-24T22:45:10.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/7: unlink - no file 2024-04-24T22:45:10.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/8: link - no file 2024-04-24T22:45:10.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/9: fsync - no filename 2024-04-24T22:45:10.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/9: symlink d3/l4 0 2024-04-24T22:45:10.147 INFO:tasks.workunit.client.1.smithi192.stdout:0/10: fdatasync f2 0 2024-04-24T22:45:10.154 INFO:tasks.workunit.client.1.smithi192.stdout:7/3: creat f0 x:0 0 0 2024-04-24T22:45:10.157 INFO:tasks.workunit.client.1.smithi192.stdout:4/2: getdents . 0 2024-04-24T22:45:10.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/11: mknod d3/c5 0 2024-04-24T22:45:10.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/1: symlink l0 0 2024-04-24T22:45:10.167 INFO:tasks.workunit.client.1.smithi192.stdout:0/12: dread f1 [0,4194304] 0 2024-04-24T22:45:10.168 INFO:tasks.workunit.client.1.smithi192.stdout:0/13: write f1 [1422921,43611] 0 2024-04-24T22:45:10.170 INFO:tasks.workunit.client.1.smithi192.stdout:7/4: creat f1 x:0 0 0 2024-04-24T22:45:10.171 INFO:tasks.workunit.client.1.smithi192.stdout:6/5: mknod d0/c1 0 2024-04-24T22:45:10.181 INFO:tasks.workunit.client.1.smithi192.stdout:0/14: rename l0 to d3/l6 0 2024-04-24T22:45:10.183 INFO:tasks.workunit.client.1.smithi192.stdout:7/5: mkdir d2 0 2024-04-24T22:45:10.184 INFO:tasks.workunit.client.1.smithi192.stdout:7/6: chown f1 3 1 2024-04-24T22:45:10.184 INFO:tasks.workunit.client.1.smithi192.stdout:7/7: dread - f0 zero size 2024-04-24T22:45:10.185 INFO:tasks.workunit.client.1.smithi192.stdout:6/6: creat d0/f2 x:0 0 0 2024-04-24T22:45:10.186 INFO:tasks.workunit.client.1.smithi192.stdout:6/7: rename d0 to d0/d3 22 2024-04-24T22:45:10.187 INFO:tasks.workunit.client.1.smithi192.stdout:4/3: symlink l0 0 2024-04-24T22:45:10.189 INFO:tasks.workunit.client.1.smithi192.stdout:3/10: rmdir d0/d1 0 2024-04-24T22:45:10.193 INFO:tasks.workunit.client.1.smithi192.stdout:9/0: symlink l0 0 2024-04-24T22:45:10.203 INFO:tasks.workunit.client.1.smithi192.stdout:6/8: rename d0/c1 to d0/c4 0 2024-04-24T22:45:10.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/15: dwrite f1 [0,4194304] 0 2024-04-24T22:45:10.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/9: write d0/f2 [566939,80055] 0 2024-04-24T22:45:10.205 INFO:tasks.workunit.client.1.smithi192.stdout:4/4: symlink l1 0 2024-04-24T22:45:10.206 INFO:tasks.workunit.client.1.smithi192.stdout:4/5: readlink l1 0 2024-04-24T22:45:10.210 INFO:tasks.workunit.client.1.smithi192.stdout:0/16: dread f1 [0,4194304] 0 2024-04-24T22:45:10.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/17: write f2 [1409818,115089] 0 2024-04-24T22:45:10.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/18: chown d3/l4 215535 1 2024-04-24T22:45:10.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/1: creat f1 x:0 0 0 2024-04-24T22:45:10.215 INFO:tasks.workunit.client.1.smithi192.stdout:1/3: fsync f0 0 2024-04-24T22:45:10.217 INFO:tasks.workunit.client.1.smithi192.stdout:7/8: symlink d2/l3 0 2024-04-24T22:45:10.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/4: write f0 [707462,14448] 0 2024-04-24T22:45:10.222 INFO:tasks.workunit.client.1.smithi192.stdout:6/10: mknod d0/c5 0 2024-04-24T22:45:10.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/11: truncate d0/f2 866981 0 2024-04-24T22:45:10.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/12: readlink - no filename 2024-04-24T22:45:10.223 INFO:tasks.workunit.client.1.smithi192.stdout:0/19: dwrite f2 [0,4194304] 0 2024-04-24T22:45:10.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/20: chown f2 176 1 2024-04-24T22:45:10.225 INFO:tasks.workunit.client.1.smithi192.stdout:2/8: fsync f0 0 2024-04-24T22:45:10.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/6: mkdir d2 0 2024-04-24T22:45:10.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/7: fsync - no filename 2024-04-24T22:45:10.228 INFO:tasks.workunit.client.1.smithi192.stdout:4/8: chown d2 488 1 2024-04-24T22:45:10.232 INFO:tasks.workunit.client.1.smithi192.stdout:9/2: creat f2 x:0 0 0 2024-04-24T22:45:10.236 INFO:tasks.workunit.client.1.smithi192.stdout:1/5: symlink l1 0 2024-04-24T22:45:10.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/9: dwrite f0 [0,4194304] 0 2024-04-24T22:45:10.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/10: dread f0 [0,4194304] 0 2024-04-24T22:45:10.249 INFO:tasks.workunit.client.1.smithi192.stdout:9/3: mkdir d3 0 2024-04-24T22:45:10.252 INFO:tasks.workunit.client.1.smithi192.stdout:1/6: creat f2 x:0 0 0 2024-04-24T22:45:10.253 INFO:tasks.workunit.client.1.smithi192.stdout:1/7: write f0 [1665218,110298] 0 2024-04-24T22:45:10.260 INFO:tasks.workunit.client.1.smithi192.stdout:4/9: creat d2/f3 x:0 0 0 2024-04-24T22:45:10.262 INFO:tasks.workunit.client.1.smithi192.stdout:2/11: mknod c1 0 2024-04-24T22:45:10.269 INFO:tasks.workunit.client.1.smithi192.stdout:4/10: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:10.269 INFO:tasks.workunit.client.1.smithi192.stdout:1/8: rename f0 to f3 0 2024-04-24T22:45:10.271 INFO:tasks.workunit.client.1.smithi192.stdout:7/9: link d2/l3 d2/l4 0 2024-04-24T22:45:10.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/4: creat d3/f4 x:0 0 0 2024-04-24T22:45:10.278 INFO:tasks.workunit.client.1.smithi192.stdout:9/5: dread - f2 zero size 2024-04-24T22:45:10.279 INFO:tasks.workunit.client.1.smithi192.stdout:9/6: dread - f2 zero size 2024-04-24T22:45:10.280 INFO:tasks.workunit.client.1.smithi192.stdout:9/7: chown f2 4 1 2024-04-24T22:45:10.280 INFO:tasks.workunit.client.1.smithi192.stdout:9/8: dread - f1 zero size 2024-04-24T22:45:10.280 INFO:tasks.workunit.client.1.smithi192.stdout:9/9: fdatasync f1 0 2024-04-24T22:45:10.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/10: chown f1 4607 1 2024-04-24T22:45:10.284 INFO:tasks.workunit.client.1.smithi192.stdout:1/9: symlink l4 0 2024-04-24T22:45:10.293 INFO:tasks.workunit.client.1.smithi192.stdout:1/10: dread f3 [0,4194304] 0 2024-04-24T22:45:10.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/21: readlink d3/l6 0 2024-04-24T22:45:10.296 INFO:tasks.workunit.client.1.smithi192.stdout:9/11: mknod d3/c5 0 2024-04-24T22:45:10.298 INFO:tasks.workunit.client.1.smithi192.stdout:7/10: chown d2/l3 369 1 2024-04-24T22:45:10.301 INFO:tasks.workunit.client.1.smithi192.stdout:1/11: mknod c5 0 2024-04-24T22:45:10.301 INFO:tasks.workunit.client.1.smithi192.stdout:1/12: read f3 [504121,40481] 0 2024-04-24T22:45:10.305 INFO:tasks.workunit.client.1.smithi192.stdout:4/11: fdatasync d2/f3 0 2024-04-24T22:45:10.308 INFO:tasks.workunit.client.1.smithi192.stdout:9/12: rename f1 to d3/f6 0 2024-04-24T22:45:10.309 INFO:tasks.workunit.client.1.smithi192.stdout:9/13: dread - d3/f4 zero size 2024-04-24T22:45:10.310 INFO:tasks.workunit.client.1.smithi192.stdout:9/14: write d3/f4 [987000,65029] 0 2024-04-24T22:45:10.317 INFO:tasks.workunit.client.1.smithi192.stdout:4/12: mknod d2/c4 0 2024-04-24T22:45:10.326 INFO:tasks.workunit.client.1.smithi192.stdout:9/15: mknod d3/c7 0 2024-04-24T22:45:10.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/16: dread - f2 zero size 2024-04-24T22:45:10.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/17: truncate f2 581080 0 2024-04-24T22:45:10.328 INFO:tasks.workunit.client.1.smithi192.stdout:9/18: truncate f2 1477335 0 2024-04-24T22:45:10.330 INFO:tasks.workunit.client.1.smithi192.stdout:9/19: dread f2 [0,4194304] 0 2024-04-24T22:45:10.331 INFO:tasks.workunit.client.1.smithi192.stdout:4/13: mknod d2/c5 0 2024-04-24T22:45:10.332 INFO:tasks.workunit.client.1.smithi192.stdout:0/22: truncate f1 2586922 0 2024-04-24T22:45:10.333 INFO:tasks.workunit.client.1.smithi192.stdout:0/23: read f2 [231472,88269] 0 2024-04-24T22:45:10.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/11: rmdir d0 0 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/12: truncate - no filename 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/13: rename - no filename 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/14: fsync - no filename 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/15: readlink - no filename 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:3/16: write - no filename 2024-04-24T22:45:10.336 INFO:tasks.workunit.client.1.smithi192.stdout:4/14: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:10.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/13: link c5 c6 0 2024-04-24T22:45:10.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/14: dread - f2 zero size 2024-04-24T22:45:10.351 INFO:tasks.workunit.client.1.smithi192.stdout:1/15: dwrite f2 [0,4194304] 0 2024-04-24T22:45:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/16: rmdir - no directory 2024-04-24T22:45:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:2/12: write f0 [4870605,70028] 0 2024-04-24T22:45:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/17: stat f2 0 2024-04-24T22:45:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:2/13: chown f0 14049021 1 2024-04-24T22:45:10.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/17: mkdir d2 0 2024-04-24T22:45:10.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/18: write - no filename 2024-04-24T22:45:10.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/19: fsync - no filename 2024-04-24T22:45:10.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/20: dread - no filename 2024-04-24T22:45:10.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/21: dwrite - no filename 2024-04-24T22:45:10.359 INFO:tasks.workunit.client.1.smithi192.stdout:7/11: getdents d2 0 2024-04-24T22:45:10.372 INFO:tasks.workunit.client.1.smithi192.stdout:2/14: creat f2 x:0 0 0 2024-04-24T22:45:10.380 INFO:tasks.workunit.client.1.smithi192.stdout:7/12: creat d2/f5 x:0 0 0 2024-04-24T22:45:10.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/15: dwrite f2 [0,4194304] 0 2024-04-24T22:45:10.385 INFO:tasks.workunit.client.1.smithi192.stdout:1/18: unlink f2 0 2024-04-24T22:45:10.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/19: truncate f3 1838632 0 2024-04-24T22:45:10.389 INFO:tasks.workunit.client.1.smithi192.stdout:3/22: creat d2/f3 x:0 0 0 2024-04-24T22:45:10.390 INFO:tasks.workunit.client.1.smithi192.stdout:1/20: dread f3 [0,4194304] 0 2024-04-24T22:45:10.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/15: link l1 d2/l6 0 2024-04-24T22:45:10.398 INFO:tasks.workunit.client.1.smithi192.stdout:3/23: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:10.400 INFO:tasks.workunit.client.1.smithi192.stdout:7/13: creat d2/f6 x:0 0 0 2024-04-24T22:45:10.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/20: rename d3/f6 to d3/f8 0 2024-04-24T22:45:10.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/21: chown d3/c5 263668 1 2024-04-24T22:45:10.415 INFO:tasks.workunit.client.1.smithi192.stdout:3/24: mkdir d2/d4 0 2024-04-24T22:45:10.416 INFO:tasks.workunit.client.1.smithi192.stdout:1/21: dwrite f3 [0,4194304] 0 2024-04-24T22:45:10.417 INFO:tasks.workunit.client.1.smithi192.stdout:3/25: write d2/f3 [3570980,60929] 0 2024-04-24T22:45:10.418 INFO:tasks.workunit.client.1.smithi192.stdout:7/14: creat d2/f7 x:0 0 0 2024-04-24T22:45:10.421 INFO:tasks.workunit.client.1.smithi192.stdout:9/22: creat d3/f9 x:0 0 0 2024-04-24T22:45:10.426 INFO:tasks.workunit.client.1.smithi192.stdout:2/16: link c1 c3 0 2024-04-24T22:45:10.426 INFO:tasks.workunit.client.1.smithi192.stdout:1/22: dwrite f3 [0,4194304] 0 2024-04-24T22:45:10.429 INFO:tasks.workunit.client.1.smithi192.stdout:0/24: dread f1 [0,4194304] 0 2024-04-24T22:45:10.432 INFO:tasks.workunit.client.1.smithi192.stdout:9/23: dread d3/f4 [0,4194304] 0 2024-04-24T22:45:10.433 INFO:tasks.workunit.client.1.smithi192.stdout:9/24: readlink l0 0 2024-04-24T22:45:10.434 INFO:tasks.workunit.client.1.smithi192.stdout:9/25: dread - d3/f8 zero size 2024-04-24T22:45:10.436 INFO:tasks.workunit.client.1.smithi192.stdout:9/26: read d3/f4 [921941,21966] 0 2024-04-24T22:45:10.439 INFO:tasks.workunit.client.1.smithi192.stdout:1/23: dwrite f3 [0,4194304] 0 2024-04-24T22:45:10.439 INFO:tasks.workunit.client.1.smithi192.stdout:7/15: symlink d2/l8 0 2024-04-24T22:45:10.440 INFO:tasks.workunit.client.1.smithi192.stdout:9/27: dread d3/f4 [0,4194304] 0 2024-04-24T22:45:10.445 INFO:tasks.workunit.client.1.smithi192.stdout:2/17: mkdir d4 0 2024-04-24T22:45:10.445 INFO:tasks.workunit.client.1.smithi192.stdout:2/18: chown d4 3147474 1 2024-04-24T22:45:10.450 INFO:tasks.workunit.client.1.smithi192.stdout:0/25: symlink d3/l7 0 2024-04-24T22:45:10.453 INFO:tasks.workunit.client.1.smithi192.stdout:2/19: dwrite f2 [0,4194304] 0 2024-04-24T22:45:10.454 INFO:tasks.workunit.client.1.smithi192.stdout:2/20: chown f2 262 1 2024-04-24T22:45:10.456 INFO:tasks.workunit.client.1.smithi192.stdout:3/26: creat d2/d4/f5 x:0 0 0 2024-04-24T22:45:10.457 INFO:tasks.workunit.client.1.smithi192.stdout:3/27: chown d2/f3 123 1 2024-04-24T22:45:10.457 INFO:tasks.workunit.client.1.smithi192.stdout:3/28: readlink - no filename 2024-04-24T22:45:10.460 INFO:tasks.workunit.client.1.smithi192.stdout:7/16: readlink d2/l3 0 2024-04-24T22:45:10.462 INFO:tasks.workunit.client.1.smithi192.stdout:1/24: symlink l7 0 2024-04-24T22:45:10.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/28: unlink d3/c7 0 2024-04-24T22:45:10.469 INFO:tasks.workunit.client.1.smithi192.stdout:1/25: dread f3 [0,4194304] 0 2024-04-24T22:45:10.470 INFO:tasks.workunit.client.1.smithi192.stdout:3/29: dwrite d2/d4/f5 [0,4194304] 0 2024-04-24T22:45:10.475 INFO:tasks.workunit.client.1.smithi192.stdout:0/26: creat d3/f8 x:0 0 0 2024-04-24T22:45:10.478 INFO:tasks.workunit.client.1.smithi192.stdout:2/21: write f2 [4280163,112528] 0 2024-04-24T22:45:10.481 INFO:tasks.workunit.client.1.smithi192.stdout:7/17: symlink d2/l9 0 2024-04-24T22:45:10.481 INFO:tasks.workunit.client.1.smithi192.stdout:7/18: truncate f0 158820 0 2024-04-24T22:45:10.488 INFO:tasks.workunit.client.1.smithi192.stdout:9/29: write d3/f4 [114243,34180] 0 2024-04-24T22:45:10.490 INFO:tasks.workunit.client.1.smithi192.stdout:1/26: mknod c8 0 2024-04-24T22:45:10.491 INFO:tasks.workunit.client.1.smithi192.stdout:0/27: symlink d3/l9 0 2024-04-24T22:45:10.496 INFO:tasks.workunit.client.1.smithi192.stdout:7/19: creat d2/fa x:0 0 0 2024-04-24T22:45:10.497 INFO:tasks.workunit.client.1.smithi192.stdout:9/30: write d3/f8 [1032840,34901] 0 2024-04-24T22:45:10.498 INFO:tasks.workunit.client.1.smithi192.stdout:7/20: write d2/fa [740837,19666] 0 2024-04-24T22:45:10.498 INFO:tasks.workunit.client.1.smithi192.stdout:7/21: dread - f1 zero size 2024-04-24T22:45:10.499 INFO:tasks.workunit.client.1.smithi192.stdout:9/31: write d3/f4 [2003375,67949] 0 2024-04-24T22:45:10.499 INFO:tasks.workunit.client.1.smithi192.stdout:7/22: chown f0 523031 1 2024-04-24T22:45:10.500 INFO:tasks.workunit.client.1.smithi192.stdout:1/27: mkdir d9 0 2024-04-24T22:45:10.501 INFO:tasks.workunit.client.1.smithi192.stdout:0/28: rename d3/l7 to d3/la 0 2024-04-24T22:45:10.502 INFO:tasks.workunit.client.1.smithi192.stdout:0/29: stat d3/f8 0 2024-04-24T22:45:10.504 INFO:tasks.workunit.client.1.smithi192.stdout:2/22: creat d4/f5 x:0 0 0 2024-04-24T22:45:10.505 INFO:tasks.workunit.client.1.smithi192.stdout:0/30: dread f1 [0,4194304] 0 2024-04-24T22:45:10.563 INFO:tasks.workunit.client.1.smithi192.stdout:7/23: mkdir d2/db 0 2024-04-24T22:45:10.712 INFO:tasks.workunit.client.1.smithi192.stdout:7/24: symlink d2/db/lc 0 2024-04-24T22:45:10.713 INFO:tasks.workunit.client.1.smithi192.stdout:1/28: creat d9/fa x:0 0 0 2024-04-24T22:45:10.713 INFO:tasks.workunit.client.1.smithi192.stdout:7/25: read - d2/f5 zero size 2024-04-24T22:45:10.754 INFO:tasks.workunit.client.1.smithi192.stdout:2/23: rmdir d4 39 2024-04-24T22:45:10.761 INFO:tasks.workunit.client.1.smithi192.stdout:0/31: dwrite f2 [0,4194304] 0 2024-04-24T22:45:10.762 INFO:tasks.workunit.client.1.smithi192.stdout:0/32: stat d3 0 2024-04-24T22:45:10.780 INFO:tasks.workunit.client.1.smithi192.stdout:0/33: fdatasync f2 0 2024-04-24T22:45:10.840 INFO:tasks.workunit.client.1.smithi192.stdout:1/29: symlink d9/lb 0 2024-04-24T22:45:10.840 INFO:tasks.workunit.client.1.smithi192.stdout:1/30: chown l4 12249329 1 2024-04-24T22:45:10.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/31: truncate d9/fa 829860 0 2024-04-24T22:45:10.843 INFO:tasks.workunit.client.1.smithi192.stdout:0/34: rmdir d3 39 2024-04-24T22:45:10.845 INFO:tasks.workunit.client.1.smithi192.stdout:2/24: creat d4/f6 x:0 0 0 2024-04-24T22:45:10.846 INFO:tasks.workunit.client.1.smithi192.stdout:1/32: creat d9/fc x:0 0 0 2024-04-24T22:45:10.847 INFO:tasks.workunit.client.1.smithi192.stdout:0/35: dread f1 [0,4194304] 0 2024-04-24T22:45:10.849 INFO:tasks.workunit.client.1.smithi192.stdout:2/25: rename d4/f6 to d4/f7 0 2024-04-24T22:45:10.851 INFO:tasks.workunit.client.1.smithi192.stdout:2/26: dread - d4/f7 zero size 2024-04-24T22:45:10.854 INFO:tasks.workunit.client.1.smithi192.stdout:1/33: dwrite d9/fc [0,4194304] 0 2024-04-24T22:45:10.855 INFO:tasks.workunit.client.1.smithi192.stdout:0/36: rename f1 to d3/fb 0 2024-04-24T22:45:10.859 INFO:tasks.workunit.client.1.smithi192.stdout:1/34: dread d9/fc [0,4194304] 0 2024-04-24T22:45:10.860 INFO:tasks.workunit.client.1.smithi192.stdout:1/35: truncate d9/fa 1653187 0 2024-04-24T22:45:10.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/27: mknod d4/c8 0 2024-04-24T22:45:10.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/28: chown d4/c8 30990540 1 2024-04-24T22:45:10.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/37: write d3/fb [2306755,64074] 0 2024-04-24T22:45:10.868 INFO:tasks.workunit.client.1.smithi192.stdout:2/29: mkdir d4/d9 0 2024-04-24T22:45:10.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/38: creat d3/fc x:0 0 0 2024-04-24T22:45:10.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/39: stat d3/fc 0 2024-04-24T22:45:10.871 INFO:tasks.workunit.client.1.smithi192.stdout:1/36: fdatasync d9/fc 0 2024-04-24T22:45:10.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/30: rename c1 to d4/ca 0 2024-04-24T22:45:10.878 INFO:tasks.workunit.client.1.smithi192.stdout:0/40: dwrite d3/f8 [0,4194304] 0 2024-04-24T22:45:10.879 INFO:tasks.workunit.client.1.smithi192.stdout:0/41: chown d3/l4 1875221 1 2024-04-24T22:45:10.880 INFO:tasks.workunit.client.1.smithi192.stdout:0/42: stat d3/c5 0 2024-04-24T22:45:10.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/43: stat d3 0 2024-04-24T22:45:10.890 INFO:tasks.workunit.client.1.smithi192.stdout:0/44: dwrite d3/fc [0,4194304] 0 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr fail", "who": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:45:10.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: Activating manager daemon smithi192.ctmwwc 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "mgr fail", "who": "smithi063.upcnvd"}]': finished 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: mgrmap e21: smithi192.ctmwwc(active, starting, since 0.00903463s) 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:45:10.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:45:10.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:45:10.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:45:10.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: Manager daemon smithi192.ctmwwc is now available 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.ctmwwc/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.ctmwwc/trash_purge_schedule"}]: dispatch 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:10.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:10.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: Deploying cephadm binary to smithi063 2024-04-24T22:45:10.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:10 smithi192 ceph-mon[33991]: Deploying cephadm binary to smithi192 2024-04-24T22:45:10.942 INFO:tasks.workunit.client.1.smithi192.stdout:0/45: unlink d3/fb 0 2024-04-24T22:45:10.943 INFO:tasks.workunit.client.1.smithi192.stdout:0/46: chown d3/l4 4 1 2024-04-24T22:45:10.943 INFO:tasks.workunit.client.1.smithi192.stdout:0/47: write f2 [3355142,7510] 0 2024-04-24T22:45:10.950 INFO:tasks.workunit.client.1.smithi192.stdout:0/48: dwrite d3/fc [0,4194304] 0 2024-04-24T22:45:10.952 INFO:tasks.workunit.client.1.smithi192.stdout:2/31: rmdir d4 39 2024-04-24T22:45:10.952 INFO:tasks.workunit.client.1.smithi192.stdout:2/32: readlink - no filename 2024-04-24T22:45:10.962 INFO:tasks.workunit.client.1.smithi192.stdout:2/33: creat d4/d9/fb x:0 0 0 2024-04-24T22:45:10.970 INFO:tasks.workunit.client.1.smithi192.stdout:2/34: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:10.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/35: dread - d4/f5 zero size 2024-04-24T22:45:10.973 INFO:tasks.workunit.client.1.smithi192.stdout:2/36: rmdir d4 39 2024-04-24T22:45:10.977 INFO:tasks.workunit.client.1.smithi192.stdout:2/37: creat d4/d9/fc x:0 0 0 2024-04-24T22:45:10.978 INFO:tasks.workunit.client.1.smithi192.stdout:0/49: truncate d3/fc 2657736 0 2024-04-24T22:45:10.978 INFO:tasks.workunit.client.1.smithi192.stdout:2/38: truncate d4/d9/fc 541798 0 2024-04-24T22:45:10.983 INFO:tasks.workunit.client.1.smithi192.stdout:0/50: dread d3/f8 [0,4194304] 0 2024-04-24T22:45:10.983 INFO:tasks.workunit.client.1.smithi192.stdout:0/51: fsync f2 0 2024-04-24T22:45:10.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/39: link c3 d4/d9/cd 0 2024-04-24T22:45:10.990 INFO:tasks.workunit.client.1.smithi192.stdout:2/40: chown d4/c8 23459 1 2024-04-24T22:45:10.990 INFO:tasks.workunit.client.1.smithi192.stdout:2/41: chown d4/d9/fb 64111 1 2024-04-24T22:45:10.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/52: rmdir d3 39 2024-04-24T22:45:10.995 INFO:tasks.workunit.client.1.smithi192.stdout:0/53: symlink d3/ld 0 2024-04-24T22:45:11.001 INFO:tasks.workunit.client.1.smithi192.stdout:0/54: rename d3/l9 to d3/le 0 2024-04-24T22:45:11.002 INFO:tasks.workunit.client.1.smithi192.stdout:2/42: getdents d4/d9 0 2024-04-24T22:45:11.004 INFO:tasks.workunit.client.1.smithi192.stdout:0/55: dread d3/fc [0,4194304] 0 2024-04-24T22:45:11.010 INFO:tasks.workunit.client.1.smithi192.stdout:2/43: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:11.013 INFO:tasks.workunit.client.1.smithi192.stdout:2/44: truncate d4/f5 396954 0 2024-04-24T22:45:11.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/56: dwrite d3/fc [0,4194304] 0 2024-04-24T22:45:11.014 INFO:tasks.workunit.client.1.smithi192.stdout:2/45: fdatasync f0 0 2024-04-24T22:45:11.023 INFO:tasks.workunit.client.1.smithi192.stdout:0/57: link d3/c5 d3/cf 0 2024-04-24T22:45:11.026 INFO:tasks.workunit.client.1.smithi192.stdout:0/58: creat d3/f10 x:0 0 0 2024-04-24T22:45:11.029 INFO:tasks.workunit.client.1.smithi192.stdout:2/46: dwrite d4/f7 [0,4194304] 0 2024-04-24T22:45:11.031 INFO:tasks.workunit.client.1.smithi192.stdout:2/47: write d4/f5 [328620,41046] 0 2024-04-24T22:45:11.034 INFO:tasks.workunit.client.1.smithi192.stdout:0/59: dread d3/fc [0,4194304] 0 2024-04-24T22:45:11.035 INFO:tasks.workunit.client.1.smithi192.stdout:2/48: creat d4/fe x:0 0 0 2024-04-24T22:45:11.038 INFO:tasks.workunit.client.1.smithi192.stdout:2/49: dread - d4/fe zero size 2024-04-24T22:45:11.039 INFO:tasks.workunit.client.1.smithi192.stdout:2/50: creat d4/d9/ff x:0 0 0 2024-04-24T22:45:11.039 INFO:tasks.workunit.client.1.smithi192.stdout:0/60: dread f2 [0,4194304] 0 2024-04-24T22:45:11.040 INFO:tasks.workunit.client.1.smithi192.stdout:0/61: write d3/f10 [231715,51816] 0 2024-04-24T22:45:11.042 INFO:tasks.workunit.client.1.smithi192.stdout:2/51: mkdir d4/d9/d10 0 2024-04-24T22:45:11.042 INFO:tasks.workunit.client.1.smithi192.stdout:2/52: readlink - no filename 2024-04-24T22:45:11.046 INFO:tasks.workunit.client.1.smithi192.stdout:2/53: unlink d4/d9/fc 0 2024-04-24T22:45:11.050 INFO:tasks.workunit.client.1.smithi192.stdout:0/62: unlink d3/la 0 2024-04-24T22:45:11.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:11.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:11.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr fail", "who": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: Activating manager daemon smithi192.ctmwwc 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.14216 172.21.15.63:0/3333975089' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "mgr fail", "who": "smithi063.upcnvd"}]': finished 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: mgrmap e21: smithi192.ctmwwc(active, starting, since 0.00903463s) 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:45:11.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: Manager daemon smithi192.ctmwwc is now available 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:11.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.ctmwwc/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:45:11.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.ctmwwc/trash_purge_schedule"}]: dispatch 2024-04-24T22:45:11.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:11.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:11.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: Deploying cephadm binary to smithi063 2024-04-24T22:45:11.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:10 smithi063 ceph-mon[26912]: Deploying cephadm binary to smithi192 2024-04-24T22:45:11.057 INFO:tasks.workunit.client.1.smithi192.stdout:2/54: rename d4/d9/ff to d4/d9/d10/f11 0 2024-04-24T22:45:11.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/63: dwrite d3/f10 [0,4194304] 0 2024-04-24T22:45:11.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/55: dread f0 [0,4194304] 0 2024-04-24T22:45:11.068 INFO:tasks.workunit.client.1.smithi192.stdout:2/56: creat d4/d9/f12 x:0 0 0 2024-04-24T22:45:11.069 INFO:tasks.workunit.client.1.smithi192.stdout:2/57: chown d4/c8 29706 1 2024-04-24T22:45:11.070 INFO:tasks.workunit.client.1.smithi192.stdout:0/64: dwrite d3/f10 [0,4194304] 0 2024-04-24T22:45:11.073 INFO:tasks.workunit.client.1.smithi192.stdout:2/58: read d4/f5 [303194,101744] 0 2024-04-24T22:45:11.074 INFO:tasks.workunit.client.1.smithi192.stdout:0/65: write d3/f8 [43667,28758] 0 2024-04-24T22:45:11.080 INFO:tasks.workunit.client.1.smithi192.stdout:2/59: rename d4/f7 to d4/f13 0 2024-04-24T22:45:11.083 INFO:tasks.workunit.client.1.smithi192.stdout:2/60: write d4/d9/f12 [641868,18081] 0 2024-04-24T22:45:11.087 INFO:tasks.workunit.client.1.smithi192.stdout:0/66: dwrite d3/f10 [0,4194304] 0 2024-04-24T22:45:11.088 INFO:tasks.workunit.client.1.smithi192.stdout:0/67: chown d3/l4 179 1 2024-04-24T22:45:11.090 INFO:tasks.workunit.client.1.smithi192.stdout:2/61: mknod d4/d9/d10/c14 0 2024-04-24T22:45:11.091 INFO:tasks.workunit.client.1.smithi192.stdout:0/68: creat d3/f11 x:0 0 0 2024-04-24T22:45:11.096 INFO:tasks.workunit.client.1.smithi192.stdout:2/62: mknod d4/d9/d10/c15 0 2024-04-24T22:45:11.100 INFO:tasks.workunit.client.1.smithi192.stdout:0/69: dwrite f2 [0,4194304] 0 2024-04-24T22:45:11.108 INFO:tasks.workunit.client.1.smithi192.stdout:2/63: unlink c3 0 2024-04-24T22:45:11.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/70: creat d3/f12 x:0 0 0 2024-04-24T22:45:11.116 INFO:tasks.workunit.client.1.smithi192.stdout:0/71: symlink d3/l13 0 2024-04-24T22:45:11.119 INFO:tasks.workunit.client.1.smithi192.stdout:0/72: creat d3/f14 x:0 0 0 2024-04-24T22:45:11.124 INFO:tasks.workunit.client.1.smithi192.stdout:2/64: getdents d4/d9 0 2024-04-24T22:45:11.127 INFO:tasks.workunit.client.1.smithi192.stdout:2/65: rename d4/d9/f12 to d4/d9/d10/f16 0 2024-04-24T22:45:11.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/66: dread f2 [0,4194304] 0 2024-04-24T22:45:11.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/67: dread d4/f5 [0,4194304] 0 2024-04-24T22:45:11.140 INFO:tasks.workunit.client.1.smithi192.stdout:2/68: truncate f0 51353 0 2024-04-24T22:45:11.144 INFO:tasks.workunit.client.1.smithi192.stdout:2/69: dread d4/f13 [0,4194304] 0 2024-04-24T22:45:11.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/73: rmdir d3 39 2024-04-24T22:45:11.149 INFO:tasks.workunit.client.1.smithi192.stdout:2/70: fdatasync d4/d9/d10/f16 0 2024-04-24T22:45:11.153 INFO:tasks.workunit.client.1.smithi192.stdout:2/71: write d4/f5 [997582,58440] 0 2024-04-24T22:45:11.157 INFO:tasks.workunit.client.1.smithi192.stdout:0/74: mkdir d3/d15 0 2024-04-24T22:45:11.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/72: unlink f2 0 2024-04-24T22:45:11.161 INFO:tasks.workunit.client.1.smithi192.stdout:0/75: mkdir d3/d16 0 2024-04-24T22:45:11.161 INFO:tasks.workunit.client.1.smithi192.stdout:0/76: chown d3/l13 1539 1 2024-04-24T22:45:11.162 INFO:tasks.workunit.client.1.smithi192.stdout:0/77: fdatasync d3/f10 0 2024-04-24T22:45:11.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/73: write d4/f13 [331948,126928] 0 2024-04-24T22:45:11.164 INFO:tasks.workunit.client.1.smithi192.stdout:2/74: read d4/f13 [1451603,65235] 0 2024-04-24T22:45:11.167 INFO:tasks.workunit.client.1.smithi192.stdout:0/78: dread f2 [0,4194304] 0 2024-04-24T22:45:11.170 INFO:tasks.workunit.client.1.smithi192.stdout:0/79: dread f2 [0,4194304] 0 2024-04-24T22:45:11.171 INFO:tasks.workunit.client.1.smithi192.stdout:2/75: mknod d4/d9/d10/c17 0 2024-04-24T22:45:11.176 INFO:tasks.workunit.client.1.smithi192.stdout:2/76: creat d4/d9/f18 x:0 0 0 2024-04-24T22:45:11.179 INFO:tasks.workunit.client.1.smithi192.stdout:2/77: creat d4/f19 x:0 0 0 2024-04-24T22:45:11.179 INFO:tasks.workunit.client.1.smithi192.stdout:2/78: write d4/d9/fb [4332669,41750] 0 2024-04-24T22:45:11.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/79: link d4/ca d4/d9/d10/c1a 0 2024-04-24T22:45:11.234 INFO:tasks.workunit.client.1.smithi192.stdout:2/80: getdents d4 0 2024-04-24T22:45:11.237 INFO:tasks.workunit.client.1.smithi192.stdout:2/81: dread d4/f5 [0,4194304] 0 2024-04-24T22:45:11.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/82: write d4/d9/d10/f11 [556678,86976] 0 2024-04-24T22:45:11.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/83: chown d4/d9/d10/c14 3388 1 2024-04-24T22:45:11.245 INFO:tasks.workunit.client.1.smithi192.stdout:2/84: mknod d4/c1b 0 2024-04-24T22:45:11.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/85: fsync d4/f5 0 2024-04-24T22:45:11.250 INFO:tasks.workunit.client.1.smithi192.stdout:2/86: chown d4 32638027 1 2024-04-24T22:45:11.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/87: mkdir d4/d9/d10/d1c 0 2024-04-24T22:45:11.259 INFO:tasks.workunit.client.1.smithi192.stdout:2/88: rename d4/d9/d10/f16 to d4/d9/d10/d1c/f1d 0 2024-04-24T22:45:11.264 INFO:tasks.workunit.client.1.smithi192.stdout:2/89: creat d4/d9/d10/f1e x:0 0 0 2024-04-24T22:45:11.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/90: rmdir d4 39 2024-04-24T22:45:11.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/91: mknod d4/d9/c1f 0 2024-04-24T22:45:11.272 INFO:tasks.workunit.client.1.smithi192.stdout:2/92: fdatasync d4/fe 0 2024-04-24T22:45:11.274 INFO:tasks.workunit.client.1.smithi192.stdout:2/93: dread d4/f5 [0,4194304] 0 2024-04-24T22:45:11.275 INFO:tasks.workunit.client.1.smithi192.stdout:2/94: truncate d4/f19 37267 0 2024-04-24T22:45:11.314 INFO:tasks.workunit.client.1.smithi192.stdout:2/95: rmdir d4/d9/d10 39 2024-04-24T22:45:11.319 INFO:tasks.workunit.client.1.smithi192.stdout:2/96: truncate d4/f5 698187 0 2024-04-24T22:45:11.322 INFO:tasks.workunit.client.1.smithi192.stdout:2/97: chown d4/d9/cd 16 1 2024-04-24T22:45:11.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/98: creat d4/d9/d10/d1c/f20 x:0 0 0 2024-04-24T22:45:11.328 INFO:tasks.workunit.client.1.smithi192.stdout:2/99: truncate d4/f19 948523 0 2024-04-24T22:45:11.331 INFO:tasks.workunit.client.1.smithi192.stdout:2/100: creat d4/d9/d10/d1c/f21 x:0 0 0 2024-04-24T22:45:11.341 INFO:tasks.workunit.client.1.smithi192.stdout:2/101: link d4/d9/d10/c15 d4/d9/d10/c22 0 2024-04-24T22:45:12.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:11 smithi063 ceph-mon[26912]: mgrmap e22: smithi192.ctmwwc(active, since 1.00796s) 2024-04-24T22:45:12.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:11 smithi063 ceph-mon[26912]: pgmap v3: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:11 smithi192 ceph-mon[33991]: mgrmap e22: smithi192.ctmwwc(active, since 1.00796s) 2024-04-24T22:45:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:11 smithi192 ceph-mon[33991]: pgmap v3: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:12.991 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T22:45:12.995 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T22:45:12.995 INFO:tasks.workunit.client.0.smithi063.stderr:+ make 2024-04-24T22:45:13.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: [24/Apr/2024:22:45:11] ENGINE Bus STARTING 2024-04-24T22:45:13.112 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: mgrmap e23: smithi192.ctmwwc(active, since 2s) 2024-04-24T22:45:13.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: pgmap v4: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:13.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: [24/Apr/2024:22:45:11] ENGINE Serving on http://172.21.15.192:8765 2024-04-24T22:45:13.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: [24/Apr/2024:22:45:12] ENGINE Serving on https://172.21.15.192:7150 2024-04-24T22:45:13.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:12 smithi063 ceph-mon[26912]: [24/Apr/2024:22:45:12] ENGINE Bus STARTED 2024-04-24T22:45:13.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: [24/Apr/2024:22:45:11] ENGINE Bus STARTING 2024-04-24T22:45:13.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: mgrmap e23: smithi192.ctmwwc(active, since 2s) 2024-04-24T22:45:13.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: pgmap v4: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:13.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: [24/Apr/2024:22:45:11] ENGINE Serving on http://172.21.15.192:8765 2024-04-24T22:45:13.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: [24/Apr/2024:22:45:12] ENGINE Serving on https://172.21.15.192:7150 2024-04-24T22:45:13.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:12 smithi192 ceph-mon[33991]: [24/Apr/2024:22:45:12] ENGINE Bus STARTED 2024-04-24T22:45:13.364 INFO:tasks.workunit.client.0.smithi063.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T22:45:13.728 INFO:tasks.workunit.client.0.smithi063.stderr:++ readlink -f fsstress 2024-04-24T22:45:13.730 INFO:tasks.workunit.client.0.smithi063.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T22:45:13.730 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-04-24T22:45:13.731 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T22:45:13.731 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-04-24T22:45:13.731 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-24T22:45:13.731 INFO:tasks.workunit.client.0.smithi063.stderr:++ mktemp -d -p . 2024-04-24T22:45:13.734 INFO:tasks.workunit.client.0.smithi063.stderr:+ T=./tmp.4gY25p0yZa 2024-04-24T22:45:13.734 INFO:tasks.workunit.client.0.smithi063.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.4gY25p0yZa -l 1 -n 1000 -p 10 -v 2024-04-24T22:45:13.736 INFO:tasks.workunit.client.0.smithi063.stdout:seed = 1713549432 2024-04-24T22:45:13.739 INFO:tasks.workunit.client.0.smithi063.stdout:0/0: rename - no filename 2024-04-24T22:45:13.739 INFO:tasks.workunit.client.0.smithi063.stdout:0/1: rmdir - no directory 2024-04-24T22:45:13.740 INFO:tasks.workunit.client.0.smithi063.stdout:1/0: dwrite - no filename 2024-04-24T22:45:13.740 INFO:tasks.workunit.client.0.smithi063.stdout:1/1: rename - no filename 2024-04-24T22:45:13.740 INFO:tasks.workunit.client.0.smithi063.stdout:1/2: rmdir - no directory 2024-04-24T22:45:13.741 INFO:tasks.workunit.client.0.smithi063.stdout:1/3: unlink - no file 2024-04-24T22:45:13.741 INFO:tasks.workunit.client.0.smithi063.stdout:1/4: truncate - no filename 2024-04-24T22:45:13.741 INFO:tasks.workunit.client.0.smithi063.stdout:0/2: symlink l0 0 2024-04-24T22:45:13.742 INFO:tasks.workunit.client.0.smithi063.stdout:2/0: dread - no filename 2024-04-24T22:45:13.742 INFO:tasks.workunit.client.0.smithi063.stdout:2/1: rename - no filename 2024-04-24T22:45:13.742 INFO:tasks.workunit.client.0.smithi063.stdout:2/2: dread - no filename 2024-04-24T22:45:13.742 INFO:tasks.workunit.client.0.smithi063.stdout:2/3: rename - no filename 2024-04-24T22:45:13.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/3: creat f1 x:0 0 0 2024-04-24T22:45:13.743 INFO:tasks.workunit.client.0.smithi063.stdout:1/5: creat f0 x:0 0 0 2024-04-24T22:45:13.744 INFO:tasks.workunit.client.0.smithi063.stdout:0/4: mknod c2 0 2024-04-24T22:45:13.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/0: symlink l0 0 2024-04-24T22:45:13.747 INFO:tasks.workunit.client.0.smithi063.stdout:3/1: dwrite - no filename 2024-04-24T22:45:13.747 INFO:tasks.workunit.client.0.smithi063.stdout:4/0: readlink - no filename 2024-04-24T22:45:13.747 INFO:tasks.workunit.client.0.smithi063.stdout:4/1: chown . 0 1 2024-04-24T22:45:13.750 INFO:tasks.workunit.client.0.smithi063.stdout:3/2: symlink l1 0 2024-04-24T22:45:13.751 INFO:tasks.workunit.client.0.smithi063.stdout:3/3: readlink l1 0 2024-04-24T22:45:13.751 INFO:tasks.workunit.client.0.smithi063.stdout:3/4: dread - no filename 2024-04-24T22:45:13.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/6: dwrite f0 [0,4194304] 0 2024-04-24T22:45:13.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/5: dwrite f1 [0,4194304] 0 2024-04-24T22:45:13.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/5: unlink l0 0 2024-04-24T22:45:13.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/6: dread - no filename 2024-04-24T22:45:13.762 INFO:tasks.workunit.client.0.smithi063.stdout:4/2: creat f0 x:0 0 0 2024-04-24T22:45:13.762 INFO:tasks.workunit.client.0.smithi063.stdout:4/3: fdatasync f0 0 2024-04-24T22:45:13.766 INFO:tasks.workunit.client.0.smithi063.stdout:5/0: creat f0 x:0 0 0 2024-04-24T22:45:13.767 INFO:tasks.workunit.client.0.smithi063.stdout:5/1: chown f0 27528 1 2024-04-24T22:45:13.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/7: creat f1 x:0 0 0 2024-04-24T22:45:13.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/6: rename l0 to l3 0 2024-04-24T22:45:13.772 INFO:tasks.workunit.client.0.smithi063.stdout:3/7: creat f2 x:0 0 0 2024-04-24T22:45:13.773 INFO:tasks.workunit.client.0.smithi063.stdout:3/8: chown f2 354416373 1 2024-04-24T22:45:13.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/4: creat f1 x:0 0 0 2024-04-24T22:45:13.775 INFO:tasks.workunit.client.0.smithi063.stdout:3/9: chown f2 6779805 1 2024-04-24T22:45:13.776 INFO:tasks.workunit.client.0.smithi063.stdout:3/10: readlink l1 0 2024-04-24T22:45:13.777 INFO:tasks.workunit.client.0.smithi063.stdout:8/0: truncate - no filename 2024-04-24T22:45:13.777 INFO:tasks.workunit.client.0.smithi063.stdout:8/1: readlink - no filename 2024-04-24T22:45:13.777 INFO:tasks.workunit.client.0.smithi063.stdout:1/8: dwrite f1 [0,4194304] 0 2024-04-24T22:45:13.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/2: chown . 65 1 2024-04-24T22:45:13.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/3: dread - no filename 2024-04-24T22:45:13.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/4: rename - no filename 2024-04-24T22:45:13.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/5: dread - no filename 2024-04-24T22:45:13.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/6: dread - no filename 2024-04-24T22:45:13.779 INFO:tasks.workunit.client.0.smithi063.stdout:6/0: mknod c0 0 2024-04-24T22:45:13.779 INFO:tasks.workunit.client.0.smithi063.stdout:0/7: dread f1 [0,4194304] 0 2024-04-24T22:45:13.781 INFO:tasks.workunit.client.0.smithi063.stdout:5/2: creat f1 x:0 0 0 2024-04-24T22:45:13.783 INFO:tasks.workunit.client.0.smithi063.stdout:5/3: dread - f1 zero size 2024-04-24T22:45:13.787 INFO:tasks.workunit.client.0.smithi063.stdout:4/5: mkdir d2 0 2024-04-24T22:45:13.788 INFO:tasks.workunit.client.0.smithi063.stdout:1/9: dread f1 [0,4194304] 0 2024-04-24T22:45:13.788 INFO:tasks.workunit.client.0.smithi063.stdout:4/6: dread - f0 zero size 2024-04-24T22:45:13.789 INFO:tasks.workunit.client.0.smithi063.stdout:1/10: stat f0 0 2024-04-24T22:45:13.789 INFO:tasks.workunit.client.0.smithi063.stdout:3/11: dwrite f2 [0,4194304] 0 2024-04-24T22:45:13.789 INFO:tasks.workunit.client.0.smithi063.stdout:4/7: rename d2 to d2/d3 22 2024-04-24T22:45:13.789 INFO:tasks.workunit.client.0.smithi063.stdout:9/0: link - no file 2024-04-24T22:45:13.790 INFO:tasks.workunit.client.0.smithi063.stdout:9/1: chown . 29381867 1 2024-04-24T22:45:13.790 INFO:tasks.workunit.client.0.smithi063.stdout:9/2: unlink - no file 2024-04-24T22:45:13.790 INFO:tasks.workunit.client.0.smithi063.stdout:7/0: mkdir d0 0 2024-04-24T22:45:13.791 INFO:tasks.workunit.client.0.smithi063.stdout:9/3: chown . 2597052 1 2024-04-24T22:45:13.791 INFO:tasks.workunit.client.0.smithi063.stdout:4/8: write f1 [764915,84369] 0 2024-04-24T22:45:13.798 INFO:tasks.workunit.client.0.smithi063.stdout:4/9: dread f1 [0,4194304] 0 2024-04-24T22:45:13.803 INFO:tasks.workunit.client.0.smithi063.stdout:1/11: mkdir d2 0 2024-04-24T22:45:13.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/7: mknod c0 0 2024-04-24T22:45:13.813 INFO:tasks.workunit.client.0.smithi063.stdout:8/8: chown c0 6028 1 2024-04-24T22:45:13.813 INFO:tasks.workunit.client.0.smithi063.stdout:8/9: truncate - no filename 2024-04-24T22:45:13.823 INFO:tasks.workunit.client.0.smithi063.stdout:1/12: dread f1 [0,4194304] 0 2024-04-24T22:45:13.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/4: symlink l0 0 2024-04-24T22:45:13.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/5: truncate - no filename 2024-04-24T22:45:13.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/6: dwrite - no filename 2024-04-24T22:45:13.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/7: write - no filename 2024-04-24T22:45:13.829 INFO:tasks.workunit.client.0.smithi063.stdout:7/1: creat d0/f1 x:0 0 0 2024-04-24T22:45:13.830 INFO:tasks.workunit.client.0.smithi063.stdout:7/2: chown d0 270015416 1 2024-04-24T22:45:13.831 INFO:tasks.workunit.client.0.smithi063.stdout:8/10: mkdir d1 0 2024-04-24T22:45:13.831 INFO:tasks.workunit.client.0.smithi063.stdout:8/11: dread - no filename 2024-04-24T22:45:13.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/10: mknod d2/c4 0 2024-04-24T22:45:13.843 INFO:tasks.workunit.client.0.smithi063.stdout:9/8: unlink l0 0 2024-04-24T22:45:13.844 INFO:tasks.workunit.client.0.smithi063.stdout:9/9: rename - no filename 2024-04-24T22:45:13.844 INFO:tasks.workunit.client.0.smithi063.stdout:9/10: dread - no filename 2024-04-24T22:45:13.844 INFO:tasks.workunit.client.0.smithi063.stdout:9/11: chown . 0 1 2024-04-24T22:45:13.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/3: symlink d0/l2 0 2024-04-24T22:45:13.856 INFO:tasks.workunit.client.0.smithi063.stdout:9/12: mknod c1 0 2024-04-24T22:45:13.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/4: rename d0/f1 to d0/f3 0 2024-04-24T22:45:13.862 INFO:tasks.workunit.client.0.smithi063.stdout:1/13: rmdir d2 0 2024-04-24T22:45:13.864 INFO:tasks.workunit.client.0.smithi063.stdout:9/13: rename c1 to c2 0 2024-04-24T22:45:13.865 INFO:tasks.workunit.client.0.smithi063.stdout:9/14: read - no filename 2024-04-24T22:45:13.867 INFO:tasks.workunit.client.0.smithi063.stdout:7/5: mkdir d0/d4 0 2024-04-24T22:45:13.867 INFO:tasks.workunit.client.0.smithi063.stdout:7/6: rename d0/d4 to d0/d4/d5 22 2024-04-24T22:45:13.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/7: truncate d0/f3 27514 0 2024-04-24T22:45:13.872 INFO:tasks.workunit.client.0.smithi063.stdout:9/15: creat f3 x:0 0 0 2024-04-24T22:45:13.877 INFO:tasks.workunit.client.0.smithi063.stdout:9/16: rename f3 to f4 0 2024-04-24T22:45:13.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/17: dread - f4 zero size 2024-04-24T22:45:13.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/18: chown f4 18 1 2024-04-24T22:45:13.883 INFO:tasks.workunit.client.0.smithi063.stdout:9/19: fdatasync f4 0 2024-04-24T22:45:14.031 INFO:tasks.workunit.client.0.smithi063.stdout:0/8: readlink l3 0 2024-04-24T22:45:14.033 INFO:tasks.workunit.client.0.smithi063.stdout:0/9: creat f4 x:0 0 0 2024-04-24T22:45:14.034 INFO:tasks.workunit.client.0.smithi063.stdout:0/10: write f1 [4971265,21774] 0 2024-04-24T22:45:14.034 INFO:tasks.workunit.client.0.smithi063.stdout:0/11: readlink l3 0 2024-04-24T22:45:14.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/1: getdents . 0 2024-04-24T22:45:14.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/12: creat f5 x:0 0 0 2024-04-24T22:45:14.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/2: mkdir d1 0 2024-04-24T22:45:14.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/3: dread - no filename 2024-04-24T22:45:14.037 INFO:tasks.workunit.client.0.smithi063.stdout:5/4: getdents . 0 2024-04-24T22:45:14.038 INFO:tasks.workunit.client.0.smithi063.stdout:0/13: mkdir d6 0 2024-04-24T22:45:14.038 INFO:tasks.workunit.client.0.smithi063.stdout:0/14: fdatasync f4 0 2024-04-24T22:45:14.040 INFO:tasks.workunit.client.0.smithi063.stdout:5/5: symlink l2 0 2024-04-24T22:45:14.042 INFO:tasks.workunit.client.0.smithi063.stdout:5/6: mknod c3 0 2024-04-24T22:45:14.042 INFO:tasks.workunit.client.0.smithi063.stdout:5/7: truncate f1 417432 0 2024-04-24T22:45:14.043 INFO:tasks.workunit.client.0.smithi063.stdout:6/4: rmdir d1 0 2024-04-24T22:45:14.043 INFO:tasks.workunit.client.0.smithi063.stdout:5/8: write f0 [657934,19032] 0 2024-04-24T22:45:14.044 INFO:tasks.workunit.client.0.smithi063.stdout:0/15: creat d6/f7 x:0 0 0 2024-04-24T22:45:14.045 INFO:tasks.workunit.client.0.smithi063.stdout:6/5: mknod c2 0 2024-04-24T22:45:14.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/16: write d6/f7 [866317,47407] 0 2024-04-24T22:45:14.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/6: chown c2 2136186396 1 2024-04-24T22:45:14.047 INFO:tasks.workunit.client.0.smithi063.stdout:5/9: rename c3 to c4 0 2024-04-24T22:45:14.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/17: symlink d6/l8 0 2024-04-24T22:45:14.048 INFO:tasks.workunit.client.0.smithi063.stdout:5/10: dread f1 [0,4194304] 0 2024-04-24T22:45:14.048 INFO:tasks.workunit.client.0.smithi063.stdout:6/7: creat f3 x:0 0 0 2024-04-24T22:45:14.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/11: dread f1 [0,4194304] 0 2024-04-24T22:45:14.049 INFO:tasks.workunit.client.0.smithi063.stdout:6/8: truncate f3 100008 0 2024-04-24T22:45:14.051 INFO:tasks.workunit.client.0.smithi063.stdout:0/18: unlink f1 0 2024-04-24T22:45:14.051 INFO:tasks.workunit.client.0.smithi063.stdout:0/19: fdatasync d6/f7 0 2024-04-24T22:45:14.052 INFO:tasks.workunit.client.0.smithi063.stdout:5/12: dread f0 [0,4194304] 0 2024-04-24T22:45:14.054 INFO:tasks.workunit.client.0.smithi063.stdout:6/9: mknod c4 0 2024-04-24T22:45:14.057 INFO:tasks.workunit.client.0.smithi063.stdout:5/13: mkdir d5 0 2024-04-24T22:45:14.058 INFO:tasks.workunit.client.0.smithi063.stdout:6/10: symlink l5 0 2024-04-24T22:45:14.059 INFO:tasks.workunit.client.0.smithi063.stdout:6/11: dread f3 [0,4194304] 0 2024-04-24T22:45:14.062 INFO:tasks.workunit.client.0.smithi063.stdout:5/14: creat d5/f6 x:0 0 0 2024-04-24T22:45:14.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/14: truncate f1 4386819 0 2024-04-24T22:45:14.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/12: mknod c6 0 2024-04-24T22:45:14.067 INFO:tasks.workunit.client.0.smithi063.stdout:5/15: dread f0 [0,4194304] 0 2024-04-24T22:45:14.067 INFO:tasks.workunit.client.0.smithi063.stdout:1/15: mkdir d3 0 2024-04-24T22:45:14.068 INFO:tasks.workunit.client.0.smithi063.stdout:6/13: creat f7 x:0 0 0 2024-04-24T22:45:14.069 INFO:tasks.workunit.client.0.smithi063.stdout:3/12: truncate f2 1395077 0 2024-04-24T22:45:14.069 INFO:tasks.workunit.client.0.smithi063.stdout:6/14: stat c4 0 2024-04-24T22:45:14.071 INFO:tasks.workunit.client.0.smithi063.stdout:1/16: unlink f0 0 2024-04-24T22:45:14.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/15: creat f8 x:0 0 0 2024-04-24T22:45:14.076 INFO:tasks.workunit.client.0.smithi063.stdout:6/16: symlink l9 0 2024-04-24T22:45:14.077 INFO:tasks.workunit.client.0.smithi063.stdout:1/17: mknod d3/c4 0 2024-04-24T22:45:14.078 INFO:tasks.workunit.client.0.smithi063.stdout:4/11: getdents d2 0 2024-04-24T22:45:14.078 INFO:tasks.workunit.client.0.smithi063.stdout:1/18: rmdir d3 39 2024-04-24T22:45:14.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/12: write f0 [885523,27631] 0 2024-04-24T22:45:14.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/13: fdatasync f0 0 2024-04-24T22:45:14.090 INFO:tasks.workunit.client.0.smithi063.stdout:7/8: fsync d0/f3 0 2024-04-24T22:45:14.092 INFO:tasks.workunit.client.0.smithi063.stdout:7/9: creat d0/d4/f6 x:0 0 0 2024-04-24T22:45:14.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/10: write d0/d4/f6 [759236,42318] 0 2024-04-24T22:45:14.094 INFO:tasks.workunit.client.0.smithi063.stdout:7/11: write d0/f3 [853422,33258] 0 2024-04-24T22:45:14.096 INFO:tasks.workunit.client.0.smithi063.stdout:7/12: fsync d0/f3 0 2024-04-24T22:45:14.097 INFO:tasks.workunit.client.0.smithi063.stdout:7/13: chown d0 242568 1 2024-04-24T22:45:14.178 INFO:tasks.workunit.client.0.smithi063.stdout:5/16: chown c4 10 1 2024-04-24T22:45:14.179 INFO:tasks.workunit.client.0.smithi063.stdout:0/20: getdents d6 0 2024-04-24T22:45:14.179 INFO:tasks.workunit.client.0.smithi063.stdout:0/21: dread - f4 zero size 2024-04-24T22:45:14.197 INFO:tasks.workunit.client.0.smithi063.stdout:3/13: write f2 [1322970,63947] 0 2024-04-24T22:45:14.203 INFO:tasks.workunit.client.0.smithi063.stdout:4/14: write f1 [292824,56215] 0 2024-04-24T22:45:14.205 INFO:tasks.workunit.client.0.smithi063.stdout:7/14: rmdir d0 39 2024-04-24T22:45:14.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/17: mknod d5/c7 0 2024-04-24T22:45:14.293 INFO:tasks.workunit.client.0.smithi063.stdout:0/22: symlink d6/l9 0 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:0/23: chown c2 1005173563 1 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:2/4: sync 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:8/12: sync 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:2/5: fsync - no filename 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/20: sync 2024-04-24T22:45:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:6/17: sync 2024-04-24T22:45:14.295 INFO:tasks.workunit.client.0.smithi063.stdout:1/19: mknod d3/c5 0 2024-04-24T22:45:14.295 INFO:tasks.workunit.client.0.smithi063.stdout:1/20: readlink - no filename 2024-04-24T22:45:14.295 INFO:tasks.workunit.client.0.smithi063.stdout:0/24: chown d6/l9 10214 1 2024-04-24T22:45:14.295 INFO:tasks.workunit.client.0.smithi063.stdout:0/25: readlink l3 0 2024-04-24T22:45:14.296 INFO:tasks.workunit.client.0.smithi063.stdout:3/14: rename f2 to f3 0 2024-04-24T22:45:14.299 INFO:tasks.workunit.client.0.smithi063.stdout:5/18: truncate f0 29817 0 2024-04-24T22:45:14.299 INFO:tasks.workunit.client.0.smithi063.stdout:5/19: chown d5/f6 0 1 2024-04-24T22:45:14.299 INFO:tasks.workunit.client.0.smithi063.stdout:5/20: fdatasync d5/f6 0 2024-04-24T22:45:14.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/21: fdatasync d5/f6 0 2024-04-24T22:45:14.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/22: fdatasync d5/f6 0 2024-04-24T22:45:14.301 INFO:tasks.workunit.client.0.smithi063.stdout:5/23: chown l2 93344 1 2024-04-24T22:45:14.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/18: dread f3 [0,4194304] 0 2024-04-24T22:45:14.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/19: rmdir - no directory 2024-04-24T22:45:14.302 INFO:tasks.workunit.client.0.smithi063.stdout:6/20: truncate f8 208004 0 2024-04-24T22:45:14.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/21: mkdir d3/d6 0 2024-04-24T22:45:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/15: mknod c4 0 2024-04-24T22:45:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/16: rmdir - no directory 2024-04-24T22:45:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/17: chown l1 37 1 2024-04-24T22:45:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:4/15: link f1 d2/f5 0 2024-04-24T22:45:14.304 INFO:tasks.workunit.client.0.smithi063.stdout:7/15: creat d0/f7 x:0 0 0 2024-04-24T22:45:14.305 INFO:tasks.workunit.client.0.smithi063.stdout:7/16: chown d0/l2 6207 1 2024-04-24T22:45:14.305 INFO:tasks.workunit.client.0.smithi063.stdout:7/17: dread - d0/f7 zero size 2024-04-24T22:45:14.305 INFO:tasks.workunit.client.0.smithi063.stdout:9/21: rename c2 to c5 0 2024-04-24T22:45:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/6: mkdir d0 0 2024-04-24T22:45:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/7: dread - no filename 2024-04-24T22:45:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/8: readlink - no filename 2024-04-24T22:45:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/22: write f4 [28036,77650] 0 2024-04-24T22:45:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:8/13: mkdir d1/d2 0 2024-04-24T22:45:14.307 INFO:tasks.workunit.client.0.smithi063.stdout:5/24: symlink d5/l8 0 2024-04-24T22:45:14.307 INFO:tasks.workunit.client.0.smithi063.stdout:6/21: creat fa x:0 0 0 2024-04-24T22:45:14.309 INFO:tasks.workunit.client.0.smithi063.stdout:1/22: creat d3/f7 x:0 0 0 2024-04-24T22:45:14.309 INFO:tasks.workunit.client.0.smithi063.stdout:1/23: readlink - no filename 2024-04-24T22:45:14.309 INFO:tasks.workunit.client.0.smithi063.stdout:3/18: write f3 [1337907,63830] 0 2024-04-24T22:45:14.310 INFO:tasks.workunit.client.0.smithi063.stdout:4/16: write d2/f5 [1004956,32731] 0 2024-04-24T22:45:14.311 INFO:tasks.workunit.client.0.smithi063.stdout:7/18: creat d0/d4/f8 x:0 0 0 2024-04-24T22:45:14.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/23: mkdir d6 0 2024-04-24T22:45:14.315 INFO:tasks.workunit.client.0.smithi063.stdout:6/22: mkdir db 0 2024-04-24T22:45:14.316 INFO:tasks.workunit.client.0.smithi063.stdout:3/19: symlink l5 0 2024-04-24T22:45:14.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/9: symlink d0/l1 0 2024-04-24T22:45:14.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/10: dwrite - no filename 2024-04-24T22:45:14.318 INFO:tasks.workunit.client.0.smithi063.stdout:7/19: mknod d0/d4/c9 0 2024-04-24T22:45:14.319 INFO:tasks.workunit.client.0.smithi063.stdout:1/24: dread f1 [0,4194304] 0 2024-04-24T22:45:14.319 INFO:tasks.workunit.client.0.smithi063.stdout:8/14: mkdir d1/d2/d3 0 2024-04-24T22:45:14.321 INFO:tasks.workunit.client.0.smithi063.stdout:6/23: rename c0 to db/cc 0 2024-04-24T22:45:14.322 INFO:tasks.workunit.client.0.smithi063.stdout:6/24: dread f8 [0,4194304] 0 2024-04-24T22:45:14.322 INFO:tasks.workunit.client.0.smithi063.stdout:3/20: mknod c6 0 2024-04-24T22:45:14.323 INFO:tasks.workunit.client.0.smithi063.stdout:3/21: chown f3 10048 1 2024-04-24T22:45:14.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/11: mkdir d0/d2 0 2024-04-24T22:45:14.324 INFO:tasks.workunit.client.0.smithi063.stdout:1/25: symlink d3/l8 0 2024-04-24T22:45:14.325 INFO:tasks.workunit.client.0.smithi063.stdout:9/24: creat d6/f7 x:0 0 0 2024-04-24T22:45:14.328 INFO:tasks.workunit.client.0.smithi063.stdout:7/20: rename d0/d4/f6 to d0/fa 0 2024-04-24T22:45:14.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/25: mknod db/cd 0 2024-04-24T22:45:14.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/26: stat c6 0 2024-04-24T22:45:14.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/27: stat c6 0 2024-04-24T22:45:14.330 INFO:tasks.workunit.client.0.smithi063.stdout:6/28: truncate fa 355076 0 2024-04-24T22:45:14.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/26: fdatasync f1 0 2024-04-24T22:45:14.334 INFO:tasks.workunit.client.0.smithi063.stdout:8/15: symlink d1/d2/d3/l4 0 2024-04-24T22:45:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/16: dread - no filename 2024-04-24T22:45:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/17: write - no filename 2024-04-24T22:45:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/18: chown d1 690807 1 2024-04-24T22:45:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/19: fsync - no filename 2024-04-24T22:45:14.338 INFO:tasks.workunit.client.0.smithi063.stdout:7/21: creat d0/d4/fb x:0 0 0 2024-04-24T22:45:14.340 INFO:tasks.workunit.client.0.smithi063.stdout:6/29: creat db/fe x:0 0 0 2024-04-24T22:45:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/12: mkdir d0/d2/d3 0 2024-04-24T22:45:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/13: chown d0/d2/d3 53631558 1 2024-04-24T22:45:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/14: dread - no filename 2024-04-24T22:45:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/15: write - no filename 2024-04-24T22:45:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/16: stat d0/l1 0 2024-04-24T22:45:14.343 INFO:tasks.workunit.client.0.smithi063.stdout:2/17: stat d0 0 2024-04-24T22:45:14.346 INFO:tasks.workunit.client.0.smithi063.stdout:7/22: rename d0/d4/c9 to d0/cc 0 2024-04-24T22:45:14.347 INFO:tasks.workunit.client.0.smithi063.stdout:6/30: creat db/ff x:0 0 0 2024-04-24T22:45:14.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/18: symlink d0/l4 0 2024-04-24T22:45:14.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/19: dread - no filename 2024-04-24T22:45:14.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/20: dread - no filename 2024-04-24T22:45:14.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/21: dwrite - no filename 2024-04-24T22:45:14.348 INFO:tasks.workunit.client.0.smithi063.stdout:7/23: read d0/f3 [674530,130878] 0 2024-04-24T22:45:14.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/31: write f8 [97847,70703] 0 2024-04-24T22:45:14.351 INFO:tasks.workunit.client.0.smithi063.stdout:2/22: symlink d0/l5 0 2024-04-24T22:45:14.351 INFO:tasks.workunit.client.0.smithi063.stdout:2/23: dread - no filename 2024-04-24T22:45:14.353 INFO:tasks.workunit.client.0.smithi063.stdout:2/24: symlink d0/d2/l6 0 2024-04-24T22:45:14.354 INFO:tasks.workunit.client.0.smithi063.stdout:2/25: dwrite - no filename 2024-04-24T22:45:14.354 INFO:tasks.workunit.client.0.smithi063.stdout:2/26: truncate - no filename 2024-04-24T22:45:14.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/27: mkdir d0/d2/d3/d7 0 2024-04-24T22:45:14.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/28: dread - no filename 2024-04-24T22:45:14.360 INFO:tasks.workunit.client.0.smithi063.stdout:6/32: dwrite db/ff [0,4194304] 0 2024-04-24T22:45:14.361 INFO:tasks.workunit.client.0.smithi063.stdout:2/29: creat d0/f8 x:0 0 0 2024-04-24T22:45:14.363 INFO:tasks.workunit.client.0.smithi063.stdout:2/30: fdatasync d0/f8 0 2024-04-24T22:45:14.363 INFO:tasks.workunit.client.0.smithi063.stdout:7/24: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:14.365 INFO:tasks.workunit.client.0.smithi063.stdout:2/31: stat d0/d2/d3/d7 0 2024-04-24T22:45:14.366 INFO:tasks.workunit.client.0.smithi063.stdout:7/25: creat d0/fd x:0 0 0 2024-04-24T22:45:14.368 INFO:tasks.workunit.client.0.smithi063.stdout:2/32: creat d0/d2/d3/d7/f9 x:0 0 0 2024-04-24T22:45:14.370 INFO:tasks.workunit.client.0.smithi063.stdout:7/26: mkdir d0/d4/de 0 2024-04-24T22:45:14.379 INFO:tasks.workunit.client.0.smithi063.stdout:2/33: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:14.379 INFO:tasks.workunit.client.0.smithi063.stdout:3/22: fsync f3 0 2024-04-24T22:45:14.385 INFO:tasks.workunit.client.0.smithi063.stdout:3/23: creat f7 x:0 0 0 2024-04-24T22:45:14.386 INFO:tasks.workunit.client.0.smithi063.stdout:4/17: getdents d2 0 2024-04-24T22:45:14.389 INFO:tasks.workunit.client.0.smithi063.stdout:3/24: dread f3 [0,4194304] 0 2024-04-24T22:45:14.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/34: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:14.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/35: dread - d0/f8 zero size 2024-04-24T22:45:14.391 INFO:tasks.workunit.client.0.smithi063.stdout:2/36: stat d0/l4 0 2024-04-24T22:45:14.395 INFO:tasks.workunit.client.0.smithi063.stdout:4/18: dread f0 [0,4194304] 0 2024-04-24T22:45:14.399 INFO:tasks.workunit.client.0.smithi063.stdout:9/25: getdents d6 0 2024-04-24T22:45:14.399 INFO:tasks.workunit.client.0.smithi063.stdout:4/19: creat d2/f6 x:0 0 0 2024-04-24T22:45:14.401 INFO:tasks.workunit.client.0.smithi063.stdout:2/37: rename d0/f8 to d0/d2/d3/fa 0 2024-04-24T22:45:14.404 INFO:tasks.workunit.client.0.smithi063.stdout:4/20: write d2/f6 [936231,36073] 0 2024-04-24T22:45:14.404 INFO:tasks.workunit.client.0.smithi063.stdout:1/27: write f1 [735100,24544] 0 2024-04-24T22:45:14.405 INFO:tasks.workunit.client.0.smithi063.stdout:5/25: dwrite f1 [0,4194304] 0 2024-04-24T22:45:14.405 INFO:tasks.workunit.client.0.smithi063.stdout:9/26: mkdir d6/d8 0 2024-04-24T22:45:14.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/25: dwrite f3 [0,4194304] 0 2024-04-24T22:45:14.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/21: chown f0 19560560 1 2024-04-24T22:45:14.408 INFO:tasks.workunit.client.0.smithi063.stdout:2/38: mkdir d0/d2/d3/db 0 2024-04-24T22:45:14.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/27: stat c5 0 2024-04-24T22:45:14.411 INFO:tasks.workunit.client.0.smithi063.stdout:9/28: truncate d6/f7 306907 0 2024-04-24T22:45:14.412 INFO:tasks.workunit.client.0.smithi063.stdout:5/26: creat d5/f9 x:0 0 0 2024-04-24T22:45:14.413 INFO:tasks.workunit.client.0.smithi063.stdout:1/28: mkdir d3/d9 0 2024-04-24T22:45:14.414 INFO:tasks.workunit.client.0.smithi063.stdout:3/26: rename l1 to l8 0 2024-04-24T22:45:14.414 INFO:tasks.workunit.client.0.smithi063.stdout:5/27: symlink d5/la 0 2024-04-24T22:45:14.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/29: chown d3/c4 1 1 2024-04-24T22:45:14.417 INFO:tasks.workunit.client.0.smithi063.stdout:6/33: write f3 [1113042,14232] 0 2024-04-24T22:45:14.418 INFO:tasks.workunit.client.0.smithi063.stdout:6/34: chown db/ff 369403198 1 2024-04-24T22:45:14.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/29: mknod d6/d8/c9 0 2024-04-24T22:45:14.419 INFO:tasks.workunit.client.0.smithi063.stdout:7/27: rmdir d0 39 2024-04-24T22:45:14.421 INFO:tasks.workunit.client.0.smithi063.stdout:5/28: mknod d5/cb 0 2024-04-24T22:45:14.423 INFO:tasks.workunit.client.0.smithi063.stdout:1/30: rename d3/d9 to d3/da 0 2024-04-24T22:45:14.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/35: creat db/f10 x:0 0 0 2024-04-24T22:45:14.425 INFO:tasks.workunit.client.0.smithi063.stdout:6/36: read db/ff [3946961,97130] 0 2024-04-24T22:45:14.425 INFO:tasks.workunit.client.0.smithi063.stdout:6/37: fdatasync f8 0 2024-04-24T22:45:14.426 INFO:tasks.workunit.client.0.smithi063.stdout:9/30: symlink d6/la 0 2024-04-24T22:45:14.427 INFO:tasks.workunit.client.0.smithi063.stdout:9/31: truncate d6/f7 1183655 0 2024-04-24T22:45:14.428 INFO:tasks.workunit.client.0.smithi063.stdout:5/29: creat d5/fc x:0 0 0 2024-04-24T22:45:14.430 INFO:tasks.workunit.client.0.smithi063.stdout:2/39: rename d0/d2/d3/fa to d0/d2/d3/d7/fc 0 2024-04-24T22:45:14.434 INFO:tasks.workunit.client.0.smithi063.stdout:6/38: mknod db/c11 0 2024-04-24T22:45:14.435 INFO:tasks.workunit.client.0.smithi063.stdout:7/28: symlink d0/lf 0 2024-04-24T22:45:14.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/32: dwrite f4 [0,4194304] 0 2024-04-24T22:45:14.436 INFO:tasks.workunit.client.0.smithi063.stdout:3/27: readlink l8 0 2024-04-24T22:45:14.438 INFO:tasks.workunit.client.0.smithi063.stdout:5/30: unlink d5/c7 0 2024-04-24T22:45:14.440 INFO:tasks.workunit.client.0.smithi063.stdout:4/22: truncate d2/f5 668029 0 2024-04-24T22:45:14.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/31: creat d3/d6/fb x:0 0 0 2024-04-24T22:45:14.441 INFO:tasks.workunit.client.0.smithi063.stdout:7/29: truncate d0/d4/f8 728460 0 2024-04-24T22:45:14.441 INFO:tasks.workunit.client.0.smithi063.stdout:5/31: dread - d5/f9 zero size 2024-04-24T22:45:14.442 INFO:tasks.workunit.client.0.smithi063.stdout:2/40: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:14.442 INFO:tasks.workunit.client.0.smithi063.stdout:0/26: sync 2024-04-24T22:45:14.443 INFO:tasks.workunit.client.0.smithi063.stdout:4/23: write f0 [1862264,42323] 0 2024-04-24T22:45:14.444 INFO:tasks.workunit.client.0.smithi063.stdout:1/32: dread - d3/d6/fb zero size 2024-04-24T22:45:14.445 INFO:tasks.workunit.client.0.smithi063.stdout:0/27: chown d6/l8 0 1 2024-04-24T22:45:14.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/24: write f0 [858577,1117] 0 2024-04-24T22:45:14.447 INFO:tasks.workunit.client.0.smithi063.stdout:0/28: chown f5 33 1 2024-04-24T22:45:14.449 INFO:tasks.workunit.client.0.smithi063.stdout:9/33: mkdir d6/d8/db 0 2024-04-24T22:45:14.449 INFO:tasks.workunit.client.0.smithi063.stdout:3/28: dread f3 [0,4194304] 0 2024-04-24T22:45:14.454 INFO:tasks.workunit.client.0.smithi063.stdout:2/41: write d0/d2/d3/d7/fc [121044,29154] 0 2024-04-24T22:45:14.455 INFO:tasks.workunit.client.0.smithi063.stdout:2/42: readlink d0/l1 0 2024-04-24T22:45:14.456 INFO:tasks.workunit.client.0.smithi063.stdout:3/29: symlink l9 0 2024-04-24T22:45:14.457 INFO:tasks.workunit.client.0.smithi063.stdout:9/34: symlink d6/d8/lc 0 2024-04-24T22:45:14.458 INFO:tasks.workunit.client.0.smithi063.stdout:7/30: link d0/fa d0/d4/de/f10 0 2024-04-24T22:45:14.459 INFO:tasks.workunit.client.0.smithi063.stdout:2/43: mkdir d0/dd 0 2024-04-24T22:45:14.462 INFO:tasks.workunit.client.0.smithi063.stdout:3/30: mkdir da 0 2024-04-24T22:45:14.462 INFO:tasks.workunit.client.0.smithi063.stdout:2/44: truncate d0/d2/d3/d7/fc 514083 0 2024-04-24T22:45:14.463 INFO:tasks.workunit.client.0.smithi063.stdout:6/39: truncate f8 192530 0 2024-04-24T22:45:14.463 INFO:tasks.workunit.client.0.smithi063.stdout:3/31: dread - f7 zero size 2024-04-24T22:45:14.465 INFO:tasks.workunit.client.0.smithi063.stdout:0/29: getdents d6 0 2024-04-24T22:45:14.465 INFO:tasks.workunit.client.0.smithi063.stdout:2/45: chown d0/d2/d3/d7/f9 238046581 1 2024-04-24T22:45:14.467 INFO:tasks.workunit.client.0.smithi063.stdout:7/31: dwrite d0/fd [0,4194304] 0 2024-04-24T22:45:14.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/32: getdents d5 0 2024-04-24T22:45:14.472 INFO:tasks.workunit.client.0.smithi063.stdout:1/33: rename d3/d6 to d3/da/dc 0 2024-04-24T22:45:14.472 INFO:tasks.workunit.client.0.smithi063.stdout:1/34: read f1 [1281327,59653] 0 2024-04-24T22:45:14.473 INFO:tasks.workunit.client.0.smithi063.stdout:3/32: symlink da/lb 0 2024-04-24T22:45:14.475 INFO:tasks.workunit.client.0.smithi063.stdout:8/20: sync 2024-04-24T22:45:14.475 INFO:tasks.workunit.client.0.smithi063.stdout:8/21: write - no filename 2024-04-24T22:45:14.478 INFO:tasks.workunit.client.0.smithi063.stdout:0/30: getdents d6 0 2024-04-24T22:45:14.479 INFO:tasks.workunit.client.0.smithi063.stdout:9/35: rename f4 to d6/fd 0 2024-04-24T22:45:14.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/35: creat d3/fd x:0 0 0 2024-04-24T22:45:14.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/33: fsync f0 0 2024-04-24T22:45:14.487 INFO:tasks.workunit.client.0.smithi063.stdout:8/22: creat d1/f5 x:0 0 0 2024-04-24T22:45:14.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/36: dread d6/fd [0,4194304] 0 2024-04-24T22:45:14.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/40: fsync f8 0 2024-04-24T22:45:14.490 INFO:tasks.workunit.client.0.smithi063.stdout:6/41: fdatasync f3 0 2024-04-24T22:45:14.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/31: mkdir d6/da 0 2024-04-24T22:45:14.492 INFO:tasks.workunit.client.0.smithi063.stdout:4/25: write f1 [1247494,47683] 0 2024-04-24T22:45:14.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/32: rename d0/f3 to d0/d4/f11 0 2024-04-24T22:45:14.494 INFO:tasks.workunit.client.0.smithi063.stdout:1/36: dwrite d3/da/dc/fb [0,4194304] 0 2024-04-24T22:45:14.495 INFO:tasks.workunit.client.0.smithi063.stdout:6/42: truncate db/f10 1010383 0 2024-04-24T22:45:14.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/37: chown d3/da 6 1 2024-04-24T22:45:14.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/34: unlink c4 0 2024-04-24T22:45:14.501 INFO:tasks.workunit.client.0.smithi063.stdout:4/26: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:14.501 INFO:tasks.workunit.client.0.smithi063.stdout:8/23: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:14.502 INFO:tasks.workunit.client.0.smithi063.stdout:8/24: write d1/f5 [3289683,65354] 0 2024-04-24T22:45:14.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/25: write d1/f5 [2403508,99034] 0 2024-04-24T22:45:14.504 INFO:tasks.workunit.client.0.smithi063.stdout:9/37: write d6/fd [1472701,42912] 0 2024-04-24T22:45:14.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/32: mknod d6/cb 0 2024-04-24T22:45:14.507 INFO:tasks.workunit.client.0.smithi063.stdout:0/33: chown l3 23263937 1 2024-04-24T22:45:14.510 INFO:tasks.workunit.client.0.smithi063.stdout:2/46: getdents d0/d2/d3 0 2024-04-24T22:45:14.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/43: creat db/f12 x:0 0 0 2024-04-24T22:45:14.511 INFO:tasks.workunit.client.0.smithi063.stdout:3/33: truncate f3 2941390 0 2024-04-24T22:45:14.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/47: dread d0/d2/d3/d7/fc [0,4194304] 0 2024-04-24T22:45:14.513 INFO:tasks.workunit.client.0.smithi063.stdout:1/38: creat d3/da/dc/fe x:0 0 0 2024-04-24T22:45:14.516 INFO:tasks.workunit.client.0.smithi063.stdout:0/34: dwrite f5 [0,4194304] 0 2024-04-24T22:45:14.518 INFO:tasks.workunit.client.0.smithi063.stdout:2/48: truncate d0/d2/d3/d7/fc 1361101 0 2024-04-24T22:45:14.525 INFO:tasks.workunit.client.0.smithi063.stdout:6/44: dwrite db/f12 [0,4194304] 0 2024-04-24T22:45:14.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/26: fsync d1/f5 0 2024-04-24T22:45:14.564 INFO:tasks.workunit.client.0.smithi063.stdout:4/27: mknod d2/c7 0 2024-04-24T22:45:14.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/38: mknod d6/ce 0 2024-04-24T22:45:14.571 INFO:tasks.workunit.client.0.smithi063.stdout:7/33: creat d0/d4/de/f12 x:0 0 0 2024-04-24T22:45:14.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/34: creat da/fc x:0 0 0 2024-04-24T22:45:14.581 INFO:tasks.workunit.client.0.smithi063.stdout:6/45: unlink fa 0 2024-04-24T22:45:14.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/27: mknod d1/c6 0 2024-04-24T22:45:14.585 INFO:tasks.workunit.client.0.smithi063.stdout:9/39: creat d6/d8/ff x:0 0 0 2024-04-24T22:45:14.586 INFO:tasks.workunit.client.0.smithi063.stdout:9/40: chown d6/d8/c9 966020146 1 2024-04-24T22:45:14.587 INFO:tasks.workunit.client.0.smithi063.stdout:9/41: write d6/fd [1032542,57445] 0 2024-04-24T22:45:14.587 INFO:tasks.workunit.client.0.smithi063.stdout:7/34: symlink d0/d4/l13 0 2024-04-24T22:45:14.588 INFO:tasks.workunit.client.0.smithi063.stdout:9/42: chown d6/d8/db 6 1 2024-04-24T22:45:14.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/35: mknod da/cd 0 2024-04-24T22:45:14.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/36: truncate f7 303995 0 2024-04-24T22:45:14.590 INFO:tasks.workunit.client.0.smithi063.stdout:1/39: rename d3/da/dc/fe to d3/ff 0 2024-04-24T22:45:14.592 INFO:tasks.workunit.client.0.smithi063.stdout:0/35: mkdir d6/da/dc 0 2024-04-24T22:45:14.597 INFO:tasks.workunit.client.0.smithi063.stdout:6/46: mkdir db/d13 0 2024-04-24T22:45:14.598 INFO:tasks.workunit.client.0.smithi063.stdout:5/35: link f0 d5/fd 0 2024-04-24T22:45:14.598 INFO:tasks.workunit.client.0.smithi063.stdout:5/36: readlink d5/la 0 2024-04-24T22:45:14.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/37: truncate d5/fc 662807 0 2024-04-24T22:45:14.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/28: mknod d1/d2/c7 0 2024-04-24T22:45:14.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/29: chown c0 874 1 2024-04-24T22:45:14.602 INFO:tasks.workunit.client.0.smithi063.stdout:7/35: symlink d0/d4/de/l14 0 2024-04-24T22:45:14.604 INFO:tasks.workunit.client.0.smithi063.stdout:5/38: dread f1 [0,4194304] 0 2024-04-24T22:45:14.607 INFO:tasks.workunit.client.0.smithi063.stdout:0/36: mkdir d6/dd 0 2024-04-24T22:45:14.613 INFO:tasks.workunit.client.0.smithi063.stdout:8/30: dwrite d1/f5 [4194304,4194304] 0 2024-04-24T22:45:14.613 INFO:tasks.workunit.client.0.smithi063.stdout:9/43: creat d6/d8/db/f10 x:0 0 0 2024-04-24T22:45:14.615 INFO:tasks.workunit.client.0.smithi063.stdout:8/31: chown d1/d2 51 1 2024-04-24T22:45:14.621 INFO:tasks.workunit.client.0.smithi063.stdout:7/36: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:14.622 INFO:tasks.workunit.client.0.smithi063.stdout:1/40: dwrite d3/ff [0,4194304] 0 2024-04-24T22:45:14.623 INFO:tasks.workunit.client.0.smithi063.stdout:0/37: dread f5 [0,4194304] 0 2024-04-24T22:45:14.623 INFO:tasks.workunit.client.0.smithi063.stdout:0/38: fdatasync f4 0 2024-04-24T22:45:14.624 INFO:tasks.workunit.client.0.smithi063.stdout:5/39: link d5/la d5/le 0 2024-04-24T22:45:14.625 INFO:tasks.workunit.client.0.smithi063.stdout:0/39: truncate f5 4604524 0 2024-04-24T22:45:14.626 INFO:tasks.workunit.client.0.smithi063.stdout:0/40: write f4 [4106,101246] 0 2024-04-24T22:45:14.630 INFO:tasks.workunit.client.0.smithi063.stdout:8/32: fsync d1/f5 0 2024-04-24T22:45:14.631 INFO:tasks.workunit.client.0.smithi063.stdout:7/37: rename d0/lf to d0/d4/de/l15 0 2024-04-24T22:45:14.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/40: chown f0 132 1 2024-04-24T22:45:14.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/38: dread - d0/d4/de/f12 zero size 2024-04-24T22:45:14.634 INFO:tasks.workunit.client.0.smithi063.stdout:1/41: dwrite d3/f7 [0,4194304] 0 2024-04-24T22:45:14.636 INFO:tasks.workunit.client.0.smithi063.stdout:7/39: write d0/f7 [337705,62485] 0 2024-04-24T22:45:14.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/42: stat d3/da/dc 0 2024-04-24T22:45:14.637 INFO:tasks.workunit.client.0.smithi063.stdout:0/41: truncate d6/f7 44021 0 2024-04-24T22:45:14.638 INFO:tasks.workunit.client.0.smithi063.stdout:4/28: getdents d2 0 2024-04-24T22:45:14.639 INFO:tasks.workunit.client.0.smithi063.stdout:1/43: write d3/da/dc/fb [422571,66023] 0 2024-04-24T22:45:14.640 INFO:tasks.workunit.client.0.smithi063.stdout:0/42: dread f4 [0,4194304] 0 2024-04-24T22:45:14.640 INFO:tasks.workunit.client.0.smithi063.stdout:7/40: truncate d0/f7 961789 0 2024-04-24T22:45:14.641 INFO:tasks.workunit.client.0.smithi063.stdout:5/41: readlink d5/le 0 2024-04-24T22:45:14.641 INFO:tasks.workunit.client.0.smithi063.stdout:8/33: dread d1/f5 [4194304,4194304] 0 2024-04-24T22:45:14.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/29: mkdir d2/d8 0 2024-04-24T22:45:14.646 INFO:tasks.workunit.client.0.smithi063.stdout:0/43: creat d6/fe x:0 0 0 2024-04-24T22:45:14.648 INFO:tasks.workunit.client.0.smithi063.stdout:5/42: unlink d5/cb 0 2024-04-24T22:45:14.649 INFO:tasks.workunit.client.0.smithi063.stdout:8/34: creat d1/d2/f8 x:0 0 0 2024-04-24T22:45:14.653 INFO:tasks.workunit.client.0.smithi063.stdout:3/37: fsync f3 0 2024-04-24T22:45:14.657 INFO:tasks.workunit.client.0.smithi063.stdout:4/30: symlink d2/d8/l9 0 2024-04-24T22:45:14.660 INFO:tasks.workunit.client.0.smithi063.stdout:8/35: link d1/f5 d1/d2/d3/f9 0 2024-04-24T22:45:14.662 INFO:tasks.workunit.client.0.smithi063.stdout:8/36: read - d1/d2/f8 zero size 2024-04-24T22:45:14.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/38: unlink f3 0 2024-04-24T22:45:14.664 INFO:tasks.workunit.client.0.smithi063.stdout:6/47: dwrite db/f12 [4194304,4194304] 0 2024-04-24T22:45:14.665 INFO:tasks.workunit.client.0.smithi063.stdout:8/37: write d1/d2/f8 [678329,96665] 0 2024-04-24T22:45:14.665 INFO:tasks.workunit.client.0.smithi063.stdout:3/39: dread f7 [0,4194304] 0 2024-04-24T22:45:14.666 INFO:tasks.workunit.client.0.smithi063.stdout:5/43: truncate f0 915778 0 2024-04-24T22:45:14.667 INFO:tasks.workunit.client.0.smithi063.stdout:3/40: truncate f7 1075708 0 2024-04-24T22:45:14.668 INFO:tasks.workunit.client.0.smithi063.stdout:4/31: link d2/c7 d2/ca 0 2024-04-24T22:45:14.675 INFO:tasks.workunit.client.0.smithi063.stdout:6/48: dwrite f7 [0,4194304] 0 2024-04-24T22:45:14.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/41: mknod da/ce 0 2024-04-24T22:45:14.677 INFO:tasks.workunit.client.0.smithi063.stdout:6/49: readlink l5 0 2024-04-24T22:45:14.678 INFO:tasks.workunit.client.0.smithi063.stdout:2/49: truncate d0/d2/d3/d7/f9 1774131 0 2024-04-24T22:45:14.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/42: write f7 [487886,38098] 0 2024-04-24T22:45:14.685 INFO:tasks.workunit.client.0.smithi063.stdout:8/38: dwrite d1/f5 [4194304,4194304] 0 2024-04-24T22:45:14.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/32: dwrite d2/f5 [0,4194304] 0 2024-04-24T22:45:14.689 INFO:tasks.workunit.client.0.smithi063.stdout:3/43: symlink da/lf 0 2024-04-24T22:45:14.691 INFO:tasks.workunit.client.0.smithi063.stdout:6/50: dread db/ff [0,4194304] 0 2024-04-24T22:45:14.692 INFO:tasks.workunit.client.0.smithi063.stdout:3/44: truncate da/fc 327188 0 2024-04-24T22:45:14.693 INFO:tasks.workunit.client.0.smithi063.stdout:4/33: creat d2/fb x:0 0 0 2024-04-24T22:45:14.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/39: mknod d1/d2/d3/ca 0 2024-04-24T22:45:14.694 INFO:tasks.workunit.client.0.smithi063.stdout:9/44: rmdir d6/d8 39 2024-04-24T22:45:14.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/45: truncate da/fc 612791 0 2024-04-24T22:45:14.695 INFO:tasks.workunit.client.0.smithi063.stdout:6/51: write db/f12 [8063781,31739] 0 2024-04-24T22:45:14.698 INFO:tasks.workunit.client.0.smithi063.stdout:9/45: mknod d6/c11 0 2024-04-24T22:45:14.700 INFO:tasks.workunit.client.0.smithi063.stdout:6/52: dread f7 [0,4194304] 0 2024-04-24T22:45:14.700 INFO:tasks.workunit.client.0.smithi063.stdout:3/46: symlink da/l10 0 2024-04-24T22:45:14.701 INFO:tasks.workunit.client.0.smithi063.stdout:3/47: dread da/fc [0,4194304] 0 2024-04-24T22:45:14.701 INFO:tasks.workunit.client.0.smithi063.stdout:3/48: readlink l9 0 2024-04-24T22:45:14.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/40: symlink d1/d2/d3/lb 0 2024-04-24T22:45:14.702 INFO:tasks.workunit.client.0.smithi063.stdout:3/49: fsync da/fc 0 2024-04-24T22:45:14.704 INFO:tasks.workunit.client.0.smithi063.stdout:9/46: unlink c5 0 2024-04-24T22:45:14.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/53: truncate f8 386878 0 2024-04-24T22:45:14.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/54: chown c2 64199097 1 2024-04-24T22:45:14.708 INFO:tasks.workunit.client.0.smithi063.stdout:8/41: symlink d1/lc 0 2024-04-24T22:45:14.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/55: readlink l9 0 2024-04-24T22:45:14.709 INFO:tasks.workunit.client.0.smithi063.stdout:9/47: dread d6/fd [0,4194304] 0 2024-04-24T22:45:14.712 INFO:tasks.workunit.client.0.smithi063.stdout:8/42: dread d1/d2/d3/f9 [0,4194304] 0 2024-04-24T22:45:14.713 INFO:tasks.workunit.client.0.smithi063.stdout:8/43: readlink d1/d2/d3/l4 0 2024-04-24T22:45:14.714 INFO:tasks.workunit.client.0.smithi063.stdout:8/44: chown d1/d2/d3/l4 426801847 1 2024-04-24T22:45:14.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/56: rmdir db 39 2024-04-24T22:45:14.716 INFO:tasks.workunit.client.0.smithi063.stdout:8/45: fsync d1/d2/f8 0 2024-04-24T22:45:14.717 INFO:tasks.workunit.client.0.smithi063.stdout:8/46: fdatasync d1/d2/d3/f9 0 2024-04-24T22:45:14.717 INFO:tasks.workunit.client.0.smithi063.stdout:8/47: stat d1/d2/f8 0 2024-04-24T22:45:14.723 INFO:tasks.workunit.client.0.smithi063.stdout:4/34: getdents d2/d8 0 2024-04-24T22:45:14.724 INFO:tasks.workunit.client.0.smithi063.stdout:6/57: fdatasync db/f10 0 2024-04-24T22:45:14.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/58: write db/fe [499571,3241] 0 2024-04-24T22:45:14.727 INFO:tasks.workunit.client.0.smithi063.stdout:6/59: fdatasync db/fe 0 2024-04-24T22:45:14.727 INFO:tasks.workunit.client.0.smithi063.stdout:8/48: creat d1/d2/fd x:0 0 0 2024-04-24T22:45:14.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/50: link c6 da/c11 0 2024-04-24T22:45:14.728 INFO:tasks.workunit.client.0.smithi063.stdout:8/49: chown d1/d2/f8 1 1 2024-04-24T22:45:14.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/48: creat d6/f12 x:0 0 0 2024-04-24T22:45:14.732 INFO:tasks.workunit.client.0.smithi063.stdout:2/50: read d0/d2/d3/d7/f9 [526966,19476] 0 2024-04-24T22:45:14.736 INFO:tasks.workunit.client.0.smithi063.stdout:3/51: creat da/f12 x:0 0 0 2024-04-24T22:45:14.743 INFO:tasks.workunit.client.0.smithi063.stdout:3/52: dread da/fc [0,4194304] 0 2024-04-24T22:45:14.743 INFO:tasks.workunit.client.0.smithi063.stdout:6/60: fsync f8 0 2024-04-24T22:45:14.745 INFO:tasks.workunit.client.0.smithi063.stdout:8/50: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-04-24T22:45:14.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/49: dwrite d6/d8/ff [0,4194304] 0 2024-04-24T22:45:14.749 INFO:tasks.workunit.client.0.smithi063.stdout:5/44: write d5/fd [363490,100943] 0 2024-04-24T22:45:14.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/44: sync 2024-04-24T22:45:14.751 INFO:tasks.workunit.client.0.smithi063.stdout:4/35: write f1 [5158941,129285] 0 2024-04-24T22:45:14.754 INFO:tasks.workunit.client.0.smithi063.stdout:8/51: unlink d1/d2/d3/f9 0 2024-04-24T22:45:14.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/36: readlink d2/d8/l9 0 2024-04-24T22:45:14.756 INFO:tasks.workunit.client.0.smithi063.stdout:5/45: creat d5/ff x:0 0 0 2024-04-24T22:45:14.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/44: dwrite d6/f7 [0,4194304] 0 2024-04-24T22:45:14.757 INFO:tasks.workunit.client.0.smithi063.stdout:2/51: write d0/d2/d3/d7/fc [963835,39057] 0 2024-04-24T22:45:14.759 INFO:tasks.workunit.client.0.smithi063.stdout:1/45: symlink d3/da/dc/l10 0 2024-04-24T22:45:14.759 INFO:tasks.workunit.client.0.smithi063.stdout:8/52: truncate d1/d2/fd 598422 0 2024-04-24T22:45:14.759 INFO:tasks.workunit.client.0.smithi063.stdout:5/46: write d5/f6 [874906,81642] 0 2024-04-24T22:45:14.761 INFO:tasks.workunit.client.0.smithi063.stdout:6/61: truncate db/f12 4314975 0 2024-04-24T22:45:14.764 INFO:tasks.workunit.client.0.smithi063.stdout:1/46: mkdir d3/da/dc/d11 0 2024-04-24T22:45:14.768 INFO:tasks.workunit.client.0.smithi063.stdout:5/47: rename d5/fc to d5/f10 0 2024-04-24T22:45:14.768 INFO:tasks.workunit.client.0.smithi063.stdout:8/53: dwrite d1/d2/f8 [0,4194304] 0 2024-04-24T22:45:14.769 INFO:tasks.workunit.client.0.smithi063.stdout:5/48: write f0 [57964,72539] 0 2024-04-24T22:45:14.771 INFO:tasks.workunit.client.0.smithi063.stdout:9/50: write d6/fd [4266352,120497] 0 2024-04-24T22:45:14.772 INFO:tasks.workunit.client.0.smithi063.stdout:5/49: truncate d5/fd 1821373 0 2024-04-24T22:45:14.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/62: rename db/d13 to db/d14 0 2024-04-24T22:45:14.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/63: fdatasync db/f10 0 2024-04-24T22:45:14.776 INFO:tasks.workunit.client.0.smithi063.stdout:1/47: dread f1 [4194304,4194304] 0 2024-04-24T22:45:14.776 INFO:tasks.workunit.client.0.smithi063.stdout:5/50: fdatasync f0 0 2024-04-24T22:45:14.778 INFO:tasks.workunit.client.0.smithi063.stdout:6/64: dread db/f10 [0,4194304] 0 2024-04-24T22:45:14.819 INFO:tasks.workunit.client.0.smithi063.stdout:8/54: creat d1/d2/fe x:0 0 0 2024-04-24T22:45:14.820 INFO:tasks.workunit.client.0.smithi063.stdout:0/45: getdents d6/da 0 2024-04-24T22:45:14.822 INFO:tasks.workunit.client.0.smithi063.stdout:8/55: fsync d1/d2/fd 0 2024-04-24T22:45:14.822 INFO:tasks.workunit.client.0.smithi063.stdout:9/51: dwrite d6/d8/ff [0,4194304] 0 2024-04-24T22:45:14.822 INFO:tasks.workunit.client.0.smithi063.stdout:5/51: mkdir d5/d11 0 2024-04-24T22:45:14.823 INFO:tasks.workunit.client.0.smithi063.stdout:1/48: mknod d3/da/dc/c12 0 2024-04-24T22:45:14.824 INFO:tasks.workunit.client.0.smithi063.stdout:0/46: symlink d6/da/lf 0 2024-04-24T22:45:14.825 INFO:tasks.workunit.client.0.smithi063.stdout:0/47: stat d6/cb 0 2024-04-24T22:45:14.826 INFO:tasks.workunit.client.0.smithi063.stdout:8/56: symlink d1/d2/lf 0 2024-04-24T22:45:14.828 INFO:tasks.workunit.client.0.smithi063.stdout:5/52: creat d5/f12 x:0 0 0 2024-04-24T22:45:14.829 INFO:tasks.workunit.client.0.smithi063.stdout:6/65: rmdir db/d14 0 2024-04-24T22:45:14.830 INFO:tasks.workunit.client.0.smithi063.stdout:0/48: creat d6/da/f10 x:0 0 0 2024-04-24T22:45:14.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/66: mknod db/c15 0 2024-04-24T22:45:14.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/52: dwrite d6/fd [0,4194304] 0 2024-04-24T22:45:14.833 INFO:tasks.workunit.client.0.smithi063.stdout:1/49: link d3/c4 d3/da/dc/c13 0 2024-04-24T22:45:14.835 INFO:tasks.workunit.client.0.smithi063.stdout:8/57: dread d1/d2/f8 [0,4194304] 0 2024-04-24T22:45:14.835 INFO:tasks.workunit.client.0.smithi063.stdout:0/49: mknod d6/c11 0 2024-04-24T22:45:14.838 INFO:tasks.workunit.client.0.smithi063.stdout:5/53: dread f1 [0,4194304] 0 2024-04-24T22:45:14.838 INFO:tasks.workunit.client.0.smithi063.stdout:1/50: fdatasync f1 0 2024-04-24T22:45:14.841 INFO:tasks.workunit.client.0.smithi063.stdout:1/51: write d3/fd [400459,104935] 0 2024-04-24T22:45:14.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/53: dwrite d6/fd [4194304,4194304] 0 2024-04-24T22:45:14.843 INFO:tasks.workunit.client.0.smithi063.stdout:8/58: unlink d1/d2/f8 0 2024-04-24T22:45:14.845 INFO:tasks.workunit.client.0.smithi063.stdout:5/54: dread f0 [0,4194304] 0 2024-04-24T22:45:14.845 INFO:tasks.workunit.client.0.smithi063.stdout:0/50: rename f4 to d6/dd/f12 0 2024-04-24T22:45:14.846 INFO:tasks.workunit.client.0.smithi063.stdout:5/55: fsync d5/f9 0 2024-04-24T22:45:14.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/51: read - d6/fe zero size 2024-04-24T22:45:14.849 INFO:tasks.workunit.client.0.smithi063.stdout:7/41: sync 2024-04-24T22:45:14.850 INFO:tasks.workunit.client.0.smithi063.stdout:5/56: readlink d5/la 0 2024-04-24T22:45:14.851 INFO:tasks.workunit.client.0.smithi063.stdout:9/54: symlink d6/l13 0 2024-04-24T22:45:14.853 INFO:tasks.workunit.client.0.smithi063.stdout:8/59: dwrite d1/d2/fe [0,4194304] 0 2024-04-24T22:45:14.857 INFO:tasks.workunit.client.0.smithi063.stdout:6/67: fsync db/f12 0 2024-04-24T22:45:14.860 INFO:tasks.workunit.client.0.smithi063.stdout:7/42: symlink d0/l16 0 2024-04-24T22:45:14.861 INFO:tasks.workunit.client.0.smithi063.stdout:7/43: chown d0/d4/de/f12 592 1 2024-04-24T22:45:14.863 INFO:tasks.workunit.client.0.smithi063.stdout:5/57: mkdir d5/d13 0 2024-04-24T22:45:14.867 INFO:tasks.workunit.client.0.smithi063.stdout:0/52: creat d6/da/dc/f13 x:0 0 0 2024-04-24T22:45:14.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/60: rename d1/d2/c7 to d1/d2/d3/c10 0 2024-04-24T22:45:14.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/61: dread d1/d2/fd [0,4194304] 0 2024-04-24T22:45:14.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/68: mknod db/c16 0 2024-04-24T22:45:14.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/69: truncate f3 1783371 0 2024-04-24T22:45:14.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/70: fdatasync db/f10 0 2024-04-24T22:45:14.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/44: mkdir d0/d4/de/d17 0 2024-04-24T22:45:14.873 INFO:tasks.workunit.client.0.smithi063.stdout:9/55: rmdir d6 39 2024-04-24T22:45:14.875 INFO:tasks.workunit.client.0.smithi063.stdout:1/52: truncate d3/fd 413273 0 2024-04-24T22:45:14.877 INFO:tasks.workunit.client.0.smithi063.stdout:6/71: dread db/f12 [0,4194304] 0 2024-04-24T22:45:14.881 INFO:tasks.workunit.client.0.smithi063.stdout:0/53: dwrite f5 [0,4194304] 0 2024-04-24T22:45:14.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/58: rmdir d5/d13 0 2024-04-24T22:45:14.883 INFO:tasks.workunit.client.0.smithi063.stdout:0/54: fdatasync d6/da/dc/f13 0 2024-04-24T22:45:14.883 INFO:tasks.workunit.client.0.smithi063.stdout:6/72: chown db/cc 630448 1 2024-04-24T22:45:14.885 INFO:tasks.workunit.client.0.smithi063.stdout:6/73: chown db/c16 383 1 2024-04-24T22:45:14.885 INFO:tasks.workunit.client.0.smithi063.stdout:5/59: dread d5/f10 [0,4194304] 0 2024-04-24T22:45:14.886 INFO:tasks.workunit.client.0.smithi063.stdout:6/74: fsync db/fe 0 2024-04-24T22:45:14.887 INFO:tasks.workunit.client.0.smithi063.stdout:5/60: chown d5/le 0 1 2024-04-24T22:45:14.887 INFO:tasks.workunit.client.0.smithi063.stdout:9/56: creat d6/f14 x:0 0 0 2024-04-24T22:45:14.888 INFO:tasks.workunit.client.0.smithi063.stdout:8/62: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:14.889 INFO:tasks.workunit.client.0.smithi063.stdout:8/63: readlink d1/lc 0 2024-04-24T22:45:14.889 INFO:tasks.workunit.client.0.smithi063.stdout:8/64: fdatasync d1/d2/fd 0 2024-04-24T22:45:14.890 INFO:tasks.workunit.client.0.smithi063.stdout:1/53: write f1 [3019060,121552] 0 2024-04-24T22:45:14.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/45: getdents d0/d4/de 0 2024-04-24T22:45:14.892 INFO:tasks.workunit.client.0.smithi063.stdout:6/75: truncate db/ff 1676191 0 2024-04-24T22:45:14.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/61: creat d5/f14 x:0 0 0 2024-04-24T22:45:14.894 INFO:tasks.workunit.client.0.smithi063.stdout:5/62: write f0 [783084,98740] 0 2024-04-24T22:45:14.896 INFO:tasks.workunit.client.0.smithi063.stdout:8/65: mkdir d1/d11 0 2024-04-24T22:45:14.896 INFO:tasks.workunit.client.0.smithi063.stdout:9/57: fsync d6/f14 0 2024-04-24T22:45:14.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/54: mknod d3/da/dc/c14 0 2024-04-24T22:45:14.897 INFO:tasks.workunit.client.0.smithi063.stdout:6/76: dread f3 [0,4194304] 0 2024-04-24T22:45:14.898 INFO:tasks.workunit.client.0.smithi063.stdout:8/66: dread d1/d2/fd [0,4194304] 0 2024-04-24T22:45:14.900 INFO:tasks.workunit.client.0.smithi063.stdout:8/67: fsync d1/d2/fd 0 2024-04-24T22:45:14.900 INFO:tasks.workunit.client.0.smithi063.stdout:7/46: mknod d0/d4/c18 0 2024-04-24T22:45:14.901 INFO:tasks.workunit.client.0.smithi063.stdout:7/47: readlink d0/d4/de/l15 0 2024-04-24T22:45:14.902 INFO:tasks.workunit.client.0.smithi063.stdout:8/68: truncate d1/d2/fd 1192903 0 2024-04-24T22:45:14.902 INFO:tasks.workunit.client.0.smithi063.stdout:1/55: dread f1 [0,4194304] 0 2024-04-24T22:45:14.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/63: unlink d5/f12 0 2024-04-24T22:45:14.903 INFO:tasks.workunit.client.0.smithi063.stdout:7/48: fsync d0/d4/f8 0 2024-04-24T22:45:14.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/58: unlink d6/f12 0 2024-04-24T22:45:14.908 INFO:tasks.workunit.client.0.smithi063.stdout:3/53: sync 2024-04-24T22:45:14.909 INFO:tasks.workunit.client.0.smithi063.stdout:0/55: write d6/dd/f12 [774845,87187] 0 2024-04-24T22:45:14.910 INFO:tasks.workunit.client.0.smithi063.stdout:6/77: write f3 [2404804,61508] 0 2024-04-24T22:45:14.911 INFO:tasks.workunit.client.0.smithi063.stdout:0/56: stat d6/l8 0 2024-04-24T22:45:14.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/69: rmdir d1 39 2024-04-24T22:45:14.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/49: creat d0/d4/f19 x:0 0 0 2024-04-24T22:45:14.916 INFO:tasks.workunit.client.0.smithi063.stdout:3/54: dwrite f7 [0,4194304] 0 2024-04-24T22:45:14.918 INFO:tasks.workunit.client.0.smithi063.stdout:3/55: dread da/fc [0,4194304] 0 2024-04-24T22:45:14.922 INFO:tasks.workunit.client.0.smithi063.stdout:9/59: rmdir d6 39 2024-04-24T22:45:14.922 INFO:tasks.workunit.client.0.smithi063.stdout:0/57: creat d6/da/f14 x:0 0 0 2024-04-24T22:45:14.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/70: write d1/d2/fd [244897,42177] 0 2024-04-24T22:45:14.925 INFO:tasks.workunit.client.0.smithi063.stdout:5/64: creat d5/d11/f15 x:0 0 0 2024-04-24T22:45:14.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/71: chown d1 3276 1 2024-04-24T22:45:14.926 INFO:tasks.workunit.client.0.smithi063.stdout:7/50: creat d0/d4/de/f1a x:0 0 0 2024-04-24T22:45:14.927 INFO:tasks.workunit.client.0.smithi063.stdout:3/56: rmdir da 39 2024-04-24T22:45:14.928 INFO:tasks.workunit.client.0.smithi063.stdout:7/51: read d0/fd [1683615,98342] 0 2024-04-24T22:45:14.931 INFO:tasks.workunit.client.0.smithi063.stdout:0/58: dwrite d6/da/f14 [0,4194304] 0 2024-04-24T22:45:14.933 INFO:tasks.workunit.client.0.smithi063.stdout:9/60: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:14.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/65: creat d5/f16 x:0 0 0 2024-04-24T22:45:14.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/59: truncate d6/da/f14 4213387 0 2024-04-24T22:45:14.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/60: fsync f5 0 2024-04-24T22:45:14.934 INFO:tasks.workunit.client.0.smithi063.stdout:8/72: symlink d1/l12 0 2024-04-24T22:45:14.934 INFO:tasks.workunit.client.0.smithi063.stdout:9/61: dread - d6/d8/db/f10 zero size 2024-04-24T22:45:14.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/61: truncate d6/da/f10 193255 0 2024-04-24T22:45:14.937 INFO:tasks.workunit.client.0.smithi063.stdout:1/56: write d3/fd [872308,46468] 0 2024-04-24T22:45:14.937 INFO:tasks.workunit.client.0.smithi063.stdout:3/57: dread da/fc [0,4194304] 0 2024-04-24T22:45:14.938 INFO:tasks.workunit.client.0.smithi063.stdout:1/57: truncate d3/fd 1232348 0 2024-04-24T22:45:14.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/78: link db/cc db/c17 0 2024-04-24T22:45:14.942 INFO:tasks.workunit.client.0.smithi063.stdout:2/52: sync 2024-04-24T22:45:14.942 INFO:tasks.workunit.client.0.smithi063.stdout:4/37: sync 2024-04-24T22:45:14.942 INFO:tasks.workunit.client.0.smithi063.stdout:8/73: dwrite d1/d2/fd [0,4194304] 0 2024-04-24T22:45:14.943 INFO:tasks.workunit.client.0.smithi063.stdout:8/74: fdatasync d1/f5 0 2024-04-24T22:45:14.944 INFO:tasks.workunit.client.0.smithi063.stdout:8/75: readlink d1/lc 0 2024-04-24T22:45:14.945 INFO:tasks.workunit.client.0.smithi063.stdout:4/38: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:14.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/62: unlink d6/c11 0 2024-04-24T22:45:14.948 INFO:tasks.workunit.client.0.smithi063.stdout:3/58: chown l8 123 1 2024-04-24T22:45:14.950 INFO:tasks.workunit.client.0.smithi063.stdout:3/59: dread da/fc [0,4194304] 0 2024-04-24T22:45:14.952 INFO:tasks.workunit.client.0.smithi063.stdout:9/62: mknod d6/c15 0 2024-04-24T22:45:14.953 INFO:tasks.workunit.client.0.smithi063.stdout:8/76: rename d1/l12 to d1/d11/l13 0 2024-04-24T22:45:14.955 INFO:tasks.workunit.client.0.smithi063.stdout:4/39: mknod d2/cc 0 2024-04-24T22:45:14.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/52: write d0/d4/de/f10 [4738734,5326] 0 2024-04-24T22:45:14.955 INFO:tasks.workunit.client.0.smithi063.stdout:0/63: rmdir d6/dd 39 2024-04-24T22:45:14.956 INFO:tasks.workunit.client.0.smithi063.stdout:7/53: readlink d0/d4/l13 0 2024-04-24T22:45:14.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/63: symlink d6/d8/l16 0 2024-04-24T22:45:14.962 INFO:tasks.workunit.client.0.smithi063.stdout:5/66: dwrite f1 [0,4194304] 0 2024-04-24T22:45:14.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/40: creat d2/fd x:0 0 0 2024-04-24T22:45:14.968 INFO:tasks.workunit.client.0.smithi063.stdout:7/54: symlink d0/d4/de/l1b 0 2024-04-24T22:45:14.969 INFO:tasks.workunit.client.0.smithi063.stdout:7/55: readlink d0/l16 0 2024-04-24T22:45:14.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/64: creat d6/d8/f17 x:0 0 0 2024-04-24T22:45:14.974 INFO:tasks.workunit.client.0.smithi063.stdout:2/53: truncate d0/d2/d3/d7/f9 104918 0 2024-04-24T22:45:14.975 INFO:tasks.workunit.client.0.smithi063.stdout:5/67: creat d5/d11/f17 x:0 0 0 2024-04-24T22:45:14.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/65: chown d6/c11 10210 1 2024-04-24T22:45:14.976 INFO:tasks.workunit.client.0.smithi063.stdout:8/77: link d1/f5 d1/d2/d3/f14 0 2024-04-24T22:45:14.978 INFO:tasks.workunit.client.0.smithi063.stdout:7/56: dwrite d0/d4/de/f1a [0,4194304] 0 2024-04-24T22:45:14.978 INFO:tasks.workunit.client.0.smithi063.stdout:8/78: stat d1/d11 0 2024-04-24T22:45:14.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/79: dwrite db/ff [0,4194304] 0 2024-04-24T22:45:14.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/41: stat d2/ca 0 2024-04-24T22:45:14.979 INFO:tasks.workunit.client.0.smithi063.stdout:0/64: symlink d6/dd/l15 0 2024-04-24T22:45:14.980 INFO:tasks.workunit.client.0.smithi063.stdout:3/60: truncate f7 3464402 0 2024-04-24T22:45:14.981 INFO:tasks.workunit.client.0.smithi063.stdout:3/61: readlink da/lb 0 2024-04-24T22:45:14.981 INFO:tasks.workunit.client.0.smithi063.stdout:1/58: link d3/c4 d3/da/dc/c15 0 2024-04-24T22:45:14.981 INFO:tasks.workunit.client.0.smithi063.stdout:7/57: write d0/d4/f19 [269131,120662] 0 2024-04-24T22:45:14.982 INFO:tasks.workunit.client.0.smithi063.stdout:2/54: creat d0/d2/fe x:0 0 0 2024-04-24T22:45:14.984 INFO:tasks.workunit.client.0.smithi063.stdout:5/68: mkdir d5/d11/d18 0 2024-04-24T22:45:14.986 INFO:tasks.workunit.client.0.smithi063.stdout:8/79: creat d1/d11/f15 x:0 0 0 2024-04-24T22:45:14.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/42: creat d2/d8/fe x:0 0 0 2024-04-24T22:45:14.990 INFO:tasks.workunit.client.0.smithi063.stdout:6/80: unlink db/cd 0 2024-04-24T22:45:14.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/43: write d2/fd [760686,60968] 0 2024-04-24T22:45:14.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/65: rmdir d6/dd 39 2024-04-24T22:45:14.993 INFO:tasks.workunit.client.0.smithi063.stdout:5/69: dread f1 [0,4194304] 0 2024-04-24T22:45:14.994 INFO:tasks.workunit.client.0.smithi063.stdout:1/59: mknod d3/da/dc/c16 0 2024-04-24T22:45:14.997 INFO:tasks.workunit.client.0.smithi063.stdout:8/80: dwrite d1/d2/fe [0,4194304] 0 2024-04-24T22:45:14.997 INFO:tasks.workunit.client.0.smithi063.stdout:9/66: symlink d6/l18 0 2024-04-24T22:45:15.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/66: dread f5 [0,4194304] 0 2024-04-24T22:45:15.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/44: creat d2/ff x:0 0 0 2024-04-24T22:45:15.006 INFO:tasks.workunit.client.0.smithi063.stdout:5/70: rmdir d5 39 2024-04-24T22:45:15.007 INFO:tasks.workunit.client.0.smithi063.stdout:1/60: creat d3/f17 x:0 0 0 2024-04-24T22:45:15.008 INFO:tasks.workunit.client.0.smithi063.stdout:2/55: mknod d0/dd/cf 0 2024-04-24T22:45:15.009 INFO:tasks.workunit.client.0.smithi063.stdout:7/58: link d0/d4/f11 d0/d4/de/f1c 0 2024-04-24T22:45:15.011 INFO:tasks.workunit.client.0.smithi063.stdout:8/81: rmdir d1/d2/d3 39 2024-04-24T22:45:15.012 INFO:tasks.workunit.client.0.smithi063.stdout:9/67: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:15.013 INFO:tasks.workunit.client.0.smithi063.stdout:8/82: write d1/d2/fe [4124796,75665] 0 2024-04-24T22:45:15.016 INFO:tasks.workunit.client.0.smithi063.stdout:4/45: creat d2/d8/f10 x:0 0 0 2024-04-24T22:45:15.018 INFO:tasks.workunit.client.0.smithi063.stdout:4/46: readlink d2/d8/l9 0 2024-04-24T22:45:15.018 INFO:tasks.workunit.client.0.smithi063.stdout:7/59: dwrite d0/d4/f8 [0,4194304] 0 2024-04-24T22:45:15.019 INFO:tasks.workunit.client.0.smithi063.stdout:2/56: mknod d0/dd/c10 0 2024-04-24T22:45:15.021 INFO:tasks.workunit.client.0.smithi063.stdout:2/57: stat d0/d2/d3 0 2024-04-24T22:45:15.022 INFO:tasks.workunit.client.0.smithi063.stdout:9/68: rmdir d6/d8 39 2024-04-24T22:45:15.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/81: truncate f3 154131 0 2024-04-24T22:45:15.025 INFO:tasks.workunit.client.0.smithi063.stdout:8/83: symlink d1/d11/l16 0 2024-04-24T22:45:15.029 INFO:tasks.workunit.client.0.smithi063.stdout:1/61: symlink d3/da/dc/d11/l18 0 2024-04-24T22:45:15.030 INFO:tasks.workunit.client.0.smithi063.stdout:0/67: dwrite d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:15.031 INFO:tasks.workunit.client.0.smithi063.stdout:8/84: write d1/d2/fe [859065,125746] 0 2024-04-24T22:45:15.034 INFO:tasks.workunit.client.0.smithi063.stdout:4/47: creat d2/d8/f11 x:0 0 0 2024-04-24T22:45:15.034 INFO:tasks.workunit.client.0.smithi063.stdout:7/60: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:15.037 INFO:tasks.workunit.client.0.smithi063.stdout:9/69: chown d6/d8 1280846 1 2024-04-24T22:45:15.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/82: mknod db/c18 0 2024-04-24T22:45:15.038 INFO:tasks.workunit.client.0.smithi063.stdout:3/62: dread f7 [0,4194304] 0 2024-04-24T22:45:15.040 INFO:tasks.workunit.client.0.smithi063.stdout:7/61: write d0/fa [1271198,14509] 0 2024-04-24T22:45:15.042 INFO:tasks.workunit.client.0.smithi063.stdout:5/71: creat d5/d11/d18/f19 x:0 0 0 2024-04-24T22:45:15.048 INFO:tasks.workunit.client.0.smithi063.stdout:8/85: mkdir d1/d2/d17 0 2024-04-24T22:45:15.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/72: rename d5/d11/f17 to d5/d11/f1a 0 2024-04-24T22:45:15.051 INFO:tasks.workunit.client.0.smithi063.stdout:0/68: creat d6/f16 x:0 0 0 2024-04-24T22:45:15.052 INFO:tasks.workunit.client.0.smithi063.stdout:4/48: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:15.052 INFO:tasks.workunit.client.0.smithi063.stdout:9/70: dread d6/fd [4194304,4194304] 0 2024-04-24T22:45:15.053 INFO:tasks.workunit.client.0.smithi063.stdout:1/62: getdents d3 0 2024-04-24T22:45:15.053 INFO:tasks.workunit.client.0.smithi063.stdout:3/63: getdents da 0 2024-04-24T22:45:15.054 INFO:tasks.workunit.client.0.smithi063.stdout:5/73: mknod d5/d11/d18/c1b 0 2024-04-24T22:45:15.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/86: symlink d1/d2/d17/l18 0 2024-04-24T22:45:15.056 INFO:tasks.workunit.client.0.smithi063.stdout:9/71: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:15.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/64: dread da/fc [0,4194304] 0 2024-04-24T22:45:15.057 INFO:tasks.workunit.client.0.smithi063.stdout:9/72: write d6/d8/f17 [513132,47467] 0 2024-04-24T22:45:15.057 INFO:tasks.workunit.client.0.smithi063.stdout:9/73: readlink d6/l13 0 2024-04-24T22:45:15.058 INFO:tasks.workunit.client.0.smithi063.stdout:4/49: rmdir d2 39 2024-04-24T22:45:15.059 INFO:tasks.workunit.client.0.smithi063.stdout:7/62: write d0/d4/f11 [960672,113964] 0 2024-04-24T22:45:15.060 INFO:tasks.workunit.client.0.smithi063.stdout:8/87: dread d1/d2/fd [0,4194304] 0 2024-04-24T22:45:15.061 INFO:tasks.workunit.client.0.smithi063.stdout:0/69: creat d6/da/dc/f17 x:0 0 0 2024-04-24T22:45:15.062 INFO:tasks.workunit.client.0.smithi063.stdout:1/63: rename d3/ff to d3/f19 0 2024-04-24T22:45:15.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/88: write d1/d11/f15 [343127,106293] 0 2024-04-24T22:45:15.064 INFO:tasks.workunit.client.0.smithi063.stdout:9/74: dread d6/fd [4194304,4194304] 0 2024-04-24T22:45:15.064 INFO:tasks.workunit.client.0.smithi063.stdout:5/74: write d5/f10 [70022,49925] 0 2024-04-24T22:45:15.066 INFO:tasks.workunit.client.0.smithi063.stdout:3/65: symlink da/l13 0 2024-04-24T22:45:15.068 INFO:tasks.workunit.client.0.smithi063.stdout:3/66: write da/f12 [542175,63826] 0 2024-04-24T22:45:15.071 INFO:tasks.workunit.client.0.smithi063.stdout:8/89: symlink d1/l19 0 2024-04-24T22:45:15.072 INFO:tasks.workunit.client.0.smithi063.stdout:5/75: creat d5/f1c x:0 0 0 2024-04-24T22:45:15.073 INFO:tasks.workunit.client.0.smithi063.stdout:9/75: mkdir d6/d8/db/d19 0 2024-04-24T22:45:15.073 INFO:tasks.workunit.client.0.smithi063.stdout:0/70: dwrite d6/f7 [4194304,4194304] 0 2024-04-24T22:45:15.076 INFO:tasks.workunit.client.0.smithi063.stdout:0/71: rename d6 to d6/dd/d18 22 2024-04-24T22:45:15.076 INFO:tasks.workunit.client.0.smithi063.stdout:3/67: creat da/f14 x:0 0 0 2024-04-24T22:45:15.076 INFO:tasks.workunit.client.0.smithi063.stdout:5/76: write d5/fd [365259,20319] 0 2024-04-24T22:45:15.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/50: dwrite d2/d8/fe [0,4194304] 0 2024-04-24T22:45:15.079 INFO:tasks.workunit.client.0.smithi063.stdout:1/64: write f1 [1369428,5808] 0 2024-04-24T22:45:15.081 INFO:tasks.workunit.client.0.smithi063.stdout:1/65: chown d3/f19 14 1 2024-04-24T22:45:15.081 INFO:tasks.workunit.client.0.smithi063.stdout:3/68: dread da/fc [0,4194304] 0 2024-04-24T22:45:15.081 INFO:tasks.workunit.client.0.smithi063.stdout:9/76: rename d6/d8/lc to d6/d8/db/d19/l1a 0 2024-04-24T22:45:15.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/77: stat d6/c11 0 2024-04-24T22:45:15.083 INFO:tasks.workunit.client.0.smithi063.stdout:1/66: write d3/f17 [173923,94268] 0 2024-04-24T22:45:15.083 INFO:tasks.workunit.client.0.smithi063.stdout:5/77: mkdir d5/d1d 0 2024-04-24T22:45:15.085 INFO:tasks.workunit.client.0.smithi063.stdout:0/72: mknod d6/dd/c19 0 2024-04-24T22:45:15.085 INFO:tasks.workunit.client.0.smithi063.stdout:1/67: chown d3/da/dc/l10 11581329 1 2024-04-24T22:45:15.086 INFO:tasks.workunit.client.0.smithi063.stdout:0/73: chown d6/l8 18657618 1 2024-04-24T22:45:15.086 INFO:tasks.workunit.client.0.smithi063.stdout:1/68: chown d3/da/dc 2 1 2024-04-24T22:45:15.086 INFO:tasks.workunit.client.0.smithi063.stdout:0/74: readlink d6/dd/l15 0 2024-04-24T22:45:15.087 INFO:tasks.workunit.client.0.smithi063.stdout:0/75: chown d6/cb 27359686 1 2024-04-24T22:45:15.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/51: mknod d2/c12 0 2024-04-24T22:45:15.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/52: chown f1 5850280 1 2024-04-24T22:45:15.089 INFO:tasks.workunit.client.0.smithi063.stdout:9/78: dread d6/d8/ff [0,4194304] 0 2024-04-24T22:45:15.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/69: creat da/f15 x:0 0 0 2024-04-24T22:45:15.091 INFO:tasks.workunit.client.0.smithi063.stdout:5/78: fsync f1 0 2024-04-24T22:45:15.093 INFO:tasks.workunit.client.0.smithi063.stdout:8/90: getdents d1/d11 0 2024-04-24T22:45:15.093 INFO:tasks.workunit.client.0.smithi063.stdout:0/76: dread d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:15.093 INFO:tasks.workunit.client.0.smithi063.stdout:5/79: truncate d5/d11/f1a 460771 0 2024-04-24T22:45:15.093 INFO:tasks.workunit.client.0.smithi063.stdout:1/69: fsync d3/f19 0 2024-04-24T22:45:15.094 INFO:tasks.workunit.client.0.smithi063.stdout:0/77: fdatasync d6/da/dc/f13 0 2024-04-24T22:45:15.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/53: unlink d2/cc 0 2024-04-24T22:45:15.096 INFO:tasks.workunit.client.0.smithi063.stdout:3/70: rename c4 to da/c16 0 2024-04-24T22:45:15.098 INFO:tasks.workunit.client.0.smithi063.stdout:5/80: mknod d5/d11/d18/c1e 0 2024-04-24T22:45:15.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/81: write d5/fd [1494306,25927] 0 2024-04-24T22:45:15.100 INFO:tasks.workunit.client.0.smithi063.stdout:1/70: symlink d3/da/l1a 0 2024-04-24T22:45:15.100 INFO:tasks.workunit.client.0.smithi063.stdout:5/82: chown d5/la 0 1 2024-04-24T22:45:15.101 INFO:tasks.workunit.client.0.smithi063.stdout:0/78: mknod d6/da/c1a 0 2024-04-24T22:45:15.102 INFO:tasks.workunit.client.0.smithi063.stdout:3/71: mknod da/c17 0 2024-04-24T22:45:15.102 INFO:tasks.workunit.client.0.smithi063.stdout:3/72: readlink l5 0 2024-04-24T22:45:15.104 INFO:tasks.workunit.client.0.smithi063.stdout:8/91: rename d1/lc to d1/d2/d17/l1a 0 2024-04-24T22:45:15.104 INFO:tasks.workunit.client.0.smithi063.stdout:3/73: dread da/fc [0,4194304] 0 2024-04-24T22:45:15.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/79: creat d6/f1b x:0 0 0 2024-04-24T22:45:15.105 INFO:tasks.workunit.client.0.smithi063.stdout:9/80: write d6/d8/db/f10 [84358,107930] 0 2024-04-24T22:45:15.106 INFO:tasks.workunit.client.0.smithi063.stdout:1/71: creat d3/da/f1b x:0 0 0 2024-04-24T22:45:15.109 INFO:tasks.workunit.client.0.smithi063.stdout:0/79: rename c2 to d6/dd/c1b 0 2024-04-24T22:45:15.111 INFO:tasks.workunit.client.0.smithi063.stdout:8/92: creat d1/d11/f1b x:0 0 0 2024-04-24T22:45:15.113 INFO:tasks.workunit.client.0.smithi063.stdout:5/83: dwrite d5/d11/f1a [0,4194304] 0 2024-04-24T22:45:15.114 INFO:tasks.workunit.client.0.smithi063.stdout:9/81: unlink d6/d8/c9 0 2024-04-24T22:45:15.114 INFO:tasks.workunit.client.0.smithi063.stdout:5/84: chown d5/d11/d18/c1e 764530 1 2024-04-24T22:45:15.115 INFO:tasks.workunit.client.0.smithi063.stdout:9/82: write d6/d8/db/f10 [573569,123247] 0 2024-04-24T22:45:15.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/72: rename d3/da/dc/fb to d3/da/dc/f1c 0 2024-04-24T22:45:15.118 INFO:tasks.workunit.client.0.smithi063.stdout:1/73: write f1 [3499444,18585] 0 2024-04-24T22:45:15.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/80: symlink d6/da/dc/l1c 0 2024-04-24T22:45:15.120 INFO:tasks.workunit.client.0.smithi063.stdout:4/54: getdents d2 0 2024-04-24T22:45:15.135 INFO:tasks.workunit.client.0.smithi063.stdout:2/58: sync 2024-04-24T22:45:15.142 INFO:tasks.workunit.client.0.smithi063.stdout:1/74: mknod d3/c1d 0 2024-04-24T22:45:15.143 INFO:tasks.workunit.client.0.smithi063.stdout:3/74: write da/fc [644976,19660] 0 2024-04-24T22:45:15.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/85: dwrite d5/f16 [0,4194304] 0 2024-04-24T22:45:15.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/81: rename d6/cb to d6/da/dc/c1d 0 2024-04-24T22:45:15.144 INFO:tasks.workunit.client.0.smithi063.stdout:3/75: chown da 5 1 2024-04-24T22:45:15.145 INFO:tasks.workunit.client.0.smithi063.stdout:1/75: dread - d3/da/f1b zero size 2024-04-24T22:45:15.146 INFO:tasks.workunit.client.0.smithi063.stdout:3/76: fsync da/f12 0 2024-04-24T22:45:15.146 INFO:tasks.workunit.client.0.smithi063.stdout:5/86: chown d5/le 2838 1 2024-04-24T22:45:15.147 INFO:tasks.workunit.client.0.smithi063.stdout:5/87: chown l2 2634884 1 2024-04-24T22:45:15.150 INFO:tasks.workunit.client.0.smithi063.stdout:4/55: dwrite d2/d8/fe [0,4194304] 0 2024-04-24T22:45:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:14 smithi192 ceph-mon[33991]: pgmap v5: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:14 smithi192 ceph-mon[33991]: mgrmap e24: smithi192.ctmwwc(active, since 4s) 2024-04-24T22:45:15.243 INFO:tasks.workunit.client.0.smithi063.stdout:6/83: sync 2024-04-24T22:45:15.251 INFO:tasks.workunit.client.0.smithi063.stdout:8/93: creat d1/d2/f1c x:0 0 0 2024-04-24T22:45:15.252 INFO:tasks.workunit.client.0.smithi063.stdout:2/59: creat d0/dd/f11 x:0 0 0 2024-04-24T22:45:15.253 INFO:tasks.workunit.client.0.smithi063.stdout:0/82: rmdir d6/da 39 2024-04-24T22:45:15.254 INFO:tasks.workunit.client.0.smithi063.stdout:1/76: creat d3/f1e x:0 0 0 2024-04-24T22:45:15.255 INFO:tasks.workunit.client.0.smithi063.stdout:1/77: write d3/f1e [804523,109962] 0 2024-04-24T22:45:15.256 INFO:tasks.workunit.client.0.smithi063.stdout:4/56: creat d2/f13 x:0 0 0 2024-04-24T22:45:15.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/78: write d3/f7 [1015607,16884] 0 2024-04-24T22:45:15.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/57: dread - d2/d8/f10 zero size 2024-04-24T22:45:15.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/60: symlink d0/l12 0 2024-04-24T22:45:15.259 INFO:tasks.workunit.client.0.smithi063.stdout:4/58: write d2/fd [78740,5427] 0 2024-04-24T22:45:15.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/63: sync 2024-04-24T22:45:15.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/77: sync 2024-04-24T22:45:15.260 INFO:tasks.workunit.client.0.smithi063.stdout:9/83: sync 2024-04-24T22:45:15.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/88: getdents d5/d1d 0 2024-04-24T22:45:15.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/89: readlink l2 0 2024-04-24T22:45:15.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/61: chown d0/d2/d3/d7/fc 1193092 1 2024-04-24T22:45:15.262 INFO:tasks.workunit.client.0.smithi063.stdout:9/84: chown d6/d8/ff 63 1 2024-04-24T22:45:15.263 INFO:tasks.workunit.client.0.smithi063.stdout:5/90: chown d5/f14 586 1 2024-04-24T22:45:15.264 INFO:tasks.workunit.client.0.smithi063.stdout:8/94: dread d1/d2/d3/f14 [4194304,4194304] 0 2024-04-24T22:45:15.265 INFO:tasks.workunit.client.0.smithi063.stdout:5/91: dread - d5/d11/f15 zero size 2024-04-24T22:45:15.266 INFO:tasks.workunit.client.0.smithi063.stdout:3/78: mkdir da/d18 0 2024-04-24T22:45:15.266 INFO:tasks.workunit.client.0.smithi063.stdout:7/64: dread d0/f7 [0,4194304] 0 2024-04-24T22:45:15.267 INFO:tasks.workunit.client.0.smithi063.stdout:8/95: write d1/d11/f1b [694530,125717] 0 2024-04-24T22:45:15.267 INFO:tasks.workunit.client.0.smithi063.stdout:4/59: symlink d2/d8/l14 0 2024-04-24T22:45:15.268 INFO:tasks.workunit.client.0.smithi063.stdout:5/92: truncate d5/ff 692191 0 2024-04-24T22:45:15.269 INFO:tasks.workunit.client.0.smithi063.stdout:0/83: rename d6/f16 to d6/da/dc/f1e 0 2024-04-24T22:45:15.270 INFO:tasks.workunit.client.0.smithi063.stdout:4/60: readlink d2/d8/l9 0 2024-04-24T22:45:15.270 INFO:tasks.workunit.client.0.smithi063.stdout:6/84: link c2 db/c19 0 2024-04-24T22:45:15.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/62: creat d0/d2/d3/d7/f13 x:0 0 0 2024-04-24T22:45:15.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/85: mkdir d6/d8/db/d1c 0 2024-04-24T22:45:15.273 INFO:tasks.workunit.client.0.smithi063.stdout:6/85: fdatasync db/fe 0 2024-04-24T22:45:15.273 INFO:tasks.workunit.client.0.smithi063.stdout:4/61: read d2/f6 [90808,50058] 0 2024-04-24T22:45:15.273 INFO:tasks.workunit.client.0.smithi063.stdout:5/93: truncate d5/f9 804616 0 2024-04-24T22:45:15.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/63: stat d0/dd 0 2024-04-24T22:45:15.274 INFO:tasks.workunit.client.0.smithi063.stdout:1/79: dwrite d3/da/dc/f1c [0,4194304] 0 2024-04-24T22:45:15.276 INFO:tasks.workunit.client.0.smithi063.stdout:8/96: symlink d1/l1d 0 2024-04-24T22:45:15.277 INFO:tasks.workunit.client.0.smithi063.stdout:2/64: read d0/d2/d3/d7/fc [828038,97888] 0 2024-04-24T22:45:15.278 INFO:tasks.workunit.client.0.smithi063.stdout:7/65: dread d0/fa [0,4194304] 0 2024-04-24T22:45:15.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/84: rename d6/dd/l15 to d6/dd/l1f 0 2024-04-24T22:45:15.279 INFO:tasks.workunit.client.0.smithi063.stdout:2/65: dread - d0/dd/f11 zero size 2024-04-24T22:45:15.279 INFO:tasks.workunit.client.0.smithi063.stdout:9/86: dread d6/d8/f17 [0,4194304] 0 2024-04-24T22:45:15.280 INFO:tasks.workunit.client.0.smithi063.stdout:0/85: fsync d6/fe 0 2024-04-24T22:45:15.280 INFO:tasks.workunit.client.0.smithi063.stdout:7/66: read d0/d4/de/f1c [708441,60706] 0 2024-04-24T22:45:15.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/79: creat da/d18/f19 x:0 0 0 2024-04-24T22:45:15.282 INFO:tasks.workunit.client.0.smithi063.stdout:4/62: mknod d2/c15 0 2024-04-24T22:45:15.283 INFO:tasks.workunit.client.0.smithi063.stdout:2/66: dread - d0/d2/d3/d7/f13 zero size 2024-04-24T22:45:15.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/80: dread d3/f19 [0,4194304] 0 2024-04-24T22:45:15.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/67: chown d0/d2/l6 1738360914 1 2024-04-24T22:45:15.285 INFO:tasks.workunit.client.0.smithi063.stdout:1/81: write d3/fd [1296775,92372] 0 2024-04-24T22:45:15.286 INFO:tasks.workunit.client.0.smithi063.stdout:6/86: dread f7 [0,4194304] 0 2024-04-24T22:45:15.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/80: creat da/f1a x:0 0 0 2024-04-24T22:45:15.290 INFO:tasks.workunit.client.0.smithi063.stdout:0/86: mkdir d6/dd/d20 0 2024-04-24T22:45:15.290 INFO:tasks.workunit.client.0.smithi063.stdout:3/81: dread - da/f14 zero size 2024-04-24T22:45:15.291 INFO:tasks.workunit.client.0.smithi063.stdout:9/87: mknod d6/d8/db/c1d 0 2024-04-24T22:45:15.292 INFO:tasks.workunit.client.0.smithi063.stdout:3/82: readlink l8 0 2024-04-24T22:45:15.293 INFO:tasks.workunit.client.0.smithi063.stdout:4/63: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:15.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/88: chown d6/c11 2 1 2024-04-24T22:45:15.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/89: chown d6/d8/db/c1d 43 1 2024-04-24T22:45:15.295 INFO:tasks.workunit.client.0.smithi063.stdout:7/67: mknod d0/d4/c1d 0 2024-04-24T22:45:15.296 INFO:tasks.workunit.client.0.smithi063.stdout:7/68: readlink d0/d4/de/l15 0 2024-04-24T22:45:15.297 INFO:tasks.workunit.client.0.smithi063.stdout:5/94: link d5/f16 d5/f1f 0 2024-04-24T22:45:15.297 INFO:tasks.workunit.client.0.smithi063.stdout:0/87: dread d6/f7 [4194304,4194304] 0 2024-04-24T22:45:15.298 INFO:tasks.workunit.client.0.smithi063.stdout:5/95: chown d5/d11/d18/c1e 533470534 1 2024-04-24T22:45:15.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:14 smithi063 ceph-mon[26912]: pgmap v5: 65 pgs: 65 active+clean; 252 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:45:15.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:14 smithi063 ceph-mon[26912]: mgrmap e24: smithi192.ctmwwc(active, since 4s) 2024-04-24T22:45:15.305 INFO:tasks.workunit.client.0.smithi063.stdout:2/68: rename d0/dd/c10 to d0/d2/d3/db/c14 0 2024-04-24T22:45:15.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/69: dread - d0/d2/fe zero size 2024-04-24T22:45:15.307 INFO:tasks.workunit.client.0.smithi063.stdout:2/70: dread - d0/d2/fe zero size 2024-04-24T22:45:15.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/71: fsync d0/d2/fe 0 2024-04-24T22:45:15.309 INFO:tasks.workunit.client.0.smithi063.stdout:1/82: dread d3/fd [0,4194304] 0 2024-04-24T22:45:15.309 INFO:tasks.workunit.client.0.smithi063.stdout:6/87: rename db/fe to db/f1a 0 2024-04-24T22:45:15.311 INFO:tasks.workunit.client.0.smithi063.stdout:3/83: mknod da/d18/c1b 0 2024-04-24T22:45:15.313 INFO:tasks.workunit.client.0.smithi063.stdout:9/90: creat d6/d8/db/d19/f1e x:0 0 0 2024-04-24T22:45:15.315 INFO:tasks.workunit.client.0.smithi063.stdout:5/96: creat d5/d11/d18/f20 x:0 0 0 2024-04-24T22:45:15.315 INFO:tasks.workunit.client.0.smithi063.stdout:9/91: stat d6/d8/db/d19 0 2024-04-24T22:45:15.316 INFO:tasks.workunit.client.0.smithi063.stdout:5/97: stat d5/f1c 0 2024-04-24T22:45:15.317 INFO:tasks.workunit.client.0.smithi063.stdout:6/88: creat db/f1b x:0 0 0 2024-04-24T22:45:15.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/83: dwrite d3/da/dc/f1c [4194304,4194304] 0 2024-04-24T22:45:15.318 INFO:tasks.workunit.client.0.smithi063.stdout:2/72: creat d0/d2/f15 x:0 0 0 2024-04-24T22:45:15.318 INFO:tasks.workunit.client.0.smithi063.stdout:6/89: read db/f12 [1676817,58730] 0 2024-04-24T22:45:15.319 INFO:tasks.workunit.client.0.smithi063.stdout:1/84: write d3/da/dc/f1c [437600,36710] 0 2024-04-24T22:45:15.320 INFO:tasks.workunit.client.0.smithi063.stdout:3/84: write f7 [2453219,111393] 0 2024-04-24T22:45:15.321 INFO:tasks.workunit.client.0.smithi063.stdout:7/69: creat d0/d4/de/d17/f1e x:0 0 0 2024-04-24T22:45:15.322 INFO:tasks.workunit.client.0.smithi063.stdout:0/88: mkdir d6/dd/d20/d21 0 2024-04-24T22:45:15.324 INFO:tasks.workunit.client.0.smithi063.stdout:9/92: mkdir d6/d8/d1f 0 2024-04-24T22:45:15.328 INFO:tasks.workunit.client.0.smithi063.stdout:6/90: mkdir db/d1c 0 2024-04-24T22:45:15.329 INFO:tasks.workunit.client.0.smithi063.stdout:2/73: mknod d0/d2/d3/d7/c16 0 2024-04-24T22:45:15.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/85: mkdir d3/da/dc/d1f 0 2024-04-24T22:45:15.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/85: mknod da/d18/c1c 0 2024-04-24T22:45:15.339 INFO:tasks.workunit.client.0.smithi063.stdout:6/91: mkdir db/d1d 0 2024-04-24T22:45:15.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/86: write d3/fd [921640,43479] 0 2024-04-24T22:45:15.344 INFO:tasks.workunit.client.0.smithi063.stdout:2/74: dread d0/d2/d3/d7/fc [0,4194304] 0 2024-04-24T22:45:15.344 INFO:tasks.workunit.client.0.smithi063.stdout:9/93: dwrite d6/d8/f17 [0,4194304] 0 2024-04-24T22:45:15.345 INFO:tasks.workunit.client.0.smithi063.stdout:0/89: mknod d6/dd/d20/d21/c22 0 2024-04-24T22:45:15.345 INFO:tasks.workunit.client.0.smithi063.stdout:6/92: fsync f8 0 2024-04-24T22:45:15.350 INFO:tasks.workunit.client.0.smithi063.stdout:9/94: mkdir d6/d8/d1f/d20 0 2024-04-24T22:45:15.357 INFO:tasks.workunit.client.0.smithi063.stdout:6/93: dwrite db/f1a [0,4194304] 0 2024-04-24T22:45:15.364 INFO:tasks.workunit.client.0.smithi063.stdout:9/95: dwrite d6/d8/db/d19/f1e [0,4194304] 0 2024-04-24T22:45:15.365 INFO:tasks.workunit.client.0.smithi063.stdout:9/96: readlink d6/l18 0 2024-04-24T22:45:15.367 INFO:tasks.workunit.client.0.smithi063.stdout:6/94: dread db/ff [0,4194304] 0 2024-04-24T22:45:15.402 INFO:tasks.workunit.client.0.smithi063.stdout:4/64: getdents d2/d8 0 2024-04-24T22:45:15.404 INFO:tasks.workunit.client.0.smithi063.stdout:4/65: creat d2/d8/f16 x:0 0 0 2024-04-24T22:45:15.405 INFO:tasks.workunit.client.0.smithi063.stdout:4/66: write d2/f5 [5924179,94857] 0 2024-04-24T22:45:15.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/67: mknod d2/c17 0 2024-04-24T22:45:15.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/68: readlink d2/d8/l14 0 2024-04-24T22:45:15.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/69: chown d2/f13 9086852 1 2024-04-24T22:45:15.410 INFO:tasks.workunit.client.0.smithi063.stdout:4/70: symlink d2/l18 0 2024-04-24T22:45:15.410 INFO:tasks.workunit.client.0.smithi063.stdout:8/97: write d1/d2/fd [5208864,120579] 0 2024-04-24T22:45:15.410 INFO:tasks.workunit.client.0.smithi063.stdout:4/71: fsync d2/ff 0 2024-04-24T22:45:15.411 INFO:tasks.workunit.client.0.smithi063.stdout:8/98: readlink d1/d11/l13 0 2024-04-24T22:45:15.412 INFO:tasks.workunit.client.0.smithi063.stdout:4/72: mkdir d2/d19 0 2024-04-24T22:45:15.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/99: creat d1/f1e x:0 0 0 2024-04-24T22:45:15.414 INFO:tasks.workunit.client.0.smithi063.stdout:4/73: creat d2/d8/f1a x:0 0 0 2024-04-24T22:45:15.415 INFO:tasks.workunit.client.0.smithi063.stdout:8/100: creat d1/f1f x:0 0 0 2024-04-24T22:45:15.416 INFO:tasks.workunit.client.0.smithi063.stdout:8/101: write d1/d2/f1c [692930,26547] 0 2024-04-24T22:45:15.418 INFO:tasks.workunit.client.0.smithi063.stdout:8/102: mknod d1/c20 0 2024-04-24T22:45:15.419 INFO:tasks.workunit.client.0.smithi063.stdout:4/74: link d2/d8/l9 d2/l1b 0 2024-04-24T22:45:15.421 INFO:tasks.workunit.client.0.smithi063.stdout:8/103: mkdir d1/d21 0 2024-04-24T22:45:15.424 INFO:tasks.workunit.client.0.smithi063.stdout:7/70: truncate d0/d4/f11 1629765 0 2024-04-24T22:45:15.425 INFO:tasks.workunit.client.0.smithi063.stdout:8/104: mkdir d1/d22 0 2024-04-24T22:45:15.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/105: link d1/d2/d3/ca d1/d2/c23 0 2024-04-24T22:45:15.432 INFO:tasks.workunit.client.0.smithi063.stdout:5/98: write f1 [3939782,120196] 0 2024-04-24T22:45:15.435 INFO:tasks.workunit.client.0.smithi063.stdout:3/86: rmdir da 39 2024-04-24T22:45:15.436 INFO:tasks.workunit.client.0.smithi063.stdout:3/87: write f7 [2825821,126583] 0 2024-04-24T22:45:15.437 INFO:tasks.workunit.client.0.smithi063.stdout:5/99: rename d5/d11/d18/c1b to d5/d1d/c21 0 2024-04-24T22:45:15.438 INFO:tasks.workunit.client.0.smithi063.stdout:8/106: dread d1/d2/fd [4194304,4194304] 0 2024-04-24T22:45:15.439 INFO:tasks.workunit.client.0.smithi063.stdout:5/100: mknod d5/c22 0 2024-04-24T22:45:15.440 INFO:tasks.workunit.client.0.smithi063.stdout:5/101: dread - d5/f14 zero size 2024-04-24T22:45:15.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/107: symlink d1/l24 0 2024-04-24T22:45:15.442 INFO:tasks.workunit.client.0.smithi063.stdout:3/88: creat da/f1d x:0 0 0 2024-04-24T22:45:15.443 INFO:tasks.workunit.client.0.smithi063.stdout:8/108: symlink d1/d2/d3/l25 0 2024-04-24T22:45:15.443 INFO:tasks.workunit.client.0.smithi063.stdout:5/102: truncate d5/d11/d18/f20 635723 0 2024-04-24T22:45:15.444 INFO:tasks.workunit.client.0.smithi063.stdout:3/89: unlink l8 0 2024-04-24T22:45:15.445 INFO:tasks.workunit.client.0.smithi063.stdout:0/90: dwrite d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:15.447 INFO:tasks.workunit.client.0.smithi063.stdout:5/103: mkdir d5/d11/d23 0 2024-04-24T22:45:15.449 INFO:tasks.workunit.client.0.smithi063.stdout:8/109: creat d1/d22/f26 x:0 0 0 2024-04-24T22:45:15.449 INFO:tasks.workunit.client.0.smithi063.stdout:3/90: mknod da/c1e 0 2024-04-24T22:45:15.450 INFO:tasks.workunit.client.0.smithi063.stdout:0/91: symlink d6/dd/d20/l23 0 2024-04-24T22:45:15.452 INFO:tasks.workunit.client.0.smithi063.stdout:8/110: dread - d1/f1f zero size 2024-04-24T22:45:15.453 INFO:tasks.workunit.client.0.smithi063.stdout:5/104: mkdir d5/d24 0 2024-04-24T22:45:15.454 INFO:tasks.workunit.client.0.smithi063.stdout:1/87: sync 2024-04-24T22:45:15.454 INFO:tasks.workunit.client.0.smithi063.stdout:3/91: read da/fc [473290,86976] 0 2024-04-24T22:45:15.456 INFO:tasks.workunit.client.0.smithi063.stdout:6/95: dwrite db/f12 [4194304,4194304] 0 2024-04-24T22:45:15.456 INFO:tasks.workunit.client.0.smithi063.stdout:4/75: rmdir d2/d8 39 2024-04-24T22:45:15.456 INFO:tasks.workunit.client.0.smithi063.stdout:5/105: read d5/f9 [767758,35766] 0 2024-04-24T22:45:15.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/92: mkdir d6/dd/d20/d24 0 2024-04-24T22:45:15.462 INFO:tasks.workunit.client.0.smithi063.stdout:8/111: rename d1/f1e to d1/d2/d17/f27 0 2024-04-24T22:45:15.464 INFO:tasks.workunit.client.0.smithi063.stdout:8/112: chown d1/d2/d3/l25 218 1 2024-04-24T22:45:15.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/93: mkdir d6/dd/d20/d21/d25 0 2024-04-24T22:45:15.466 INFO:tasks.workunit.client.0.smithi063.stdout:8/113: write d1/d2/fe [5000134,84587] 0 2024-04-24T22:45:15.467 INFO:tasks.workunit.client.0.smithi063.stdout:7/71: read d0/d4/de/f1c [1182767,42848] 0 2024-04-24T22:45:15.468 INFO:tasks.workunit.client.0.smithi063.stdout:4/76: dwrite d2/d8/f10 [0,4194304] 0 2024-04-24T22:45:15.469 INFO:tasks.workunit.client.0.smithi063.stdout:8/114: write d1/d22/f26 [448048,29181] 0 2024-04-24T22:45:15.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/88: creat d3/da/dc/d1f/f20 x:0 0 0 2024-04-24T22:45:15.475 INFO:tasks.workunit.client.0.smithi063.stdout:0/94: mkdir d6/da/dc/d26 0 2024-04-24T22:45:15.476 INFO:tasks.workunit.client.0.smithi063.stdout:3/92: rmdir da 39 2024-04-24T22:45:15.478 INFO:tasks.workunit.client.0.smithi063.stdout:1/89: mkdir d3/da/dc/d1f/d21 0 2024-04-24T22:45:15.479 INFO:tasks.workunit.client.0.smithi063.stdout:4/77: dwrite d2/d8/f11 [0,4194304] 0 2024-04-24T22:45:15.479 INFO:tasks.workunit.client.0.smithi063.stdout:1/90: stat d3/fd 0 2024-04-24T22:45:15.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/115: write d1/d2/d17/f27 [425493,21768] 0 2024-04-24T22:45:15.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/91: dread - d3/da/dc/d1f/f20 zero size 2024-04-24T22:45:15.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/92: write f1 [1337457,55038] 0 2024-04-24T22:45:15.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/106: truncate d5/f6 591202 0 2024-04-24T22:45:15.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/93: mknod da/d18/c1f 0 2024-04-24T22:45:15.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/0: sync 2024-04-24T22:45:15.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/1: rename - no filename 2024-04-24T22:45:15.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/2: write - no filename 2024-04-24T22:45:15.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/3: fdatasync - no filename 2024-04-24T22:45:15.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/4: link - no file 2024-04-24T22:45:15.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/72: getdents d0/d4/de 0 2024-04-24T22:45:15.496 INFO:tasks.workunit.client.0.smithi063.stdout:1/93: dwrite d3/fd [0,4194304] 0 2024-04-24T22:45:15.497 INFO:tasks.workunit.client.0.smithi063.stdout:3/94: symlink da/l20 0 2024-04-24T22:45:15.497 INFO:tasks.workunit.client.0.smithi063.stdout:0/95: write d6/f7 [5318457,101857] 0 2024-04-24T22:45:15.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/95: dread - da/f14 zero size 2024-04-24T22:45:15.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/107: symlink d5/d11/d18/l25 0 2024-04-24T22:45:15.500 INFO:tasks.workunit.client.1.smithi192.stdout:5/5: mkdir d0 0 2024-04-24T22:45:15.501 INFO:tasks.workunit.client.0.smithi063.stdout:0/96: fdatasync d6/dd/f12 0 2024-04-24T22:45:15.501 INFO:tasks.workunit.client.0.smithi063.stdout:7/73: write d0/d4/de/d17/f1e [926939,114494] 0 2024-04-24T22:45:15.501 INFO:tasks.workunit.client.0.smithi063.stdout:6/96: dwrite f3 [0,4194304] 0 2024-04-24T22:45:15.502 INFO:tasks.workunit.client.0.smithi063.stdout:4/78: truncate d2/d8/fe 2097117 0 2024-04-24T22:45:15.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/116: link d1/l19 d1/d2/l28 0 2024-04-24T22:45:15.506 INFO:tasks.workunit.client.1.smithi192.stdout:5/6: creat d0/f1 x:0 0 0 2024-04-24T22:45:15.507 INFO:tasks.workunit.client.0.smithi063.stdout:8/117: chown d1/d11/f15 59338 1 2024-04-24T22:45:15.507 INFO:tasks.workunit.client.1.smithi192.stdout:5/7: write d0/f1 [800471,11306] 0 2024-04-24T22:45:15.508 INFO:tasks.workunit.client.1.smithi192.stdout:5/8: stat d0/f1 0 2024-04-24T22:45:15.508 INFO:tasks.workunit.client.1.smithi192.stdout:5/9: chown d0 6 1 2024-04-24T22:45:15.509 INFO:tasks.workunit.client.0.smithi063.stdout:8/118: chown d1/d2/d3/f14 626 1 2024-04-24T22:45:15.510 INFO:tasks.workunit.client.0.smithi063.stdout:2/75: sync 2024-04-24T22:45:15.510 INFO:tasks.workunit.client.0.smithi063.stdout:9/97: sync 2024-04-24T22:45:15.511 INFO:tasks.workunit.client.0.smithi063.stdout:3/96: creat da/f21 x:0 0 0 2024-04-24T22:45:15.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/97: dread f7 [0,4194304] 0 2024-04-24T22:45:15.513 INFO:tasks.workunit.client.0.smithi063.stdout:5/108: creat d5/d11/f26 x:0 0 0 2024-04-24T22:45:15.513 INFO:tasks.workunit.client.0.smithi063.stdout:3/97: dread - da/f1d zero size 2024-04-24T22:45:15.514 INFO:tasks.workunit.client.0.smithi063.stdout:4/79: readlink d2/l1b 0 2024-04-24T22:45:15.514 INFO:tasks.workunit.client.0.smithi063.stdout:0/97: dwrite f5 [4194304,4194304] 0 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:9/32: sync 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/2: sync 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/3: dwrite - no filename 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/4: fsync - no filename 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/5: dread - no filename 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/6: dwrite - no filename 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:2/102: sync 2024-04-24T22:45:15.515 INFO:tasks.workunit.client.1.smithi192.stdout:1/37: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:4/16: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:0/80: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:3/30: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:6/13: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:7/26: sync 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.0.smithi063.stdout:2/76: dread - d0/d2/d3/d7/f13 zero size 2024-04-24T22:45:15.516 INFO:tasks.workunit.client.1.smithi192.stdout:1/38: fdatasync d9/fa 0 2024-04-24T22:45:15.517 INFO:tasks.workunit.client.0.smithi063.stdout:5/109: write d5/d11/d18/f20 [1628362,130406] 0 2024-04-24T22:45:15.519 INFO:tasks.workunit.client.0.smithi063.stdout:7/74: symlink d0/d4/de/d17/l1f 0 2024-04-24T22:45:15.520 INFO:tasks.workunit.client.0.smithi063.stdout:1/94: link d3/f19 d3/da/dc/d1f/f22 0 2024-04-24T22:45:15.521 INFO:tasks.workunit.client.1.smithi192.stdout:8/7: unlink l0 0 2024-04-24T22:45:15.522 INFO:tasks.workunit.client.1.smithi192.stdout:0/81: stat d3/le 0 2024-04-24T22:45:15.524 INFO:tasks.workunit.client.1.smithi192.stdout:0/82: truncate d3/f14 128583 0 2024-04-24T22:45:15.526 INFO:tasks.workunit.client.0.smithi063.stdout:8/119: dread d1/d2/fe [0,4194304] 0 2024-04-24T22:45:15.527 INFO:tasks.workunit.client.1.smithi192.stdout:3/31: mknod d2/c6 0 2024-04-24T22:45:15.530 INFO:tasks.workunit.client.1.smithi192.stdout:9/33: creat d3/fa x:0 0 0 2024-04-24T22:45:15.532 INFO:tasks.workunit.client.0.smithi063.stdout:4/80: dwrite d2/fb [0,4194304] 0 2024-04-24T22:45:15.533 INFO:tasks.workunit.client.0.smithi063.stdout:0/98: dwrite d6/da/dc/f13 [0,4194304] 0 2024-04-24T22:45:15.533 INFO:tasks.workunit.client.1.smithi192.stdout:6/14: symlink d0/l6 0 2024-04-24T22:45:15.535 INFO:tasks.workunit.client.0.smithi063.stdout:2/77: symlink d0/d2/d3/l17 0 2024-04-24T22:45:15.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/103: rmdir d4/d9/d10/d1c 39 2024-04-24T22:45:15.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/75: mkdir d0/d4/d20 0 2024-04-24T22:45:15.539 INFO:tasks.workunit.client.1.smithi192.stdout:1/39: mkdir d9/dd 0 2024-04-24T22:45:15.539 INFO:tasks.workunit.client.0.smithi063.stdout:2/78: write d0/dd/f11 [290390,42447] 0 2024-04-24T22:45:15.540 INFO:tasks.workunit.client.1.smithi192.stdout:1/40: read d9/fa [327964,118709] 0 2024-04-24T22:45:15.540 INFO:tasks.workunit.client.1.smithi192.stdout:1/41: chown l4 1447612656 1 2024-04-24T22:45:15.541 INFO:tasks.workunit.client.0.smithi063.stdout:2/79: fdatasync d0/d2/fe 0 2024-04-24T22:45:15.544 INFO:tasks.workunit.client.0.smithi063.stdout:0/99: dwrite d6/fe [0,4194304] 0 2024-04-24T22:45:15.547 INFO:tasks.workunit.client.0.smithi063.stdout:1/95: creat d3/f23 x:0 0 0 2024-04-24T22:45:15.548 INFO:tasks.workunit.client.0.smithi063.stdout:3/98: mknod da/c22 0 2024-04-24T22:45:15.549 INFO:tasks.workunit.client.1.smithi192.stdout:8/8: creat f1 x:0 0 0 2024-04-24T22:45:15.552 INFO:tasks.workunit.client.1.smithi192.stdout:0/83: rename d3/f11 to d3/d15/f17 0 2024-04-24T22:45:15.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/84: write d3/f10 [2974449,86254] 0 2024-04-24T22:45:15.554 INFO:tasks.workunit.client.1.smithi192.stdout:3/32: mkdir d2/d4/d7 0 2024-04-24T22:45:15.557 INFO:tasks.workunit.client.0.smithi063.stdout:6/98: symlink db/d1c/l1e 0 2024-04-24T22:45:15.558 INFO:tasks.workunit.client.1.smithi192.stdout:6/15: symlink d0/l7 0 2024-04-24T22:45:15.560 INFO:tasks.workunit.client.1.smithi192.stdout:6/16: chown d0/l7 3 1 2024-04-24T22:45:15.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/98: creat d6/d8/d1f/d20/f21 x:0 0 0 2024-04-24T22:45:15.564 INFO:tasks.workunit.client.1.smithi192.stdout:0/85: dwrite d3/f12 [0,4194304] 0 2024-04-24T22:45:15.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/99: write d6/f1b [238637,8121] 0 2024-04-24T22:45:15.565 INFO:tasks.workunit.client.1.smithi192.stdout:3/33: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:15.568 INFO:tasks.workunit.client.1.smithi192.stdout:5/10: getdents d0 0 2024-04-24T22:45:15.571 INFO:tasks.workunit.client.0.smithi063.stdout:5/110: creat d5/d11/d23/f27 x:0 0 0 2024-04-24T22:45:15.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/9: symlink l2 0 2024-04-24T22:45:15.577 INFO:tasks.workunit.client.0.smithi063.stdout:8/120: write d1/f5 [3416344,75273] 0 2024-04-24T22:45:15.579 INFO:tasks.workunit.client.0.smithi063.stdout:2/80: unlink d0/d2/fe 0 2024-04-24T22:45:15.579 INFO:tasks.workunit.client.0.smithi063.stdout:0/100: mknod d6/dd/d20/d21/c27 0 2024-04-24T22:45:15.580 INFO:tasks.workunit.client.0.smithi063.stdout:1/96: mknod d3/da/dc/c24 0 2024-04-24T22:45:15.581 INFO:tasks.workunit.client.0.smithi063.stdout:1/97: stat d3/c5 0 2024-04-24T22:45:15.582 INFO:tasks.workunit.client.0.smithi063.stdout:3/99: rename da/l10 to da/d18/l23 0 2024-04-24T22:45:15.583 INFO:tasks.workunit.client.0.smithi063.stdout:3/100: fsync da/f1a 0 2024-04-24T22:45:15.584 INFO:tasks.workunit.client.1.smithi192.stdout:0/86: symlink d3/d15/l18 0 2024-04-24T22:45:15.586 INFO:tasks.workunit.client.1.smithi192.stdout:3/34: mkdir d2/d4/d8 0 2024-04-24T22:45:15.588 INFO:tasks.workunit.client.1.smithi192.stdout:5/11: symlink d0/l2 0 2024-04-24T22:45:15.589 INFO:tasks.workunit.client.0.smithi063.stdout:1/98: dwrite d3/da/f1b [0,4194304] 0 2024-04-24T22:45:15.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/121: creat d1/d11/f29 x:0 0 0 2024-04-24T22:45:15.592 INFO:tasks.workunit.client.1.smithi192.stdout:8/10: creat f3 x:0 0 0 2024-04-24T22:45:15.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/122: write d1/d2/d17/f27 [1428006,58020] 0 2024-04-24T22:45:15.593 INFO:tasks.workunit.client.1.smithi192.stdout:8/11: write f1 [987882,20598] 0 2024-04-24T22:45:15.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/17: getdents d2 0 2024-04-24T22:45:15.600 INFO:tasks.workunit.client.0.smithi063.stdout:5/111: rename d5/d11/f26 to d5/f28 0 2024-04-24T22:45:15.601 INFO:tasks.workunit.client.1.smithi192.stdout:8/12: dread f1 [0,4194304] 0 2024-04-24T22:45:15.603 INFO:tasks.workunit.client.1.smithi192.stdout:4/18: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:15.605 INFO:tasks.workunit.client.0.smithi063.stdout:4/81: creat d2/f1c x:0 0 0 2024-04-24T22:45:15.605 INFO:tasks.workunit.client.0.smithi063.stdout:4/82: write d2/fb [2488641,53122] 0 2024-04-24T22:45:15.607 INFO:tasks.workunit.client.0.smithi063.stdout:3/101: creat da/f24 x:0 0 0 2024-04-24T22:45:15.612 INFO:tasks.workunit.client.0.smithi063.stdout:7/76: rmdir d0/d4/d20 0 2024-04-24T22:45:15.614 INFO:tasks.workunit.client.0.smithi063.stdout:7/77: stat d0/d4/de/l15 0 2024-04-24T22:45:15.615 INFO:tasks.workunit.client.1.smithi192.stdout:4/19: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:15.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/99: truncate f3 3361398 0 2024-04-24T22:45:15.616 INFO:tasks.workunit.client.1.smithi192.stdout:8/13: dwrite f3 [0,4194304] 0 2024-04-24T22:45:15.616 INFO:tasks.workunit.client.0.smithi063.stdout:9/100: dwrite d6/d8/f17 [4194304,4194304] 0 2024-04-24T22:45:15.617 INFO:tasks.workunit.client.1.smithi192.stdout:0/87: dwrite d3/fc [0,4194304] 0 2024-04-24T22:45:15.618 INFO:tasks.workunit.client.1.smithi192.stdout:8/14: write f3 [2763594,88332] 0 2024-04-24T22:45:15.618 INFO:tasks.workunit.client.1.smithi192.stdout:0/88: dread - d3/d15/f17 zero size 2024-04-24T22:45:15.619 INFO:tasks.workunit.client.0.smithi063.stdout:7/78: truncate d0/d4/de/f10 5751991 0 2024-04-24T22:45:15.619 INFO:tasks.workunit.client.0.smithi063.stdout:3/102: dwrite da/f1a [0,4194304] 0 2024-04-24T22:45:15.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/27: getdents d2/db 0 2024-04-24T22:45:15.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/28: read - f1 zero size 2024-04-24T22:45:15.620 INFO:tasks.workunit.client.1.smithi192.stdout:8/15: dread f1 [0,4194304] 0 2024-04-24T22:45:15.621 INFO:tasks.workunit.client.0.smithi063.stdout:3/103: write da/d18/f19 [910248,81745] 0 2024-04-24T22:45:15.621 INFO:tasks.workunit.client.1.smithi192.stdout:7/29: write d2/f5 [903172,25037] 0 2024-04-24T22:45:15.623 INFO:tasks.workunit.client.1.smithi192.stdout:7/30: write f1 [585394,19143] 0 2024-04-24T22:45:15.626 INFO:tasks.workunit.client.1.smithi192.stdout:9/34: getdents d3 0 2024-04-24T22:45:15.628 INFO:tasks.workunit.client.0.smithi063.stdout:2/81: rename d0/l5 to d0/d2/d3/d7/l18 0 2024-04-24T22:45:15.629 INFO:tasks.workunit.client.0.smithi063.stdout:2/82: dread - d0/d2/f15 zero size 2024-04-24T22:45:15.630 INFO:tasks.workunit.client.0.smithi063.stdout:2/83: fdatasync d0/d2/f15 0 2024-04-24T22:45:15.630 INFO:tasks.workunit.client.1.smithi192.stdout:2/104: rename d4/d9/d10/d1c/f21 to d4/d9/d10/f23 0 2024-04-24T22:45:15.633 INFO:tasks.workunit.client.1.smithi192.stdout:8/16: dwrite f1 [0,4194304] 0 2024-04-24T22:45:15.634 INFO:tasks.workunit.client.0.smithi063.stdout:4/83: creat d2/d8/f1d x:0 0 0 2024-04-24T22:45:15.635 INFO:tasks.workunit.client.0.smithi063.stdout:1/99: symlink d3/da/dc/d1f/d21/l25 0 2024-04-24T22:45:15.636 INFO:tasks.workunit.client.1.smithi192.stdout:1/42: getdents d9 0 2024-04-24T22:45:15.636 INFO:tasks.workunit.client.1.smithi192.stdout:3/35: creat d2/d4/d7/f9 x:0 0 0 2024-04-24T22:45:15.639 INFO:tasks.workunit.client.1.smithi192.stdout:2/105: truncate d4/d9/f18 354217 0 2024-04-24T22:45:15.641 INFO:tasks.workunit.client.0.smithi063.stdout:6/100: chown db/cc 3486 1 2024-04-24T22:45:15.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/35: dwrite d3/f9 [0,4194304] 0 2024-04-24T22:45:15.642 INFO:tasks.workunit.client.0.smithi063.stdout:6/101: write db/f12 [8436333,69477] 0 2024-04-24T22:45:15.642 INFO:tasks.workunit.client.1.smithi192.stdout:1/43: write d9/fc [2159758,83428] 0 2024-04-24T22:45:15.643 INFO:tasks.workunit.client.1.smithi192.stdout:3/36: write d2/d4/f5 [4512042,8655] 0 2024-04-24T22:45:15.643 INFO:tasks.workunit.client.0.smithi063.stdout:7/79: write d0/d4/de/f10 [5950853,52161] 0 2024-04-24T22:45:15.645 INFO:tasks.workunit.client.0.smithi063.stdout:8/123: mknod d1/d21/c2a 0 2024-04-24T22:45:15.647 INFO:tasks.workunit.client.1.smithi192.stdout:9/36: chown d3/f4 391990650 1 2024-04-24T22:45:15.651 INFO:tasks.workunit.client.1.smithi192.stdout:3/37: write d2/d4/f5 [5124927,126081] 0 2024-04-24T22:45:15.651 INFO:tasks.workunit.client.1.smithi192.stdout:0/89: unlink d3/fc 0 2024-04-24T22:45:15.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/17: dwrite f3 [0,4194304] 0 2024-04-24T22:45:15.653 INFO:tasks.workunit.client.1.smithi192.stdout:3/38: chown d2/d4/d8 1297223712 1 2024-04-24T22:45:15.654 INFO:tasks.workunit.client.1.smithi192.stdout:0/90: write d3/f14 [592699,68726] 0 2024-04-24T22:45:15.655 INFO:tasks.workunit.client.0.smithi063.stdout:3/104: stat da/c16 0 2024-04-24T22:45:15.656 INFO:tasks.workunit.client.0.smithi063.stdout:5/112: dwrite d5/f9 [0,4194304] 0 2024-04-24T22:45:15.662 INFO:tasks.workunit.client.1.smithi192.stdout:8/18: dread f3 [0,4194304] 0 2024-04-24T22:45:15.662 INFO:tasks.workunit.client.1.smithi192.stdout:6/17: getdents d0 0 2024-04-24T22:45:15.665 INFO:tasks.workunit.client.0.smithi063.stdout:4/84: symlink d2/d8/l1e 0 2024-04-24T22:45:15.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/39: dread d2/d4/f5 [0,4194304] 0 2024-04-24T22:45:15.667 INFO:tasks.workunit.client.1.smithi192.stdout:7/31: rename d2/l9 to d2/db/ld 0 2024-04-24T22:45:15.668 INFO:tasks.workunit.client.1.smithi192.stdout:3/40: fdatasync d2/d4/d7/f9 0 2024-04-24T22:45:15.669 INFO:tasks.workunit.client.1.smithi192.stdout:2/106: rename d4 to d4/d24 22 2024-04-24T22:45:15.670 INFO:tasks.workunit.client.1.smithi192.stdout:2/107: write d4/d9/d10/f1e [699269,27207] 0 2024-04-24T22:45:15.672 INFO:tasks.workunit.client.1.smithi192.stdout:2/108: truncate d4/d9/d10/f23 901561 0 2024-04-24T22:45:15.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/102: symlink db/l1f 0 2024-04-24T22:45:15.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/12: truncate d0/f1 244492 0 2024-04-24T22:45:15.677 INFO:tasks.workunit.client.0.smithi063.stdout:7/80: rename d0/d4/f8 to d0/d4/de/f21 0 2024-04-24T22:45:15.677 INFO:tasks.workunit.client.0.smithi063.stdout:5/113: rename d5/d11/d23 to d5/d11/d23/d29 22 2024-04-24T22:45:15.678 INFO:tasks.workunit.client.0.smithi063.stdout:8/124: symlink d1/d11/l2b 0 2024-04-24T22:45:15.679 INFO:tasks.workunit.client.0.smithi063.stdout:8/125: chown d1/c6 256216 1 2024-04-24T22:45:15.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/20: getdents d2 0 2024-04-24T22:45:15.684 INFO:tasks.workunit.client.0.smithi063.stdout:2/84: unlink d0/d2/d3/db/c14 0 2024-04-24T22:45:15.685 INFO:tasks.workunit.client.1.smithi192.stdout:0/91: dread d3/f14 [0,4194304] 0 2024-04-24T22:45:15.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/101: sync 2024-04-24T22:45:15.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/102: write d6/da/f14 [1208135,77560] 0 2024-04-24T22:45:15.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/21: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:15.688 INFO:tasks.workunit.client.0.smithi063.stdout:0/103: dread d6/da/f10 [0,4194304] 0 2024-04-24T22:45:15.688 INFO:tasks.workunit.client.1.smithi192.stdout:9/37: creat d3/fb x:0 0 0 2024-04-24T22:45:15.692 INFO:tasks.workunit.client.1.smithi192.stdout:9/38: dread d3/f9 [0,4194304] 0 2024-04-24T22:45:15.692 INFO:tasks.workunit.client.0.smithi063.stdout:6/103: chown db/c19 17587009 1 2024-04-24T22:45:15.696 INFO:tasks.workunit.client.0.smithi063.stdout:7/81: creat d0/d4/de/d17/f22 x:0 0 0 2024-04-24T22:45:15.697 INFO:tasks.workunit.client.1.smithi192.stdout:6/18: mkdir d0/d8 0 2024-04-24T22:45:15.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/114: mkdir d5/d11/d18/d2a 0 2024-04-24T22:45:15.698 INFO:tasks.workunit.client.1.smithi192.stdout:8/19: creat f4 x:0 0 0 2024-04-24T22:45:15.699 INFO:tasks.workunit.client.0.smithi063.stdout:8/126: creat d1/f2c x:0 0 0 2024-04-24T22:45:15.700 INFO:tasks.workunit.client.1.smithi192.stdout:7/32: symlink d2/db/le 0 2024-04-24T22:45:15.704 INFO:tasks.workunit.client.0.smithi063.stdout:3/105: link da/f12 da/d18/f25 0 2024-04-24T22:45:15.706 INFO:tasks.workunit.client.0.smithi063.stdout:8/127: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:15.706 INFO:tasks.workunit.client.0.smithi063.stdout:8/128: stat d1/d2/d3 0 2024-04-24T22:45:15.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/33: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:15.707 INFO:tasks.workunit.client.0.smithi063.stdout:8/129: write d1/d2/d17/f27 [1599108,42654] 0 2024-04-24T22:45:15.708 INFO:tasks.workunit.client.1.smithi192.stdout:2/109: write d4/d9/d10/d1c/f1d [457173,29624] 0 2024-04-24T22:45:15.708 INFO:tasks.workunit.client.0.smithi063.stdout:8/130: stat d1/d2/d3/l4 0 2024-04-24T22:45:15.712 INFO:tasks.workunit.client.1.smithi192.stdout:1/44: symlink d9/dd/le 0 2024-04-24T22:45:15.712 INFO:tasks.workunit.client.1.smithi192.stdout:1/45: readlink l1 0 2024-04-24T22:45:15.713 INFO:tasks.workunit.client.0.smithi063.stdout:4/85: creat d2/d19/f1f x:0 0 0 2024-04-24T22:45:15.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/86: write d2/fb [3429306,76541] 0 2024-04-24T22:45:15.715 INFO:tasks.workunit.client.0.smithi063.stdout:0/104: mknod d6/da/c28 0 2024-04-24T22:45:15.719 INFO:tasks.workunit.client.1.smithi192.stdout:4/22: symlink d2/l7 0 2024-04-24T22:45:15.720 INFO:tasks.workunit.client.0.smithi063.stdout:1/100: link d3/c5 d3/da/dc/d1f/c26 0 2024-04-24T22:45:15.725 INFO:tasks.workunit.client.0.smithi063.stdout:7/82: truncate d0/d4/de/f1c 1996083 0 2024-04-24T22:45:15.726 INFO:tasks.workunit.client.1.smithi192.stdout:9/39: fsync f2 0 2024-04-24T22:45:15.727 INFO:tasks.workunit.client.0.smithi063.stdout:9/101: sync 2024-04-24T22:45:15.728 INFO:tasks.workunit.client.0.smithi063.stdout:5/115: rename d5/c22 to d5/c2b 0 2024-04-24T22:45:15.729 INFO:tasks.workunit.client.0.smithi063.stdout:7/83: dread d0/d4/de/d17/f1e [0,4194304] 0 2024-04-24T22:45:15.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/131: fsync d1/d2/d3/f14 0 2024-04-24T22:45:15.735 INFO:tasks.workunit.client.1.smithi192.stdout:6/19: unlink d0/f2 0 2024-04-24T22:45:15.736 INFO:tasks.workunit.client.1.smithi192.stdout:8/20: fsync f3 0 2024-04-24T22:45:15.736 INFO:tasks.workunit.client.1.smithi192.stdout:8/21: chown f4 13691 1 2024-04-24T22:45:15.737 INFO:tasks.workunit.client.1.smithi192.stdout:8/22: chown f4 0 1 2024-04-24T22:45:15.738 INFO:tasks.workunit.client.1.smithi192.stdout:3/41: creat d2/d4/d8/fa x:0 0 0 2024-04-24T22:45:15.743 INFO:tasks.workunit.client.1.smithi192.stdout:8/23: dwrite f3 [0,4194304] 0 2024-04-24T22:45:15.744 INFO:tasks.workunit.client.1.smithi192.stdout:7/34: creat d2/db/ff x:0 0 0 2024-04-24T22:45:15.746 INFO:tasks.workunit.client.0.smithi063.stdout:0/105: unlink d6/fe 0 2024-04-24T22:45:15.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/101: creat d3/da/f27 x:0 0 0 2024-04-24T22:45:15.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/106: dread f5 [4194304,4194304] 0 2024-04-24T22:45:15.752 INFO:tasks.workunit.client.0.smithi063.stdout:6/104: mkdir db/d1d/d20 0 2024-04-24T22:45:15.752 INFO:tasks.workunit.client.1.smithi192.stdout:8/24: dwrite f4 [0,4194304] 0 2024-04-24T22:45:15.754 INFO:tasks.workunit.client.0.smithi063.stdout:2/85: write d0/d2/d3/d7/f9 [305533,107989] 0 2024-04-24T22:45:15.755 INFO:tasks.workunit.client.1.smithi192.stdout:1/46: dread f3 [0,4194304] 0 2024-04-24T22:45:15.758 INFO:tasks.workunit.client.1.smithi192.stdout:4/23: unlink d2/c4 0 2024-04-24T22:45:15.761 INFO:tasks.workunit.client.1.smithi192.stdout:9/40: symlink d3/lc 0 2024-04-24T22:45:15.762 INFO:tasks.workunit.client.1.smithi192.stdout:9/41: chown d3/fb 359 1 2024-04-24T22:45:15.764 INFO:tasks.workunit.client.0.smithi063.stdout:3/106: dwrite da/fc [0,4194304] 0 2024-04-24T22:45:15.766 INFO:tasks.workunit.client.0.smithi063.stdout:1/102: dwrite d3/da/f27 [0,4194304] 0 2024-04-24T22:45:15.770 INFO:tasks.workunit.client.1.smithi192.stdout:3/42: mknod d2/d4/d8/cb 0 2024-04-24T22:45:15.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/110: symlink d4/d9/d10/l25 0 2024-04-24T22:45:15.773 INFO:tasks.workunit.client.1.smithi192.stdout:2/111: read d4/d9/fb [3434822,111867] 0 2024-04-24T22:45:15.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/87: link d2/f13 d2/d19/f20 0 2024-04-24T22:45:15.776 INFO:tasks.workunit.client.1.smithi192.stdout:7/35: creat d2/db/f10 x:0 0 0 2024-04-24T22:45:15.779 INFO:tasks.workunit.client.0.smithi063.stdout:0/107: creat d6/da/f29 x:0 0 0 2024-04-24T22:45:15.780 INFO:tasks.workunit.client.0.smithi063.stdout:4/88: dwrite d2/d8/f11 [0,4194304] 0 2024-04-24T22:45:15.781 INFO:tasks.workunit.client.0.smithi063.stdout:4/89: chown d2/ff 2568 1 2024-04-24T22:45:15.782 INFO:tasks.workunit.client.0.smithi063.stdout:4/90: chown d2/d8 675 1 2024-04-24T22:45:15.782 INFO:tasks.workunit.client.0.smithi063.stdout:4/91: chown d2/f5 48107 1 2024-04-24T22:45:15.785 INFO:tasks.workunit.client.1.smithi192.stdout:0/92: rmdir d3/d16 0 2024-04-24T22:45:15.789 INFO:tasks.workunit.client.1.smithi192.stdout:1/47: creat d9/dd/ff x:0 0 0 2024-04-24T22:45:15.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/86: creat d0/d2/d3/d7/f19 x:0 0 0 2024-04-24T22:45:15.790 INFO:tasks.workunit.client.1.smithi192.stdout:4/24: symlink d2/l8 0 2024-04-24T22:45:15.790 INFO:tasks.workunit.client.1.smithi192.stdout:0/93: dread f2 [0,4194304] 0 2024-04-24T22:45:15.794 INFO:tasks.workunit.client.1.smithi192.stdout:9/42: mkdir d3/dd 0 2024-04-24T22:45:15.796 INFO:tasks.workunit.client.0.smithi063.stdout:3/107: unlink da/l13 0 2024-04-24T22:45:15.797 INFO:tasks.workunit.client.1.smithi192.stdout:6/20: symlink d0/d8/l9 0 2024-04-24T22:45:15.797 INFO:tasks.workunit.client.1.smithi192.stdout:6/21: dread - no filename 2024-04-24T22:45:15.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/43: dread d3/f9 [0,4194304] 0 2024-04-24T22:45:15.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/102: sync 2024-04-24T22:45:15.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/108: mknod d6/da/dc/c2a 0 2024-04-24T22:45:15.806 INFO:tasks.workunit.client.0.smithi063.stdout:7/84: truncate d0/d4/de/d17/f1e 41303 0 2024-04-24T22:45:15.809 INFO:tasks.workunit.client.0.smithi063.stdout:6/105: getdents db/d1d/d20 0 2024-04-24T22:45:15.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/87: rename d0/l12 to d0/dd/l1a 0 2024-04-24T22:45:15.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/88: fdatasync d0/d2/d3/d7/f13 0 2024-04-24T22:45:15.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/112: chown f0 61 1 2024-04-24T22:45:15.813 INFO:tasks.workunit.client.0.smithi063.stdout:2/89: truncate d0/d2/d3/d7/fc 2361484 0 2024-04-24T22:45:15.813 INFO:tasks.workunit.client.1.smithi192.stdout:7/36: read - d2/f7 zero size 2024-04-24T22:45:15.814 INFO:tasks.workunit.client.1.smithi192.stdout:7/37: fsync d2/f6 0 2024-04-24T22:45:15.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/108: unlink da/d18/c1b 0 2024-04-24T22:45:15.816 INFO:tasks.workunit.client.1.smithi192.stdout:5/13: write d0/f1 [139447,16881] 0 2024-04-24T22:45:15.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/103: truncate d3/da/dc/d1f/f22 3936702 0 2024-04-24T22:45:15.818 INFO:tasks.workunit.client.1.smithi192.stdout:5/14: write d0/f1 [247841,81967] 0 2024-04-24T22:45:15.818 INFO:tasks.workunit.client.1.smithi192.stdout:7/38: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:15.819 INFO:tasks.workunit.client.1.smithi192.stdout:1/48: creat d9/f10 x:0 0 0 2024-04-24T22:45:15.820 INFO:tasks.workunit.client.1.smithi192.stdout:7/39: write d2/f5 [1059517,95313] 0 2024-04-24T22:45:15.821 INFO:tasks.workunit.client.1.smithi192.stdout:7/40: chown d2/db/f10 505476 1 2024-04-24T22:45:15.821 INFO:tasks.workunit.client.0.smithi063.stdout:9/103: mkdir d6/d8/d1f/d20/d22 0 2024-04-24T22:45:15.823 INFO:tasks.workunit.client.1.smithi192.stdout:4/25: mkdir d2/d9 0 2024-04-24T22:45:15.824 INFO:tasks.workunit.client.1.smithi192.stdout:0/94: chown d3/cf 1 1 2024-04-24T22:45:15.825 INFO:tasks.workunit.client.0.smithi063.stdout:5/116: getdents d5/d11 0 2024-04-24T22:45:15.825 INFO:tasks.workunit.client.0.smithi063.stdout:9/104: dread d6/d8/db/f10 [0,4194304] 0 2024-04-24T22:45:15.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/22: creat d0/d8/fa x:0 0 0 2024-04-24T22:45:15.826 INFO:tasks.workunit.client.0.smithi063.stdout:5/117: dread d5/ff [0,4194304] 0 2024-04-24T22:45:15.826 INFO:tasks.workunit.client.0.smithi063.stdout:7/85: symlink d0/l23 0 2024-04-24T22:45:15.827 INFO:tasks.workunit.client.0.smithi063.stdout:9/105: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:15.828 INFO:tasks.workunit.client.0.smithi063.stdout:7/86: read d0/d4/f19 [142316,111093] 0 2024-04-24T22:45:15.828 INFO:tasks.workunit.client.1.smithi192.stdout:0/95: dread f2 [0,4194304] 0 2024-04-24T22:45:15.829 INFO:tasks.workunit.client.0.smithi063.stdout:6/106: rename db/f1b to db/f21 0 2024-04-24T22:45:15.829 INFO:tasks.workunit.client.1.smithi192.stdout:0/96: read f2 [2114853,60755] 0 2024-04-24T22:45:15.829 INFO:tasks.workunit.client.0.smithi063.stdout:7/87: write d0/d4/fb [2178101,90781] 0 2024-04-24T22:45:15.830 INFO:tasks.workunit.client.0.smithi063.stdout:8/132: link d1/d2/c23 d1/d2/d17/c2d 0 2024-04-24T22:45:15.832 INFO:tasks.workunit.client.1.smithi192.stdout:2/113: creat d4/d9/d10/d1c/f26 x:0 0 0 2024-04-24T22:45:15.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/90: creat d0/d2/f1b x:0 0 0 2024-04-24T22:45:15.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/133: dread d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:15.833 INFO:tasks.workunit.client.0.smithi063.stdout:2/91: chown d0/l1 68933 1 2024-04-24T22:45:15.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/109: symlink da/l26 0 2024-04-24T22:45:15.834 INFO:tasks.workunit.client.1.smithi192.stdout:6/23: dwrite d0/d8/fa [0,4194304] 0 2024-04-24T22:45:15.834 INFO:tasks.workunit.client.0.smithi063.stdout:2/92: dread - d0/d2/d3/d7/f19 zero size 2024-04-24T22:45:15.834 INFO:tasks.workunit.client.0.smithi063.stdout:3/110: write da/f14 [870495,67139] 0 2024-04-24T22:45:15.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/104: creat d3/f28 x:0 0 0 2024-04-24T22:45:15.835 INFO:tasks.workunit.client.0.smithi063.stdout:2/93: chown d0/d2/d3/db 7389341 1 2024-04-24T22:45:15.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/24: write d0/d8/fa [1369872,110085] 0 2024-04-24T22:45:15.836 INFO:tasks.workunit.client.0.smithi063.stdout:3/111: write da/f1a [3732997,116850] 0 2024-04-24T22:45:15.838 INFO:tasks.workunit.client.0.smithi063.stdout:2/94: truncate d0/d2/d3/d7/f19 685327 0 2024-04-24T22:45:15.839 INFO:tasks.workunit.client.0.smithi063.stdout:0/109: getdents d6/dd/d20/d21/d25 0 2024-04-24T22:45:15.840 INFO:tasks.workunit.client.1.smithi192.stdout:5/15: mknod d0/c3 0 2024-04-24T22:45:15.841 INFO:tasks.workunit.client.1.smithi192.stdout:5/16: stat d0 0 2024-04-24T22:45:15.841 INFO:tasks.workunit.client.0.smithi063.stdout:0/110: dread - d6/da/dc/f17 zero size 2024-04-24T22:45:15.841 INFO:tasks.workunit.client.0.smithi063.stdout:2/95: dread - d0/d2/d3/d7/f13 zero size 2024-04-24T22:45:15.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/111: write d6/dd/f12 [1006762,19179] 0 2024-04-24T22:45:15.845 INFO:tasks.workunit.client.0.smithi063.stdout:1/105: dwrite d3/f28 [0,4194304] 0 2024-04-24T22:45:15.847 INFO:tasks.workunit.client.0.smithi063.stdout:1/106: fdatasync f1 0 2024-04-24T22:45:15.850 INFO:tasks.workunit.client.1.smithi192.stdout:7/41: unlink d2/f7 0 2024-04-24T22:45:15.850 INFO:tasks.workunit.client.1.smithi192.stdout:7/42: chown d2 679624 1 2024-04-24T22:45:15.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/26: creat d2/fa x:0 0 0 2024-04-24T22:45:15.854 INFO:tasks.workunit.client.0.smithi063.stdout:2/96: dwrite d0/dd/f11 [0,4194304] 0 2024-04-24T22:45:15.856 INFO:tasks.workunit.client.1.smithi192.stdout:9/44: mkdir d3/dd/de 0 2024-04-24T22:45:15.856 INFO:tasks.workunit.client.0.smithi063.stdout:0/112: dread d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:15.858 INFO:tasks.workunit.client.0.smithi063.stdout:9/106: creat d6/d8/db/d19/f23 x:0 0 0 2024-04-24T22:45:15.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/43: link d2/d4/d8/cb d2/d4/d8/cc 0 2024-04-24T22:45:15.858 INFO:tasks.workunit.client.0.smithi063.stdout:1/107: dwrite d3/da/f27 [0,4194304] 0 2024-04-24T22:45:15.864 INFO:tasks.workunit.client.1.smithi192.stdout:4/27: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:15.866 INFO:tasks.workunit.client.0.smithi063.stdout:9/107: dread d6/f1b [0,4194304] 0 2024-04-24T22:45:15.867 INFO:tasks.workunit.client.0.smithi063.stdout:6/107: mknod db/d1c/c22 0 2024-04-24T22:45:15.868 INFO:tasks.workunit.client.0.smithi063.stdout:9/108: truncate d6/d8/d1f/d20/f21 39504 0 2024-04-24T22:45:15.868 INFO:tasks.workunit.client.0.smithi063.stdout:6/108: truncate db/f1a 4917236 0 2024-04-24T22:45:15.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/97: creat d3/d15/f19 x:0 0 0 2024-04-24T22:45:15.869 INFO:tasks.workunit.client.0.smithi063.stdout:6/109: chown f7 51560770 1 2024-04-24T22:45:15.872 INFO:tasks.workunit.client.0.smithi063.stdout:7/88: mknod d0/c24 0 2024-04-24T22:45:15.876 INFO:tasks.workunit.client.0.smithi063.stdout:8/134: creat d1/d22/f2e x:0 0 0 2024-04-24T22:45:15.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/135: chown d1/d2/f1c 265575 1 2024-04-24T22:45:15.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/136: chown d1/d2/f1c 15 1 2024-04-24T22:45:15.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/25: unlink d0/d8/l9 0 2024-04-24T22:45:15.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/26: rename d0 to d0/d8/db 22 2024-04-24T22:45:15.881 INFO:tasks.workunit.client.1.smithi192.stdout:5/17: mkdir d0/d4 0 2024-04-24T22:45:15.882 INFO:tasks.workunit.client.0.smithi063.stdout:3/112: readlink da/d18/l23 0 2024-04-24T22:45:15.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/44: unlink d2/c6 0 2024-04-24T22:45:15.892 INFO:tasks.workunit.client.1.smithi192.stdout:1/49: dwrite f3 [4194304,4194304] 0 2024-04-24T22:45:15.894 INFO:tasks.workunit.client.0.smithi063.stdout:4/92: getdents d2 0 2024-04-24T22:45:15.896 INFO:tasks.workunit.client.1.smithi192.stdout:4/28: mkdir d2/db 0 2024-04-24T22:45:15.899 INFO:tasks.workunit.client.1.smithi192.stdout:0/98: symlink d3/d15/l1a 0 2024-04-24T22:45:15.900 INFO:tasks.workunit.client.1.smithi192.stdout:0/99: chown d3/d15 2809833 1 2024-04-24T22:45:15.900 INFO:tasks.workunit.client.0.smithi063.stdout:4/93: dwrite d2/d8/f1a [0,4194304] 0 2024-04-24T22:45:15.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/100: rename d3/d15 to d3/d15/d1b 22 2024-04-24T22:45:15.902 INFO:tasks.workunit.client.0.smithi063.stdout:4/94: write d2/fd [299833,60060] 0 2024-04-24T22:45:15.903 INFO:tasks.workunit.client.1.smithi192.stdout:4/29: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:15.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/27: creat d0/d8/fc x:0 0 0 2024-04-24T22:45:15.907 INFO:tasks.workunit.client.1.smithi192.stdout:6/28: fdatasync d0/d8/fa 0 2024-04-24T22:45:15.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/97: mkdir d0/d2/d3/d1c 0 2024-04-24T22:45:15.915 INFO:tasks.workunit.client.1.smithi192.stdout:9/45: symlink d3/dd/de/lf 0 2024-04-24T22:45:15.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/98: fsync d0/d2/f1b 0 2024-04-24T22:45:15.916 INFO:tasks.workunit.client.0.smithi063.stdout:0/113: mkdir d6/da/d2b 0 2024-04-24T22:45:15.918 INFO:tasks.workunit.client.0.smithi063.stdout:0/114: write d6/da/f14 [1725783,110974] 0 2024-04-24T22:45:15.920 INFO:tasks.workunit.client.0.smithi063.stdout:2/99: dread d0/dd/f11 [0,4194304] 0 2024-04-24T22:45:15.921 INFO:tasks.workunit.client.1.smithi192.stdout:1/50: mknod d9/c11 0 2024-04-24T22:45:15.923 INFO:tasks.workunit.client.1.smithi192.stdout:7/43: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:15.923 INFO:tasks.workunit.client.1.smithi192.stdout:1/51: write d9/f10 [56760,21819] 0 2024-04-24T22:45:15.924 INFO:tasks.workunit.client.1.smithi192.stdout:7/44: chown d2/fa 40240752 1 2024-04-24T22:45:15.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/109: rename d6/d8/db/d19 to d6/d8/d24 0 2024-04-24T22:45:15.929 INFO:tasks.workunit.client.1.smithi192.stdout:0/101: read d3/f14 [400083,114544] 0 2024-04-24T22:45:15.930 INFO:tasks.workunit.client.1.smithi192.stdout:7/45: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:15.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/46: chown d2/l4 1 1 2024-04-24T22:45:15.931 INFO:tasks.workunit.client.1.smithi192.stdout:0/102: dread d3/f14 [0,4194304] 0 2024-04-24T22:45:15.933 INFO:tasks.workunit.client.0.smithi063.stdout:7/89: creat d0/d4/de/f25 x:0 0 0 2024-04-24T22:45:15.934 INFO:tasks.workunit.client.0.smithi063.stdout:7/90: readlink d0/l2 0 2024-04-24T22:45:15.936 INFO:tasks.workunit.client.1.smithi192.stdout:6/29: rmdir d0/d8 39 2024-04-24T22:45:15.938 INFO:tasks.workunit.client.1.smithi192.stdout:5/18: getdents d0/d4 0 2024-04-24T22:45:15.938 INFO:tasks.workunit.client.0.smithi063.stdout:7/91: dwrite d0/d4/de/f1a [0,4194304] 0 2024-04-24T22:45:15.940 INFO:tasks.workunit.client.0.smithi063.stdout:3/113: creat da/d18/f27 x:0 0 0 2024-04-24T22:45:15.944 INFO:tasks.workunit.client.1.smithi192.stdout:9/46: read d3/f4 [1657870,61001] 0 2024-04-24T22:45:15.946 INFO:tasks.workunit.client.0.smithi063.stdout:8/137: sync 2024-04-24T22:45:15.946 INFO:tasks.workunit.client.1.smithi192.stdout:5/19: dwrite d0/f1 [0,4194304] 0 2024-04-24T22:45:15.947 INFO:tasks.workunit.client.1.smithi192.stdout:1/52: creat d9/f12 x:0 0 0 2024-04-24T22:45:15.947 INFO:tasks.workunit.client.1.smithi192.stdout:1/53: readlink l4 0 2024-04-24T22:45:15.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/20: rename d0 to d0/d4/d5 22 2024-04-24T22:45:15.948 INFO:tasks.workunit.client.0.smithi063.stdout:7/92: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:15.948 INFO:tasks.workunit.client.1.smithi192.stdout:1/54: read - d9/dd/ff zero size 2024-04-24T22:45:15.950 INFO:tasks.workunit.client.0.smithi063.stdout:7/93: readlink d0/d4/l13 0 2024-04-24T22:45:15.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/114: dwrite da/f14 [0,4194304] 0 2024-04-24T22:45:15.957 INFO:tasks.workunit.client.1.smithi192.stdout:1/55: dwrite f3 [4194304,4194304] 0 2024-04-24T22:45:15.960 INFO:tasks.workunit.client.1.smithi192.stdout:1/56: write f3 [3179205,111298] 0 2024-04-24T22:45:15.960 INFO:tasks.workunit.client.1.smithi192.stdout:7/47: write d2/fa [4665219,40644] 0 2024-04-24T22:45:15.961 INFO:tasks.workunit.client.1.smithi192.stdout:1/57: chown d9/fc 0 1 2024-04-24T22:45:15.962 INFO:tasks.workunit.client.1.smithi192.stdout:7/48: write d2/f6 [247216,116016] 0 2024-04-24T22:45:15.968 INFO:tasks.workunit.client.1.smithi192.stdout:7/49: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:15.969 INFO:tasks.workunit.client.0.smithi063.stdout:0/115: symlink d6/dd/d20/l2c 0 2024-04-24T22:45:15.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/114: getdents d4/d9 0 2024-04-24T22:45:15.972 INFO:tasks.workunit.client.1.smithi192.stdout:4/30: mkdir d2/db/dc 0 2024-04-24T22:45:15.972 INFO:tasks.workunit.client.1.smithi192.stdout:0/103: write f2 [4726509,35938] 0 2024-04-24T22:45:15.973 INFO:tasks.workunit.client.1.smithi192.stdout:6/30: fsync d0/d8/fc 0 2024-04-24T22:45:15.974 INFO:tasks.workunit.client.1.smithi192.stdout:4/31: stat d2/d9 0 2024-04-24T22:45:15.975 INFO:tasks.workunit.client.1.smithi192.stdout:0/104: dread - d3/d15/f19 zero size 2024-04-24T22:45:15.978 INFO:tasks.workunit.client.1.smithi192.stdout:2/115: dread d4/f13 [0,4194304] 0 2024-04-24T22:45:15.979 INFO:tasks.workunit.client.0.smithi063.stdout:6/110: creat db/d1d/d20/f23 x:0 0 0 2024-04-24T22:45:15.984 INFO:tasks.workunit.client.1.smithi192.stdout:0/105: dread d3/f12 [0,4194304] 0 2024-04-24T22:45:15.984 INFO:tasks.workunit.client.1.smithi192.stdout:9/47: dwrite d3/f9 [0,4194304] 0 2024-04-24T22:45:15.988 INFO:tasks.workunit.client.0.smithi063.stdout:7/94: mkdir d0/d26 0 2024-04-24T22:45:15.989 INFO:tasks.workunit.client.0.smithi063.stdout:7/95: chown d0/d4/de/l14 2823769 1 2024-04-24T22:45:15.990 INFO:tasks.workunit.client.0.smithi063.stdout:3/115: rename da/d18/f25 to da/f28 0 2024-04-24T22:45:15.991 INFO:tasks.workunit.client.0.smithi063.stdout:3/116: truncate da/f14 4366837 0 2024-04-24T22:45:15.993 INFO:tasks.workunit.client.1.smithi192.stdout:4/32: mknod d2/cd 0 2024-04-24T22:45:15.993 INFO:tasks.workunit.client.0.smithi063.stdout:5/118: getdents d5/d1d 0 2024-04-24T22:45:15.994 INFO:tasks.workunit.client.1.smithi192.stdout:4/33: write d2/fa [3028345,92513] 0 2024-04-24T22:45:15.999 INFO:tasks.workunit.client.1.smithi192.stdout:2/116: rmdir d4/d9 39 2024-04-24T22:45:15.999 INFO:tasks.workunit.client.0.smithi063.stdout:2/100: write d0/d2/d3/d7/fc [2727227,9776] 0 2024-04-24T22:45:16.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/108: link d3/da/dc/c15 d3/da/dc/d1f/c29 0 2024-04-24T22:45:16.001 INFO:tasks.workunit.client.0.smithi063.stdout:1/109: write d3/da/dc/f1c [4019158,60130] 0 2024-04-24T22:45:16.002 INFO:tasks.workunit.client.1.smithi192.stdout:1/58: link d9/fc d9/f13 0 2024-04-24T22:45:16.003 INFO:tasks.workunit.client.0.smithi063.stdout:9/110: write d6/d8/ff [2964545,121775] 0 2024-04-24T22:45:16.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/106: creat d3/f1c x:0 0 0 2024-04-24T22:45:16.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/111: symlink db/d1d/l24 0 2024-04-24T22:45:16.007 INFO:tasks.workunit.client.1.smithi192.stdout:0/107: fdatasync d3/f10 0 2024-04-24T22:45:16.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/112: chown c4 1 1 2024-04-24T22:45:16.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/113: truncate db/f21 878924 0 2024-04-24T22:45:16.010 INFO:tasks.workunit.client.1.smithi192.stdout:1/59: dwrite d9/f12 [0,4194304] 0 2024-04-24T22:45:16.011 INFO:tasks.workunit.client.1.smithi192.stdout:1/60: stat d9/f12 0 2024-04-24T22:45:16.012 INFO:tasks.workunit.client.0.smithi063.stdout:7/96: mknod d0/c27 0 2024-04-24T22:45:16.017 INFO:tasks.workunit.client.0.smithi063.stdout:3/117: creat da/f29 x:0 0 0 2024-04-24T22:45:16.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/34: unlink d2/fa 0 2024-04-24T22:45:16.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/21: rmdir d0/d4 0 2024-04-24T22:45:16.026 INFO:tasks.workunit.client.1.smithi192.stdout:9/48: write f2 [1835353,82957] 0 2024-04-24T22:45:16.027 INFO:tasks.workunit.client.1.smithi192.stdout:2/117: chown d4/d9/d10/c14 2 1 2024-04-24T22:45:16.027 INFO:tasks.workunit.client.0.smithi063.stdout:6/114: dread f3 [0,4194304] 0 2024-04-24T22:45:16.028 INFO:tasks.workunit.client.0.smithi063.stdout:8/138: creat d1/d2/f2f x:0 0 0 2024-04-24T22:45:16.029 INFO:tasks.workunit.client.0.smithi063.stdout:7/97: rmdir d0 39 2024-04-24T22:45:16.029 INFO:tasks.workunit.client.1.smithi192.stdout:5/22: dwrite d0/f1 [0,4194304] 0 2024-04-24T22:45:16.031 INFO:tasks.workunit.client.0.smithi063.stdout:0/116: truncate f5 4284231 0 2024-04-24T22:45:16.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/95: getdents d2 0 2024-04-24T22:45:16.032 INFO:tasks.workunit.client.0.smithi063.stdout:6/115: dread f7 [0,4194304] 0 2024-04-24T22:45:16.032 INFO:tasks.workunit.client.0.smithi063.stdout:0/117: chown d6/da/f14 0 1 2024-04-24T22:45:16.033 INFO:tasks.workunit.client.1.smithi192.stdout:0/108: creat d3/d15/f1d x:0 0 0 2024-04-24T22:45:16.034 INFO:tasks.workunit.client.0.smithi063.stdout:3/118: creat da/d18/f2a x:0 0 0 2024-04-24T22:45:16.036 INFO:tasks.workunit.client.0.smithi063.stdout:3/119: dread - da/d18/f2a zero size 2024-04-24T22:45:16.036 INFO:tasks.workunit.client.1.smithi192.stdout:1/61: creat d9/f14 x:0 0 0 2024-04-24T22:45:16.038 INFO:tasks.workunit.client.0.smithi063.stdout:4/96: dwrite f0 [0,4194304] 0 2024-04-24T22:45:16.039 INFO:tasks.workunit.client.0.smithi063.stdout:4/97: read - d2/d8/f16 zero size 2024-04-24T22:45:16.040 INFO:tasks.workunit.client.0.smithi063.stdout:8/139: symlink d1/d22/l30 0 2024-04-24T22:45:16.042 INFO:tasks.workunit.client.1.smithi192.stdout:6/31: rename d0/c4 to d0/cd 0 2024-04-24T22:45:16.043 INFO:tasks.workunit.client.0.smithi063.stdout:7/98: read d0/fa [4147305,74678] 0 2024-04-24T22:45:16.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/118: rmdir d6/da 39 2024-04-24T22:45:16.055 INFO:tasks.workunit.client.1.smithi192.stdout:2/118: chown d4/d9/d10/c22 121990 1 2024-04-24T22:45:16.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/120: mknod da/c2b 0 2024-04-24T22:45:16.057 INFO:tasks.workunit.client.0.smithi063.stdout:3/121: write da/d18/f19 [692554,26312] 0 2024-04-24T22:45:16.057 INFO:tasks.workunit.client.1.smithi192.stdout:5/23: mkdir d0/d6 0 2024-04-24T22:45:16.059 INFO:tasks.workunit.client.0.smithi063.stdout:4/98: mknod d2/c21 0 2024-04-24T22:45:16.060 INFO:tasks.workunit.client.1.smithi192.stdout:0/109: fdatasync d3/f12 0 2024-04-24T22:45:16.060 INFO:tasks.workunit.client.0.smithi063.stdout:5/119: sync 2024-04-24T22:45:16.061 INFO:tasks.workunit.client.0.smithi063.stdout:8/140: creat d1/d22/f31 x:0 0 0 2024-04-24T22:45:16.062 INFO:tasks.workunit.client.0.smithi063.stdout:8/141: write d1/d2/d17/f27 [2063930,109544] 0 2024-04-24T22:45:16.063 INFO:tasks.workunit.client.1.smithi192.stdout:1/62: mkdir d9/d15 0 2024-04-24T22:45:16.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/63: truncate d9/dd/ff 804591 0 2024-04-24T22:45:16.066 INFO:tasks.workunit.client.1.smithi192.stdout:7/50: getdents d2/db 0 2024-04-24T22:45:16.067 INFO:tasks.workunit.client.1.smithi192.stdout:4/35: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:16.067 INFO:tasks.workunit.client.1.smithi192.stdout:7/51: chown d2/l3 1224568 1 2024-04-24T22:45:16.068 INFO:tasks.workunit.client.1.smithi192.stdout:9/49: write d3/f4 [1499666,51815] 0 2024-04-24T22:45:16.070 INFO:tasks.workunit.client.1.smithi192.stdout:9/50: chown d3/dd/de 1142897 1 2024-04-24T22:45:16.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/110: getdents d3/da/dc/d1f/d21 0 2024-04-24T22:45:16.071 INFO:tasks.workunit.client.1.smithi192.stdout:1/64: dread d9/f12 [0,4194304] 0 2024-04-24T22:45:16.072 INFO:tasks.workunit.client.0.smithi063.stdout:3/122: creat da/f2c x:0 0 0 2024-04-24T22:45:16.072 INFO:tasks.workunit.client.0.smithi063.stdout:3/123: chown da 62308 1 2024-04-24T22:45:16.073 INFO:tasks.workunit.client.0.smithi063.stdout:4/99: mknod d2/d8/c22 0 2024-04-24T22:45:16.076 INFO:tasks.workunit.client.1.smithi192.stdout:7/52: truncate f0 793872 0 2024-04-24T22:45:16.078 INFO:tasks.workunit.client.1.smithi192.stdout:6/32: link d0/d8/fc d0/d8/fe 0 2024-04-24T22:45:16.083 INFO:tasks.workunit.client.1.smithi192.stdout:9/51: mknod d3/dd/de/c10 0 2024-04-24T22:45:16.088 INFO:tasks.workunit.client.1.smithi192.stdout:1/65: creat d9/f16 x:0 0 0 2024-04-24T22:45:16.089 INFO:tasks.workunit.client.0.smithi063.stdout:1/111: mknod d3/da/dc/d1f/d21/c2a 0 2024-04-24T22:45:16.090 INFO:tasks.workunit.client.1.smithi192.stdout:5/24: creat d0/d6/f7 x:0 0 0 2024-04-24T22:45:16.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/112: dread d3/f17 [0,4194304] 0 2024-04-24T22:45:16.092 INFO:tasks.workunit.client.0.smithi063.stdout:4/100: rmdir d2/d19 39 2024-04-24T22:45:16.095 INFO:tasks.workunit.client.0.smithi063.stdout:5/120: dwrite d5/f1f [0,4194304] 0 2024-04-24T22:45:16.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/124: rename da/c22 to da/c2d 0 2024-04-24T22:45:16.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/110: truncate d3/f12 166888 0 2024-04-24T22:45:16.098 INFO:tasks.workunit.client.1.smithi192.stdout:7/53: rmdir d2/db 39 2024-04-24T22:45:16.101 INFO:tasks.workunit.client.0.smithi063.stdout:0/119: mknod d6/da/dc/d26/c2d 0 2024-04-24T22:45:16.104 INFO:tasks.workunit.client.0.smithi063.stdout:7/99: dwrite d0/d4/de/d17/f1e [0,4194304] 0 2024-04-24T22:45:16.106 INFO:tasks.workunit.client.1.smithi192.stdout:9/52: rename d3/lc to d3/dd/l11 0 2024-04-24T22:45:16.106 INFO:tasks.workunit.client.0.smithi063.stdout:7/100: write d0/d4/de/d17/f22 [259233,86187] 0 2024-04-24T22:45:16.107 INFO:tasks.workunit.client.0.smithi063.stdout:1/113: mkdir d3/da/dc/d1f/d2b 0 2024-04-24T22:45:16.108 INFO:tasks.workunit.client.1.smithi192.stdout:2/119: creat d4/f27 x:0 0 0 2024-04-24T22:45:16.109 INFO:tasks.workunit.client.0.smithi063.stdout:4/101: creat d2/d8/f23 x:0 0 0 2024-04-24T22:45:16.110 INFO:tasks.workunit.client.1.smithi192.stdout:1/66: mknod d9/c17 0 2024-04-24T22:45:16.110 INFO:tasks.workunit.client.0.smithi063.stdout:4/102: chown d2/d8/f23 12861550 1 2024-04-24T22:45:16.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/103: fdatasync d2/f1c 0 2024-04-24T22:45:16.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/104: chown f0 745251 1 2024-04-24T22:45:16.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/25: symlink d0/l8 0 2024-04-24T22:45:16.114 INFO:tasks.workunit.client.0.smithi063.stdout:3/125: creat da/f2e x:0 0 0 2024-04-24T22:45:16.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/101: sync 2024-04-24T22:45:16.115 INFO:tasks.workunit.client.0.smithi063.stdout:4/105: dread d2/ff [0,4194304] 0 2024-04-24T22:45:16.116 INFO:tasks.workunit.client.0.smithi063.stdout:2/102: dread - d0/d2/f1b zero size 2024-04-24T22:45:16.116 INFO:tasks.workunit.client.1.smithi192.stdout:0/111: mknod d3/d15/c1e 0 2024-04-24T22:45:16.117 INFO:tasks.workunit.client.0.smithi063.stdout:2/103: fsync d0/d2/d3/d7/f13 0 2024-04-24T22:45:16.118 INFO:tasks.workunit.client.1.smithi192.stdout:9/53: dread f2 [0,4194304] 0 2024-04-24T22:45:16.120 INFO:tasks.workunit.client.0.smithi063.stdout:2/104: dread d0/d2/d3/d7/f19 [0,4194304] 0 2024-04-24T22:45:16.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/126: dwrite da/f14 [0,4194304] 0 2024-04-24T22:45:16.121 INFO:tasks.workunit.client.0.smithi063.stdout:2/105: read d0/d2/d3/d7/fc [425346,25912] 0 2024-04-24T22:45:16.121 INFO:tasks.workunit.client.1.smithi192.stdout:1/67: dwrite d9/f16 [0,4194304] 0 2024-04-24T22:45:16.121 INFO:tasks.workunit.client.1.smithi192.stdout:6/33: mknod d0/cf 0 2024-04-24T22:45:16.121 INFO:tasks.workunit.client.1.smithi192.stdout:4/36: link d2/c5 d2/d9/ce 0 2024-04-24T22:45:16.122 INFO:tasks.workunit.client.0.smithi063.stdout:7/101: mkdir d0/d4/de/d17/d28 0 2024-04-24T22:45:16.122 INFO:tasks.workunit.client.0.smithi063.stdout:3/127: write da/f29 [375978,17547] 0 2024-04-24T22:45:16.123 INFO:tasks.workunit.client.0.smithi063.stdout:3/128: fdatasync da/f1d 0 2024-04-24T22:45:16.123 INFO:tasks.workunit.client.1.smithi192.stdout:4/37: rename d2/db/dc to d2/db/dc/df 22 2024-04-24T22:45:16.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/112: dread d3/f8 [0,4194304] 0 2024-04-24T22:45:16.129 INFO:tasks.workunit.client.0.smithi063.stdout:5/121: mkdir d5/d24/d2c 0 2024-04-24T22:45:16.130 INFO:tasks.workunit.client.0.smithi063.stdout:5/122: chown d5/ff 102 1 2024-04-24T22:45:16.131 INFO:tasks.workunit.client.0.smithi063.stdout:5/123: write f0 [489301,113445] 0 2024-04-24T22:45:16.132 INFO:tasks.workunit.client.0.smithi063.stdout:5/124: read - d5/f14 zero size 2024-04-24T22:45:16.132 INFO:tasks.workunit.client.0.smithi063.stdout:8/142: getdents d1/d22 0 2024-04-24T22:45:16.133 INFO:tasks.workunit.client.0.smithi063.stdout:8/143: write d1/f2c [398806,94879] 0 2024-04-24T22:45:16.134 INFO:tasks.workunit.client.0.smithi063.stdout:8/144: write d1/d22/f31 [181593,106099] 0 2024-04-24T22:45:16.135 INFO:tasks.workunit.client.0.smithi063.stdout:8/145: dread - d1/f1f zero size 2024-04-24T22:45:16.138 INFO:tasks.workunit.client.0.smithi063.stdout:4/106: mknod d2/d8/c24 0 2024-04-24T22:45:16.139 INFO:tasks.workunit.client.0.smithi063.stdout:4/107: write d2/fd [100047,37144] 0 2024-04-24T22:45:16.139 INFO:tasks.workunit.client.1.smithi192.stdout:7/54: mkdir d2/db/d11 0 2024-04-24T22:45:16.141 INFO:tasks.workunit.client.1.smithi192.stdout:7/55: write d2/db/f10 [46971,75060] 0 2024-04-24T22:45:16.143 INFO:tasks.workunit.client.1.smithi192.stdout:6/34: chown d0/cd 2540 1 2024-04-24T22:45:16.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/129: rename da/l26 to da/d18/l2f 0 2024-04-24T22:45:16.149 INFO:tasks.workunit.client.1.smithi192.stdout:0/113: mknod d3/c1f 0 2024-04-24T22:45:16.149 INFO:tasks.workunit.client.0.smithi063.stdout:0/120: write d6/dd/f12 [1296647,80245] 0 2024-04-24T22:45:16.150 INFO:tasks.workunit.client.0.smithi063.stdout:3/130: read - da/d18/f2a zero size 2024-04-24T22:45:16.151 INFO:tasks.workunit.client.0.smithi063.stdout:8/146: mkdir d1/d11/d32 0 2024-04-24T22:45:16.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/120: unlink d4/f5 0 2024-04-24T22:45:16.153 INFO:tasks.workunit.client.0.smithi063.stdout:4/108: dwrite d2/fd [0,4194304] 0 2024-04-24T22:45:16.154 INFO:tasks.workunit.client.0.smithi063.stdout:4/109: chown d2/f13 0 1 2024-04-24T22:45:16.157 INFO:tasks.workunit.client.0.smithi063.stdout:8/147: dread d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:16.158 INFO:tasks.workunit.client.1.smithi192.stdout:7/56: creat d2/db/f12 x:0 0 0 2024-04-24T22:45:16.158 INFO:tasks.workunit.client.0.smithi063.stdout:3/131: dread da/d18/f19 [0,4194304] 0 2024-04-24T22:45:16.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/57: write f1 [576039,49762] 0 2024-04-24T22:45:16.159 INFO:tasks.workunit.client.1.smithi192.stdout:6/35: creat d0/d8/f10 x:0 0 0 2024-04-24T22:45:16.160 INFO:tasks.workunit.client.1.smithi192.stdout:6/36: fdatasync d0/d8/f10 0 2024-04-24T22:45:16.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/68: creat d9/d15/f18 x:0 0 0 2024-04-24T22:45:16.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/69: read d9/fa [635120,27083] 0 2024-04-24T22:45:16.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/148: dread d1/f5 [4194304,4194304] 0 2024-04-24T22:45:16.162 INFO:tasks.workunit.client.1.smithi192.stdout:6/37: chown d0/d8/f10 689318 1 2024-04-24T22:45:16.164 INFO:tasks.workunit.client.1.smithi192.stdout:4/38: unlink l1 0 2024-04-24T22:45:16.165 INFO:tasks.workunit.client.0.smithi063.stdout:8/149: dread d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:16.165 INFO:tasks.workunit.client.1.smithi192.stdout:0/114: creat d3/d15/f20 x:0 0 0 2024-04-24T22:45:16.168 INFO:tasks.workunit.client.1.smithi192.stdout:7/58: dwrite d2/f5 [0,4194304] 0 2024-04-24T22:45:16.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/59: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:16.177 INFO:tasks.workunit.client.1.smithi192.stdout:4/39: mkdir d2/db/d10 0 2024-04-24T22:45:16.187 INFO:tasks.workunit.client.1.smithi192.stdout:5/26: getdents d0 0 2024-04-24T22:45:16.188 INFO:tasks.workunit.client.1.smithi192.stdout:5/27: readlink d0/l2 0 2024-04-24T22:45:16.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/115: truncate d3/f8 4090072 0 2024-04-24T22:45:16.191 INFO:tasks.workunit.client.1.smithi192.stdout:9/54: getdents d3 0 2024-04-24T22:45:16.193 INFO:tasks.workunit.client.1.smithi192.stdout:1/70: link d9/f14 d9/dd/f19 0 2024-04-24T22:45:16.194 INFO:tasks.workunit.client.1.smithi192.stdout:1/71: chown d9/f10 34003394 1 2024-04-24T22:45:16.194 INFO:tasks.workunit.client.1.smithi192.stdout:1/72: fsync f3 0 2024-04-24T22:45:16.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/116: sync 2024-04-24T22:45:16.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/111: sync 2024-04-24T22:45:16.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/117: chown db/c18 0 1 2024-04-24T22:45:16.204 INFO:tasks.workunit.client.1.smithi192.stdout:6/38: truncate d0/d8/fa 2551509 0 2024-04-24T22:45:16.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/60: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:16.204 INFO:tasks.workunit.client.1.smithi192.stdout:1/73: dwrite d9/fa [0,4194304] 0 2024-04-24T22:45:16.205 INFO:tasks.workunit.client.0.smithi063.stdout:6/118: dread - db/d1d/d20/f23 zero size 2024-04-24T22:45:16.207 INFO:tasks.workunit.client.1.smithi192.stdout:2/121: creat d4/f28 x:0 0 0 2024-04-24T22:45:16.207 INFO:tasks.workunit.client.1.smithi192.stdout:2/122: stat f0 0 2024-04-24T22:45:16.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/28: rmdir d0 39 2024-04-24T22:45:16.215 INFO:tasks.workunit.client.1.smithi192.stdout:9/55: creat d3/dd/de/f12 x:0 0 0 2024-04-24T22:45:16.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/40: creat d2/db/d10/f11 x:0 0 0 2024-04-24T22:45:16.223 INFO:tasks.workunit.client.0.smithi063.stdout:2/106: sync 2024-04-24T22:45:16.235 INFO:tasks.workunit.client.1.smithi192.stdout:1/74: fsync d9/fa 0 2024-04-24T22:45:16.237 INFO:tasks.workunit.client.1.smithi192.stdout:5/29: dread - d0/d6/f7 zero size 2024-04-24T22:45:16.239 INFO:tasks.workunit.client.1.smithi192.stdout:5/30: write d0/d6/f7 [186581,18200] 0 2024-04-24T22:45:16.240 INFO:tasks.workunit.client.1.smithi192.stdout:0/116: write d3/f14 [449245,76080] 0 2024-04-24T22:45:16.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/117: chown d3/f14 2 1 2024-04-24T22:45:16.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/118: rmdir d3 39 2024-04-24T22:45:16.261 INFO:tasks.workunit.client.1.smithi192.stdout:3/45: sync 2024-04-24T22:45:16.262 INFO:tasks.workunit.client.1.smithi192.stdout:8/25: sync 2024-04-24T22:45:16.263 INFO:tasks.workunit.client.1.smithi192.stdout:3/46: dread - d2/d4/d8/fa zero size 2024-04-24T22:45:16.264 INFO:tasks.workunit.client.1.smithi192.stdout:7/61: rmdir d2/db/d11 0 2024-04-24T22:45:16.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/123: rename d4/d9/d10/d1c to d4/d29 0 2024-04-24T22:45:16.270 INFO:tasks.workunit.client.1.smithi192.stdout:3/47: dwrite d2/d4/d8/fa [0,4194304] 0 2024-04-24T22:45:16.270 INFO:tasks.workunit.client.1.smithi192.stdout:1/75: write d9/dd/f19 [324142,52595] 0 2024-04-24T22:45:16.271 INFO:tasks.workunit.client.1.smithi192.stdout:1/76: readlink d9/dd/le 0 2024-04-24T22:45:16.276 INFO:tasks.workunit.client.1.smithi192.stdout:9/56: truncate d3/f9 3349115 0 2024-04-24T22:45:16.280 INFO:tasks.workunit.client.1.smithi192.stdout:1/77: dwrite d9/f10 [0,4194304] 0 2024-04-24T22:45:16.286 INFO:tasks.workunit.client.1.smithi192.stdout:0/119: stat d3/cf 0 2024-04-24T22:45:16.286 INFO:tasks.workunit.client.1.smithi192.stdout:6/39: link d0/l6 d0/l11 0 2024-04-24T22:45:16.289 INFO:tasks.workunit.client.1.smithi192.stdout:8/26: unlink f4 0 2024-04-24T22:45:16.290 INFO:tasks.workunit.client.1.smithi192.stdout:8/27: fsync f3 0 2024-04-24T22:45:16.290 INFO:tasks.workunit.client.1.smithi192.stdout:7/62: creat d2/db/f13 x:0 0 0 2024-04-24T22:45:16.292 INFO:tasks.workunit.client.1.smithi192.stdout:0/120: dwrite d3/f10 [0,4194304] 0 2024-04-24T22:45:16.294 INFO:tasks.workunit.client.1.smithi192.stdout:5/31: rename d0/l2 to d0/d6/l9 0 2024-04-24T22:45:16.296 INFO:tasks.workunit.client.1.smithi192.stdout:5/32: write d0/f1 [1749542,99721] 0 2024-04-24T22:45:16.297 INFO:tasks.workunit.client.1.smithi192.stdout:5/33: rename d0/d6 to d0/d6/da 22 2024-04-24T22:45:16.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/121: dwrite f2 [4194304,4194304] 0 2024-04-24T22:45:16.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/122: truncate d3/d15/f1d 492287 0 2024-04-24T22:45:16.306 INFO:tasks.workunit.client.1.smithi192.stdout:0/123: dread - d3/d15/f20 zero size 2024-04-24T22:45:16.307 INFO:tasks.workunit.client.1.smithi192.stdout:4/41: symlink d2/db/l12 0 2024-04-24T22:45:16.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/78: rename d9/f10 to d9/f1a 0 2024-04-24T22:45:16.316 INFO:tasks.workunit.client.1.smithi192.stdout:6/40: unlink d0/l7 0 2024-04-24T22:45:16.323 INFO:tasks.workunit.client.1.smithi192.stdout:8/28: creat f5 x:0 0 0 2024-04-24T22:45:16.323 INFO:tasks.workunit.client.1.smithi192.stdout:8/29: truncate f5 413251 0 2024-04-24T22:45:16.324 INFO:tasks.workunit.client.1.smithi192.stdout:7/63: rename d2/db/f12 to d2/db/f14 0 2024-04-24T22:45:16.327 INFO:tasks.workunit.client.1.smithi192.stdout:8/30: dread f3 [0,4194304] 0 2024-04-24T22:45:16.330 INFO:tasks.workunit.client.1.smithi192.stdout:5/34: creat d0/fb x:0 0 0 2024-04-24T22:45:16.334 INFO:tasks.workunit.client.1.smithi192.stdout:9/57: unlink d3/dd/l11 0 2024-04-24T22:45:16.336 INFO:tasks.workunit.client.1.smithi192.stdout:1/79: creat d9/d15/f1b x:0 0 0 2024-04-24T22:45:16.337 INFO:tasks.workunit.client.1.smithi192.stdout:6/41: fdatasync d0/d8/fc 0 2024-04-24T22:45:16.340 INFO:tasks.workunit.client.1.smithi192.stdout:7/64: creat d2/db/f15 x:0 0 0 2024-04-24T22:45:16.341 INFO:tasks.workunit.client.1.smithi192.stdout:7/65: readlink d2/db/le 0 2024-04-24T22:45:16.346 INFO:tasks.workunit.client.1.smithi192.stdout:8/31: unlink f1 0 2024-04-24T22:45:16.348 INFO:tasks.workunit.client.1.smithi192.stdout:7/66: dwrite f1 [0,4194304] 0 2024-04-24T22:45:16.350 INFO:tasks.workunit.client.1.smithi192.stdout:7/67: write d2/f6 [3845816,56377] 0 2024-04-24T22:45:16.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/35: creat d0/fc x:0 0 0 2024-04-24T22:45:16.360 INFO:tasks.workunit.client.1.smithi192.stdout:5/36: dwrite d0/fb [0,4194304] 0 2024-04-24T22:45:16.361 INFO:tasks.workunit.client.1.smithi192.stdout:1/80: mkdir d9/d1c 0 2024-04-24T22:45:16.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/125: rename d5/d11/d18/d2a to d5/d11/d23/d2d 0 2024-04-24T22:45:16.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/42: rename d0/l6 to d0/d8/l12 0 2024-04-24T22:45:16.370 INFO:tasks.workunit.client.1.smithi192.stdout:6/43: truncate d0/d8/f10 537600 0 2024-04-24T22:45:16.373 INFO:tasks.workunit.client.1.smithi192.stdout:2/124: getdents d4/d29 0 2024-04-24T22:45:16.374 INFO:tasks.workunit.client.1.smithi192.stdout:2/125: truncate d4/f27 422882 0 2024-04-24T22:45:16.375 INFO:tasks.workunit.client.1.smithi192.stdout:2/126: chown f0 578164 1 2024-04-24T22:45:16.375 INFO:tasks.workunit.client.1.smithi192.stdout:2/127: fdatasync d4/d9/d10/f1e 0 2024-04-24T22:45:16.376 INFO:tasks.workunit.client.0.smithi063.stdout:3/132: creat da/d18/f30 x:0 0 0 2024-04-24T22:45:16.380 INFO:tasks.workunit.client.1.smithi192.stdout:8/32: creat f6 x:0 0 0 2024-04-24T22:45:16.382 INFO:tasks.workunit.client.0.smithi063.stdout:8/150: mkdir d1/d22/d33 0 2024-04-24T22:45:16.382 INFO:tasks.workunit.client.1.smithi192.stdout:2/128: dread d4/d29/f1d [0,4194304] 0 2024-04-24T22:45:16.384 INFO:tasks.workunit.client.1.smithi192.stdout:3/48: getdents d2/d4/d8 0 2024-04-24T22:45:16.387 INFO:tasks.workunit.client.1.smithi192.stdout:0/124: link d3/l6 d3/d15/l21 0 2024-04-24T22:45:16.388 INFO:tasks.workunit.client.1.smithi192.stdout:0/125: truncate d3/d15/f1d 1075539 0 2024-04-24T22:45:16.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/107: readlink d0/d2/d3/d7/l18 0 2024-04-24T22:45:16.390 INFO:tasks.workunit.client.0.smithi063.stdout:7/102: link d0/d4/c1d d0/c29 0 2024-04-24T22:45:16.391 INFO:tasks.workunit.client.0.smithi063.stdout:6/119: dwrite db/ff [0,4194304] 0 2024-04-24T22:45:16.392 INFO:tasks.workunit.client.0.smithi063.stdout:7/103: chown d0/d4/de/d17/l1f 0 1 2024-04-24T22:45:16.393 INFO:tasks.workunit.client.0.smithi063.stdout:6/120: dread - db/d1d/d20/f23 zero size 2024-04-24T22:45:16.394 INFO:tasks.workunit.client.0.smithi063.stdout:1/114: getdents d3/da/dc/d1f 0 2024-04-24T22:45:16.396 INFO:tasks.workunit.client.0.smithi063.stdout:0/121: creat d6/dd/d20/d24/f2e x:0 0 0 2024-04-24T22:45:16.397 INFO:tasks.workunit.client.1.smithi192.stdout:1/81: mknod d9/dd/c1d 0 2024-04-24T22:45:16.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/58: write d3/f8 [832755,21003] 0 2024-04-24T22:45:16.397 INFO:tasks.workunit.client.0.smithi063.stdout:0/122: dread - d6/da/f29 zero size 2024-04-24T22:45:16.398 INFO:tasks.workunit.client.1.smithi192.stdout:1/82: fdatasync d9/dd/ff 0 2024-04-24T22:45:16.398 INFO:tasks.workunit.client.0.smithi063.stdout:0/123: dread - d6/da/dc/f17 zero size 2024-04-24T22:45:16.398 INFO:tasks.workunit.client.1.smithi192.stdout:9/59: chown d3/dd/de/c10 53 1 2024-04-24T22:45:16.399 INFO:tasks.workunit.client.1.smithi192.stdout:1/83: truncate d9/fa 4675966 0 2024-04-24T22:45:16.400 INFO:tasks.workunit.client.0.smithi063.stdout:0/124: write d6/dd/d20/d24/f2e [828829,30998] 0 2024-04-24T22:45:16.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/60: fdatasync d3/f8 0 2024-04-24T22:45:16.401 INFO:tasks.workunit.client.0.smithi063.stdout:0/125: rename d6/dd/d20 to d6/dd/d20/d21/d25/d2f 22 2024-04-24T22:45:16.403 INFO:tasks.workunit.client.0.smithi063.stdout:4/110: creat d2/d19/f25 x:0 0 0 2024-04-24T22:45:16.407 INFO:tasks.workunit.client.0.smithi063.stdout:3/133: stat da/c2d 0 2024-04-24T22:45:16.408 INFO:tasks.workunit.client.0.smithi063.stdout:3/134: chown da/d18 692111 1 2024-04-24T22:45:16.412 INFO:tasks.workunit.client.1.smithi192.stdout:9/61: dwrite d3/fa [0,4194304] 0 2024-04-24T22:45:16.412 INFO:tasks.workunit.client.1.smithi192.stdout:9/62: dread - d3/dd/de/f12 zero size 2024-04-24T22:45:16.412 INFO:tasks.workunit.client.1.smithi192.stdout:8/33: unlink l2 0 2024-04-24T22:45:16.416 INFO:tasks.workunit.client.0.smithi063.stdout:1/115: dread d3/f7 [0,4194304] 0 2024-04-24T22:45:16.420 INFO:tasks.workunit.client.1.smithi192.stdout:8/34: dwrite f6 [0,4194304] 0 2024-04-24T22:45:16.421 INFO:tasks.workunit.client.0.smithi063.stdout:1/116: dwrite d3/f28 [4194304,4194304] 0 2024-04-24T22:45:16.422 INFO:tasks.workunit.client.0.smithi063.stdout:1/117: read d3/f28 [3399761,50993] 0 2024-04-24T22:45:16.423 INFO:tasks.workunit.client.1.smithi192.stdout:0/126: symlink d3/d15/l22 0 2024-04-24T22:45:16.423 INFO:tasks.workunit.client.0.smithi063.stdout:7/104: unlink d0/d4/de/l15 0 2024-04-24T22:45:16.424 INFO:tasks.workunit.client.0.smithi063.stdout:1/118: chown d3/l8 1 1 2024-04-24T22:45:16.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/121: creat db/f25 x:0 0 0 2024-04-24T22:45:16.426 INFO:tasks.workunit.client.1.smithi192.stdout:4/42: link d2/db/l12 d2/db/l13 0 2024-04-24T22:45:16.429 INFO:tasks.workunit.client.1.smithi192.stdout:5/37: link d0/fb d0/d6/fd 0 2024-04-24T22:45:16.432 INFO:tasks.workunit.client.0.smithi063.stdout:9/112: dwrite d6/f1b [0,4194304] 0 2024-04-24T22:45:16.434 INFO:tasks.workunit.client.0.smithi063.stdout:3/135: creat da/f31 x:0 0 0 2024-04-24T22:45:16.435 INFO:tasks.workunit.client.1.smithi192.stdout:1/84: mknod d9/dd/c1e 0 2024-04-24T22:45:16.436 INFO:tasks.workunit.client.1.smithi192.stdout:2/129: write d4/d9/fb [4273112,94395] 0 2024-04-24T22:45:16.439 INFO:tasks.workunit.client.0.smithi063.stdout:9/113: dwrite d6/d8/d24/f1e [4194304,4194304] 0 2024-04-24T22:45:16.439 INFO:tasks.workunit.client.1.smithi192.stdout:6/44: fdatasync d0/d8/fa 0 2024-04-24T22:45:16.439 INFO:tasks.workunit.client.0.smithi063.stdout:3/136: dread da/f14 [0,4194304] 0 2024-04-24T22:45:16.440 INFO:tasks.workunit.client.0.smithi063.stdout:3/137: fdatasync f7 0 2024-04-24T22:45:16.441 INFO:tasks.workunit.client.1.smithi192.stdout:6/45: read d0/d8/f10 [473042,87110] 0 2024-04-24T22:45:16.454 INFO:tasks.workunit.client.1.smithi192.stdout:9/63: mknod d3/c13 0 2024-04-24T22:45:16.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/105: mknod d0/d4/de/d17/c2a 0 2024-04-24T22:45:16.464 INFO:tasks.workunit.client.1.smithi192.stdout:1/85: creat d9/d15/f1f x:0 0 0 2024-04-24T22:45:16.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/108: rmdir d0/d2/d3/db 0 2024-04-24T22:45:16.468 INFO:tasks.workunit.client.0.smithi063.stdout:9/114: dwrite d6/fd [0,4194304] 0 2024-04-24T22:45:16.470 INFO:tasks.workunit.client.1.smithi192.stdout:5/38: dwrite d0/d6/fd [0,4194304] 0 2024-04-24T22:45:16.471 INFO:tasks.workunit.client.1.smithi192.stdout:6/46: symlink d0/d8/l13 0 2024-04-24T22:45:16.472 INFO:tasks.workunit.client.0.smithi063.stdout:5/126: sync 2024-04-24T22:45:16.472 INFO:tasks.workunit.client.0.smithi063.stdout:5/127: chown d5/le 15297 1 2024-04-24T22:45:16.475 INFO:tasks.workunit.client.0.smithi063.stdout:0/126: link d6/da/dc/c2a d6/dd/c30 0 2024-04-24T22:45:16.478 INFO:tasks.workunit.client.1.smithi192.stdout:8/35: link f5 f7 0 2024-04-24T22:45:16.481 INFO:tasks.workunit.client.0.smithi063.stdout:4/111: creat d2/f26 x:0 0 0 2024-04-24T22:45:16.482 INFO:tasks.workunit.client.1.smithi192.stdout:7/68: getdents d2/db 0 2024-04-24T22:45:16.484 INFO:tasks.workunit.client.1.smithi192.stdout:7/69: fdatasync d2/db/f13 0 2024-04-24T22:45:16.484 INFO:tasks.workunit.client.1.smithi192.stdout:8/36: dread f6 [0,4194304] 0 2024-04-24T22:45:16.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/127: dwrite d6/dd/d20/d24/f2e [0,4194304] 0 2024-04-24T22:45:16.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/128: stat d6/dd/f12 0 2024-04-24T22:45:16.485 INFO:tasks.workunit.client.1.smithi192.stdout:0/127: rename d3/f12 to d3/f23 0 2024-04-24T22:45:16.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/151: link d1/d2/d3/ca d1/d22/d33/c34 0 2024-04-24T22:45:16.490 INFO:tasks.workunit.client.1.smithi192.stdout:3/49: truncate d2/f3 2156420 0 2024-04-24T22:45:16.493 INFO:tasks.workunit.client.1.smithi192.stdout:8/37: dwrite f6 [0,4194304] 0 2024-04-24T22:45:16.494 INFO:tasks.workunit.client.0.smithi063.stdout:2/109: creat d0/d2/f1d x:0 0 0 2024-04-24T22:45:16.498 INFO:tasks.workunit.client.0.smithi063.stdout:7/106: mknod d0/d4/de/d17/d28/c2b 0 2024-04-24T22:45:16.500 INFO:tasks.workunit.client.1.smithi192.stdout:6/47: write d0/d8/fa [422667,45829] 0 2024-04-24T22:45:16.502 INFO:tasks.workunit.client.0.smithi063.stdout:6/122: link db/d1c/c22 db/d1d/d20/c26 0 2024-04-24T22:45:16.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/119: sync 2024-04-24T22:45:16.510 INFO:tasks.workunit.client.0.smithi063.stdout:4/112: mknod d2/d8/c27 0 2024-04-24T22:45:16.511 INFO:tasks.workunit.client.0.smithi063.stdout:1/120: dread d3/fd [0,4194304] 0 2024-04-24T22:45:16.514 INFO:tasks.workunit.client.0.smithi063.stdout:3/138: truncate da/f14 1281618 0 2024-04-24T22:45:16.516 INFO:tasks.workunit.client.1.smithi192.stdout:0/128: creat d3/d15/f24 x:0 0 0 2024-04-24T22:45:16.518 INFO:tasks.workunit.client.0.smithi063.stdout:8/152: truncate d1/d11/f15 1106572 0 2024-04-24T22:45:16.519 INFO:tasks.workunit.client.0.smithi063.stdout:3/139: dwrite da/d18/f27 [0,4194304] 0 2024-04-24T22:45:16.520 INFO:tasks.workunit.client.0.smithi063.stdout:8/153: truncate d1/d2/f2f 26764 0 2024-04-24T22:45:16.520 INFO:tasks.workunit.client.1.smithi192.stdout:4/43: rename d2/l7 to d2/d9/l14 0 2024-04-24T22:45:16.520 INFO:tasks.workunit.client.0.smithi063.stdout:8/154: chown d1/d22/f2e 7414 1 2024-04-24T22:45:16.523 INFO:tasks.workunit.client.0.smithi063.stdout:2/110: rename d0/d2/f15 to d0/d2/f1e 0 2024-04-24T22:45:16.524 INFO:tasks.workunit.client.1.smithi192.stdout:2/130: creat d4/f2a x:0 0 0 2024-04-24T22:45:16.527 INFO:tasks.workunit.client.0.smithi063.stdout:9/115: creat d6/d8/d1f/d20/d22/f25 x:0 0 0 2024-04-24T22:45:16.530 INFO:tasks.workunit.client.1.smithi192.stdout:4/44: dwrite d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:16.530 INFO:tasks.workunit.client.1.smithi192.stdout:3/50: creat d2/d4/d7/fd x:0 0 0 2024-04-24T22:45:16.531 INFO:tasks.workunit.client.0.smithi063.stdout:7/107: mknod d0/d4/de/c2c 0 2024-04-24T22:45:16.532 INFO:tasks.workunit.client.1.smithi192.stdout:8/38: creat f8 x:0 0 0 2024-04-24T22:45:16.533 INFO:tasks.workunit.client.0.smithi063.stdout:9/116: dwrite d6/fd [4194304,4194304] 0 2024-04-24T22:45:16.534 INFO:tasks.workunit.client.1.smithi192.stdout:1/86: mknod d9/d1c/c20 0 2024-04-24T22:45:16.535 INFO:tasks.workunit.client.0.smithi063.stdout:0/129: write d6/da/f10 [1131900,43868] 0 2024-04-24T22:45:16.535 INFO:tasks.workunit.client.0.smithi063.stdout:7/108: dread d0/d4/de/d17/f22 [0,4194304] 0 2024-04-24T22:45:16.537 INFO:tasks.workunit.client.0.smithi063.stdout:6/123: symlink db/d1d/d20/l27 0 2024-04-24T22:45:16.542 INFO:tasks.workunit.client.1.smithi192.stdout:6/48: rename d0/d8/f10 to d0/d8/f14 0 2024-04-24T22:45:16.543 INFO:tasks.workunit.client.0.smithi063.stdout:9/117: dread d6/d8/d24/f1e [4194304,4194304] 0 2024-04-24T22:45:16.546 INFO:tasks.workunit.client.0.smithi063.stdout:7/109: dwrite d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:16.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/110: dread - d0/d4/de/f25 zero size 2024-04-24T22:45:16.552 INFO:tasks.workunit.client.0.smithi063.stdout:9/118: dread d6/d8/d24/f1e [0,4194304] 0 2024-04-24T22:45:16.552 INFO:tasks.workunit.client.0.smithi063.stdout:3/140: fsync da/f28 0 2024-04-24T22:45:16.555 INFO:tasks.workunit.client.0.smithi063.stdout:8/155: rmdir d1/d22/d33 39 2024-04-24T22:45:16.555 INFO:tasks.workunit.client.0.smithi063.stdout:8/156: dread - d1/d22/f2e zero size 2024-04-24T22:45:16.558 INFO:tasks.workunit.client.0.smithi063.stdout:2/111: mknod d0/dd/c1f 0 2024-04-24T22:45:16.558 INFO:tasks.workunit.client.1.smithi192.stdout:0/129: mkdir d3/d25 0 2024-04-24T22:45:16.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/157: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:16.561 INFO:tasks.workunit.client.1.smithi192.stdout:5/39: truncate d0/f1 2821714 0 2024-04-24T22:45:16.563 INFO:tasks.workunit.client.0.smithi063.stdout:5/128: link d5/la d5/d11/d18/l2e 0 2024-04-24T22:45:16.564 INFO:tasks.workunit.client.1.smithi192.stdout:2/131: truncate d4/d29/f1d 1652104 0 2024-04-24T22:45:16.566 INFO:tasks.workunit.client.0.smithi063.stdout:5/129: dread d5/f9 [0,4194304] 0 2024-04-24T22:45:16.568 INFO:tasks.workunit.client.1.smithi192.stdout:4/45: mknod d2/db/d10/c15 0 2024-04-24T22:45:16.572 INFO:tasks.workunit.client.1.smithi192.stdout:2/132: dwrite d4/d9/d10/f11 [0,4194304] 0 2024-04-24T22:45:16.573 INFO:tasks.workunit.client.0.smithi063.stdout:6/124: rename db/f21 to db/d1d/d20/f28 0 2024-04-24T22:45:16.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/130: dwrite d5/fd [0,4194304] 0 2024-04-24T22:45:16.575 INFO:tasks.workunit.client.1.smithi192.stdout:2/133: read d4/d9/d10/f1e [602862,82414] 0 2024-04-24T22:45:16.575 INFO:tasks.workunit.client.0.smithi063.stdout:6/125: write db/d1d/d20/f23 [698346,70304] 0 2024-04-24T22:45:16.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/131: fsync d5/d11/f15 0 2024-04-24T22:45:16.579 INFO:tasks.workunit.client.1.smithi192.stdout:1/87: write d9/f12 [4909211,91522] 0 2024-04-24T22:45:16.583 INFO:tasks.workunit.client.1.smithi192.stdout:6/49: chown d0/d8/l12 309246919 1 2024-04-24T22:45:16.583 INFO:tasks.workunit.client.1.smithi192.stdout:2/134: dread d4/d9/d10/f1e [0,4194304] 0 2024-04-24T22:45:16.584 INFO:tasks.workunit.client.1.smithi192.stdout:2/135: readlink d4/d9/d10/l25 0 2024-04-24T22:45:16.584 INFO:tasks.workunit.client.0.smithi063.stdout:5/132: dwrite d5/d11/f15 [0,4194304] 0 2024-04-24T22:45:16.585 INFO:tasks.workunit.client.1.smithi192.stdout:2/136: dread - d4/f2a zero size 2024-04-24T22:45:16.585 INFO:tasks.workunit.client.0.smithi063.stdout:5/133: readlink l2 0 2024-04-24T22:45:16.585 INFO:tasks.workunit.client.0.smithi063.stdout:7/111: symlink d0/d4/de/d17/d28/l2d 0 2024-04-24T22:45:16.586 INFO:tasks.workunit.client.0.smithi063.stdout:7/112: fsync d0/d4/de/f25 0 2024-04-24T22:45:16.587 INFO:tasks.workunit.client.1.smithi192.stdout:9/64: getdents d3 0 2024-04-24T22:45:16.588 INFO:tasks.workunit.client.0.smithi063.stdout:9/119: symlink d6/d8/l26 0 2024-04-24T22:45:16.590 INFO:tasks.workunit.client.1.smithi192.stdout:7/70: creat d2/f16 x:0 0 0 2024-04-24T22:45:16.590 INFO:tasks.workunit.client.0.smithi063.stdout:9/120: chown d6/d8/db/c1d 4243178 1 2024-04-24T22:45:16.594 INFO:tasks.workunit.client.1.smithi192.stdout:5/40: rename d0/d6/f7 to d0/fe 0 2024-04-24T22:45:16.599 INFO:tasks.workunit.client.1.smithi192.stdout:7/71: dwrite d2/db/f13 [0,4194304] 0 2024-04-24T22:45:16.604 INFO:tasks.workunit.client.1.smithi192.stdout:1/88: unlink d9/fc 0 2024-04-24T22:45:16.609 INFO:tasks.workunit.client.1.smithi192.stdout:2/137: unlink d4/d29/f20 0 2024-04-24T22:45:16.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/130: mkdir d6/dd/d20/d21/d25/d31 0 2024-04-24T22:45:16.610 INFO:tasks.workunit.client.0.smithi063.stdout:6/126: mkdir db/d1d/d20/d29 0 2024-04-24T22:45:16.612 INFO:tasks.workunit.client.1.smithi192.stdout:0/130: truncate d3/f8 3567065 0 2024-04-24T22:45:16.612 INFO:tasks.workunit.client.0.smithi063.stdout:4/113: link d2/d8/c24 d2/d8/c28 0 2024-04-24T22:45:16.612 INFO:tasks.workunit.client.0.smithi063.stdout:0/131: dwrite d6/da/f14 [4194304,4194304] 0 2024-04-24T22:45:16.612 INFO:tasks.workunit.client.0.smithi063.stdout:5/134: creat d5/d11/f2f x:0 0 0 2024-04-24T22:45:16.617 INFO:tasks.workunit.client.0.smithi063.stdout:2/112: symlink d0/d2/d3/d1c/l20 0 2024-04-24T22:45:16.618 INFO:tasks.workunit.client.0.smithi063.stdout:2/113: rename d0 to d0/dd/d21 22 2024-04-24T22:45:16.618 INFO:tasks.workunit.client.0.smithi063.stdout:2/114: chown d0/d2/d3/d7/c16 1123 1 2024-04-24T22:45:16.623 INFO:tasks.workunit.client.1.smithi192.stdout:2/138: fdatasync d4/f13 0 2024-04-24T22:45:16.627 INFO:tasks.workunit.client.1.smithi192.stdout:3/51: truncate d2/d4/d8/fa 2673602 0 2024-04-24T22:45:16.628 INFO:tasks.workunit.client.0.smithi063.stdout:0/132: mknod d6/dd/d20/d21/d25/c32 0 2024-04-24T22:45:16.629 INFO:tasks.workunit.client.0.smithi063.stdout:0/133: write d6/da/f14 [2778571,96403] 0 2024-04-24T22:45:16.630 INFO:tasks.workunit.client.1.smithi192.stdout:8/39: truncate f6 145055 0 2024-04-24T22:45:16.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/134: chown d6/da/lf 10874 1 2024-04-24T22:45:16.630 INFO:tasks.workunit.client.1.smithi192.stdout:8/40: write f8 [674527,27717] 0 2024-04-24T22:45:16.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/121: sync 2024-04-24T22:45:16.631 INFO:tasks.workunit.client.1.smithi192.stdout:8/41: chown f7 1791914 1 2024-04-24T22:45:16.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/122: chown d3/da/dc/d11 1015271 1 2024-04-24T22:45:16.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/131: rename f2 to d3/d15/f26 0 2024-04-24T22:45:16.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/123: readlink d3/da/dc/l10 0 2024-04-24T22:45:16.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/124: readlink d3/l8 0 2024-04-24T22:45:16.633 INFO:tasks.workunit.client.0.smithi063.stdout:1/125: dread - d3/f23 zero size 2024-04-24T22:45:16.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/135: mkdir d5/d30 0 2024-04-24T22:45:16.634 INFO:tasks.workunit.client.0.smithi063.stdout:5/136: stat d5/d11/d18/c1e 0 2024-04-24T22:45:16.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/137: write d5/d11/f1a [3209692,33474] 0 2024-04-24T22:45:16.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/113: mkdir d0/d26/d2e 0 2024-04-24T22:45:16.637 INFO:tasks.workunit.client.1.smithi192.stdout:7/72: getdents d2/db 0 2024-04-24T22:45:16.639 INFO:tasks.workunit.client.0.smithi063.stdout:9/121: mknod d6/c27 0 2024-04-24T22:45:16.643 INFO:tasks.workunit.client.0.smithi063.stdout:2/115: fdatasync d0/d2/f1e 0 2024-04-24T22:45:16.644 INFO:tasks.workunit.client.1.smithi192.stdout:2/139: rmdir d4/d9/d10 39 2024-04-24T22:45:16.645 INFO:tasks.workunit.client.0.smithi063.stdout:9/122: dwrite d6/d8/d1f/d20/f21 [0,4194304] 0 2024-04-24T22:45:16.647 INFO:tasks.workunit.client.1.smithi192.stdout:3/52: creat d2/fe x:0 0 0 2024-04-24T22:45:16.650 INFO:tasks.workunit.client.1.smithi192.stdout:9/65: truncate d3/fa 3681895 0 2024-04-24T22:45:16.655 INFO:tasks.workunit.client.1.smithi192.stdout:0/132: mkdir d3/d27 0 2024-04-24T22:45:16.656 INFO:tasks.workunit.client.0.smithi063.stdout:0/135: mkdir d6/da/dc/d26/d33 0 2024-04-24T22:45:16.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/46: rename d2/cd to d2/c16 0 2024-04-24T22:45:16.662 INFO:tasks.workunit.client.0.smithi063.stdout:1/126: unlink d3/da/dc/c12 0 2024-04-24T22:45:16.664 INFO:tasks.workunit.client.0.smithi063.stdout:3/141: sync 2024-04-24T22:45:16.671 INFO:tasks.workunit.client.1.smithi192.stdout:1/89: truncate d9/f1a 2590828 0 2024-04-24T22:45:16.672 INFO:tasks.workunit.client.1.smithi192.stdout:2/140: creat d4/d29/f2b x:0 0 0 2024-04-24T22:45:16.673 INFO:tasks.workunit.client.0.smithi063.stdout:6/127: truncate db/ff 3867664 0 2024-04-24T22:45:16.675 INFO:tasks.workunit.client.1.smithi192.stdout:3/53: rename d2/fe to d2/d4/d8/ff 0 2024-04-24T22:45:16.675 INFO:tasks.workunit.client.1.smithi192.stdout:9/66: rename d3/dd to d3/dd/d14 22 2024-04-24T22:45:16.677 INFO:tasks.workunit.client.0.smithi063.stdout:8/158: getdents d1/d21 0 2024-04-24T22:45:16.682 INFO:tasks.workunit.client.0.smithi063.stdout:2/116: creat d0/dd/f22 x:0 0 0 2024-04-24T22:45:16.682 INFO:tasks.workunit.client.0.smithi063.stdout:2/117: chown d0/d2/d3/d7/c16 600 1 2024-04-24T22:45:16.683 INFO:tasks.workunit.client.0.smithi063.stdout:2/118: dread - d0/d2/d3/d7/f13 zero size 2024-04-24T22:45:16.683 INFO:tasks.workunit.client.0.smithi063.stdout:2/119: write d0/d2/f1b [833896,71957] 0 2024-04-24T22:45:16.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/114: creat d2/f29 x:0 0 0 2024-04-24T22:45:16.692 INFO:tasks.workunit.client.0.smithi063.stdout:2/120: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:16.692 INFO:tasks.workunit.client.0.smithi063.stdout:3/142: symlink da/l32 0 2024-04-24T22:45:16.696 INFO:tasks.workunit.client.1.smithi192.stdout:5/41: dwrite d0/f1 [0,4194304] 0 2024-04-24T22:45:16.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/138: rename d5/c2b to d5/d11/d23/d2d/c31 0 2024-04-24T22:45:16.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/139: dwrite d5/f14 [0,4194304] 0 2024-04-24T22:45:16.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/140: chown d5/fd 3275 1 2024-04-24T22:45:16.706 INFO:tasks.workunit.client.1.smithi192.stdout:9/67: dread f2 [0,4194304] 0 2024-04-24T22:45:16.707 INFO:tasks.workunit.client.1.smithi192.stdout:0/133: rename d3/d15 to d3/d27/d28 0 2024-04-24T22:45:16.709 INFO:tasks.workunit.client.0.smithi063.stdout:6/128: creat db/f2a x:0 0 0 2024-04-24T22:45:16.709 INFO:tasks.workunit.client.0.smithi063.stdout:6/129: chown db/f2a 0 1 2024-04-24T22:45:16.710 INFO:tasks.workunit.client.1.smithi192.stdout:7/73: truncate d2/f5 1806850 0 2024-04-24T22:45:16.720 INFO:tasks.workunit.client.0.smithi063.stdout:0/136: symlink d6/da/d2b/l34 0 2024-04-24T22:45:16.721 INFO:tasks.workunit.client.1.smithi192.stdout:1/90: chown d9/dd/c1d 165 1 2024-04-24T22:45:16.724 INFO:tasks.workunit.client.0.smithi063.stdout:7/114: truncate d0/d4/de/d17/f1e 3587088 0 2024-04-24T22:45:16.724 INFO:tasks.workunit.client.1.smithi192.stdout:2/141: write d4/d9/d10/f11 [1649338,66822] 0 2024-04-24T22:45:16.725 INFO:tasks.workunit.client.0.smithi063.stdout:2/121: symlink d0/l23 0 2024-04-24T22:45:16.725 INFO:tasks.workunit.client.1.smithi192.stdout:2/142: truncate d4/d9/d10/f1e 1168421 0 2024-04-24T22:45:16.726 INFO:tasks.workunit.client.0.smithi063.stdout:2/122: write d0/d2/f1e [97088,78950] 0 2024-04-24T22:45:16.727 INFO:tasks.workunit.client.0.smithi063.stdout:2/123: readlink d0/d2/l6 0 2024-04-24T22:45:16.727 INFO:tasks.workunit.client.0.smithi063.stdout:2/124: fsync d0/d2/f1b 0 2024-04-24T22:45:16.734 INFO:tasks.workunit.client.1.smithi192.stdout:2/143: dwrite d4/f28 [0,4194304] 0 2024-04-24T22:45:16.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/115: write d2/f13 [1044126,37683] 0 2024-04-24T22:45:16.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/116: chown d2/d8/c27 206 1 2024-04-24T22:45:16.739 INFO:tasks.workunit.client.0.smithi063.stdout:4/117: truncate d2/f26 802278 0 2024-04-24T22:45:16.740 INFO:tasks.workunit.client.0.smithi063.stdout:1/127: write d3/f19 [2905642,129805] 0 2024-04-24T22:45:16.742 INFO:tasks.workunit.client.1.smithi192.stdout:2/144: dwrite d4/f19 [0,4194304] 0 2024-04-24T22:45:16.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/42: getdents . 0 2024-04-24T22:45:16.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/43: readlink - no filename 2024-04-24T22:45:16.745 INFO:tasks.workunit.client.1.smithi192.stdout:8/44: read f6 [77859,50973] 0 2024-04-24T22:45:16.746 INFO:tasks.workunit.client.1.smithi192.stdout:2/145: chown d4/d9/d10/l25 8103 1 2024-04-24T22:45:16.746 INFO:tasks.workunit.client.1.smithi192.stdout:2/146: stat d4/c8 0 2024-04-24T22:45:16.748 INFO:tasks.workunit.client.1.smithi192.stdout:4/47: creat d2/f17 x:0 0 0 2024-04-24T22:45:16.748 INFO:tasks.workunit.client.0.smithi063.stdout:1/128: dwrite d3/f1e [0,4194304] 0 2024-04-24T22:45:16.749 INFO:tasks.workunit.client.1.smithi192.stdout:9/68: write d3/f9 [4316235,28472] 0 2024-04-24T22:45:16.752 INFO:tasks.workunit.client.1.smithi192.stdout:1/91: chown d9/f12 293 1 2024-04-24T22:45:16.754 INFO:tasks.workunit.client.0.smithi063.stdout:1/129: dread d3/da/f27 [0,4194304] 0 2024-04-24T22:45:16.754 INFO:tasks.workunit.client.1.smithi192.stdout:3/54: truncate d2/d4/f5 4561014 0 2024-04-24T22:45:16.755 INFO:tasks.workunit.client.0.smithi063.stdout:8/159: truncate d1/d2/d3/f14 1750404 0 2024-04-24T22:45:16.756 INFO:tasks.workunit.client.0.smithi063.stdout:8/160: chown d1/d2/d3/l4 66513391 1 2024-04-24T22:45:16.758 INFO:tasks.workunit.client.0.smithi063.stdout:7/115: unlink d0/d4/de/d17/c2a 0 2024-04-24T22:45:16.759 INFO:tasks.workunit.client.0.smithi063.stdout:7/116: readlink d0/l16 0 2024-04-24T22:45:16.762 INFO:tasks.workunit.client.1.smithi192.stdout:0/134: mknod d3/d25/c29 0 2024-04-24T22:45:16.766 INFO:tasks.workunit.client.0.smithi063.stdout:2/125: rename d0/d2/d3/d7/c16 to d0/c24 0 2024-04-24T22:45:16.770 INFO:tasks.workunit.client.0.smithi063.stdout:6/130: link db/f1a db/d1d/d20/f2b 0 2024-04-24T22:45:16.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/45: creat f9 x:0 0 0 2024-04-24T22:45:16.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/118: unlink d2/d8/l14 0 2024-04-24T22:45:16.776 INFO:tasks.workunit.client.1.smithi192.stdout:2/147: fsync d4/d9/fb 0 2024-04-24T22:45:16.779 INFO:tasks.workunit.client.0.smithi063.stdout:9/123: getdents d6/d8/d1f/d20/d22 0 2024-04-24T22:45:16.780 INFO:tasks.workunit.client.0.smithi063.stdout:9/124: dread - d6/d8/d1f/d20/d22/f25 zero size 2024-04-24T22:45:16.780 INFO:tasks.workunit.client.1.smithi192.stdout:9/69: mknod d3/dd/c15 0 2024-04-24T22:45:16.780 INFO:tasks.workunit.client.0.smithi063.stdout:9/125: chown d6/c15 142434413 1 2024-04-24T22:45:16.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/137: creat d6/dd/d20/d21/d25/d31/f35 x:0 0 0 2024-04-24T22:45:16.785 INFO:tasks.workunit.client.1.smithi192.stdout:7/74: write f0 [1362297,66211] 0 2024-04-24T22:45:16.785 INFO:tasks.workunit.client.0.smithi063.stdout:9/126: dwrite d6/f1b [0,4194304] 0 2024-04-24T22:45:16.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/55: read d2/d4/d8/fa [1124594,86331] 0 2024-04-24T22:45:16.787 INFO:tasks.workunit.client.0.smithi063.stdout:8/161: mknod d1/d21/c35 0 2024-04-24T22:45:16.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/162: write d1/d2/f1c [1525351,102859] 0 2024-04-24T22:45:16.789 INFO:tasks.workunit.client.1.smithi192.stdout:0/135: read d3/f23 [23438,56416] 0 2024-04-24T22:45:16.791 INFO:tasks.workunit.client.1.smithi192.stdout:1/92: dwrite d9/fa [0,4194304] 0 2024-04-24T22:45:16.791 INFO:tasks.workunit.client.0.smithi063.stdout:7/117: mkdir d0/d2f 0 2024-04-24T22:45:16.792 INFO:tasks.workunit.client.0.smithi063.stdout:3/143: link da/cd da/c33 0 2024-04-24T22:45:16.794 INFO:tasks.workunit.client.1.smithi192.stdout:8/46: mknod ca 0 2024-04-24T22:45:16.794 INFO:tasks.workunit.client.0.smithi063.stdout:3/144: write da/f1a [4723272,7549] 0 2024-04-24T22:45:16.796 INFO:tasks.workunit.client.0.smithi063.stdout:3/145: write da/f2c [665876,105270] 0 2024-04-24T22:45:16.798 INFO:tasks.workunit.client.0.smithi063.stdout:7/118: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:16.799 INFO:tasks.workunit.client.0.smithi063.stdout:7/119: chown d0/d4/de/c2c 73 1 2024-04-24T22:45:16.801 INFO:tasks.workunit.client.0.smithi063.stdout:6/131: mknod db/d1d/c2c 0 2024-04-24T22:45:16.803 INFO:tasks.workunit.client.0.smithi063.stdout:4/119: rmdir d2/d8 39 2024-04-24T22:45:16.810 INFO:tasks.workunit.client.1.smithi192.stdout:9/70: dread d3/fa [0,4194304] 0 2024-04-24T22:45:16.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/127: creat d6/d8/d1f/f28 x:0 0 0 2024-04-24T22:45:16.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/71: dwrite d3/f9 [0,4194304] 0 2024-04-24T22:45:16.819 INFO:tasks.workunit.client.1.smithi192.stdout:0/136: creat d3/d25/f2a x:0 0 0 2024-04-24T22:45:16.826 INFO:tasks.workunit.client.1.smithi192.stdout:9/72: dwrite d3/f8 [0,4194304] 0 2024-04-24T22:45:16.831 INFO:tasks.workunit.client.0.smithi063.stdout:1/130: rename d3/da/f27 to d3/f2c 0 2024-04-24T22:45:16.834 INFO:tasks.workunit.client.1.smithi192.stdout:4/48: truncate d2/db/d10/f11 1570479 0 2024-04-24T22:45:16.835 INFO:tasks.workunit.client.0.smithi063.stdout:3/146: truncate da/f28 805715 0 2024-04-24T22:45:16.836 INFO:tasks.workunit.client.0.smithi063.stdout:3/147: fsync da/d18/f30 0 2024-04-24T22:45:16.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/137: symlink d3/d27/d28/l2b 0 2024-04-24T22:45:16.837 INFO:tasks.workunit.client.0.smithi063.stdout:5/141: getdents d5/d11/d18 0 2024-04-24T22:45:16.837 INFO:tasks.workunit.client.0.smithi063.stdout:3/148: write da/f1a [1912777,87431] 0 2024-04-24T22:45:16.842 INFO:tasks.workunit.client.1.smithi192.stdout:1/93: unlink d9/d15/f18 0 2024-04-24T22:45:16.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/120: dwrite d0/d4/de/f21 [0,4194304] 0 2024-04-24T22:45:16.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/132: truncate f7 3977520 0 2024-04-24T22:45:16.845 INFO:tasks.workunit.client.1.smithi192.stdout:3/56: dwrite d2/d4/d8/fa [0,4194304] 0 2024-04-24T22:45:16.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/138: dread f5 [0,4194304] 0 2024-04-24T22:45:16.848 INFO:tasks.workunit.client.1.smithi192.stdout:8/47: write f5 [1137930,68715] 0 2024-04-24T22:45:16.848 INFO:tasks.workunit.client.0.smithi063.stdout:9/128: symlink d6/d8/l29 0 2024-04-24T22:45:16.852 INFO:tasks.workunit.client.0.smithi063.stdout:1/131: mkdir d3/d2d 0 2024-04-24T22:45:16.856 INFO:tasks.workunit.client.1.smithi192.stdout:4/49: creat d2/db/f18 x:0 0 0 2024-04-24T22:45:16.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/163: rename d1/l24 to d1/d11/d32/l36 0 2024-04-24T22:45:16.857 INFO:tasks.workunit.client.0.smithi063.stdout:8/164: chown d1/d11/f29 0 1 2024-04-24T22:45:16.859 INFO:tasks.workunit.client.1.smithi192.stdout:0/138: creat d3/f2c x:0 0 0 2024-04-24T22:45:16.860 INFO:tasks.workunit.client.0.smithi063.stdout:8/165: dread d1/d22/f31 [0,4194304] 0 2024-04-24T22:45:16.864 INFO:tasks.workunit.client.1.smithi192.stdout:1/94: mkdir d9/dd/d21 0 2024-04-24T22:45:16.869 INFO:tasks.workunit.client.1.smithi192.stdout:3/57: read d2/d4/f5 [1711000,50853] 0 2024-04-24T22:45:16.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/73: write d3/fa [192432,3028] 0 2024-04-24T22:45:16.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/142: creat d5/d24/f32 x:0 0 0 2024-04-24T22:45:16.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/133: mkdir db/d1d/d2d 0 2024-04-24T22:45:16.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/134: write db/f10 [1798497,114076] 0 2024-04-24T22:45:16.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/48: mkdir db 0 2024-04-24T22:45:16.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/148: getdents d4 0 2024-04-24T22:45:16.876 INFO:tasks.workunit.client.0.smithi063.stdout:7/121: dread d0/d4/de/d17/f1e [0,4194304] 0 2024-04-24T22:45:16.877 INFO:tasks.workunit.client.0.smithi063.stdout:4/120: mknod d2/c2a 0 2024-04-24T22:45:16.878 INFO:tasks.workunit.client.1.smithi192.stdout:4/50: mknod d2/d9/c19 0 2024-04-24T22:45:16.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/139: mknod d3/d27/d28/c2d 0 2024-04-24T22:45:16.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/140: chown d3/l13 3017085 1 2024-04-24T22:45:16.881 INFO:tasks.workunit.client.0.smithi063.stdout:9/129: mkdir d6/d8/d1f/d20/d2a 0 2024-04-24T22:45:16.884 INFO:tasks.workunit.client.1.smithi192.stdout:3/58: mknod d2/d4/d7/c10 0 2024-04-24T22:45:16.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/74: creat d3/dd/de/f16 x:0 0 0 2024-04-24T22:45:16.886 INFO:tasks.workunit.client.1.smithi192.stdout:3/59: read d2/d4/f5 [4522580,12592] 0 2024-04-24T22:45:16.887 INFO:tasks.workunit.client.0.smithi063.stdout:2/126: getdents d0/d2/d3 0 2024-04-24T22:45:16.888 INFO:tasks.workunit.client.1.smithi192.stdout:0/141: dwrite d3/f14 [0,4194304] 0 2024-04-24T22:45:16.890 INFO:tasks.workunit.client.1.smithi192.stdout:4/51: unlink d2/d9/c19 0 2024-04-24T22:45:16.891 INFO:tasks.workunit.client.1.smithi192.stdout:0/142: truncate d3/d27/d28/f19 468511 0 2024-04-24T22:45:16.893 INFO:tasks.workunit.client.0.smithi063.stdout:3/149: read da/f12 [10986,112693] 0 2024-04-24T22:45:16.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/143: mkdir d5/d11/d33 0 2024-04-24T22:45:16.894 INFO:tasks.workunit.client.0.smithi063.stdout:3/150: write da/f31 [478499,34564] 0 2024-04-24T22:45:16.895 INFO:tasks.workunit.client.1.smithi192.stdout:9/75: dwrite d3/dd/de/f16 [0,4194304] 0 2024-04-24T22:45:16.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/151: write da/f29 [829582,15321] 0 2024-04-24T22:45:16.898 INFO:tasks.workunit.client.1.smithi192.stdout:8/49: mkdir db/dc 0 2024-04-24T22:45:16.899 INFO:tasks.workunit.client.1.smithi192.stdout:8/50: write f9 [253515,12320] 0 2024-04-24T22:45:16.900 INFO:tasks.workunit.client.1.smithi192.stdout:3/60: creat d2/f11 x:0 0 0 2024-04-24T22:45:16.904 INFO:tasks.workunit.client.0.smithi063.stdout:9/130: mkdir d6/d8/db/d2b 0 2024-04-24T22:45:16.904 INFO:tasks.workunit.client.0.smithi063.stdout:4/121: dread d2/d8/f11 [0,4194304] 0 2024-04-24T22:45:16.907 INFO:tasks.workunit.client.1.smithi192.stdout:0/143: write d3/d27/d28/f17 [714792,76186] 0 2024-04-24T22:45:16.907 INFO:tasks.workunit.client.0.smithi063.stdout:4/122: dread d2/f26 [0,4194304] 0 2024-04-24T22:45:16.907 INFO:tasks.workunit.client.1.smithi192.stdout:0/144: write d3/d27/d28/f24 [1023852,25347] 0 2024-04-24T22:45:16.908 INFO:tasks.workunit.client.1.smithi192.stdout:0/145: readlink d3/ld 0 2024-04-24T22:45:16.909 INFO:tasks.workunit.client.1.smithi192.stdout:9/76: mkdir d3/dd/de/d17 0 2024-04-24T22:45:16.910 INFO:tasks.workunit.client.0.smithi063.stdout:1/132: creat d3/d2d/f2e x:0 0 0 2024-04-24T22:45:16.910 INFO:tasks.workunit.client.0.smithi063.stdout:4/123: dread d2/fd [0,4194304] 0 2024-04-24T22:45:16.911 INFO:tasks.workunit.client.0.smithi063.stdout:4/124: chown d2/d19/f20 41 1 2024-04-24T22:45:16.914 INFO:tasks.workunit.client.0.smithi063.stdout:2/127: creat d0/d2/d3/f25 x:0 0 0 2024-04-24T22:45:16.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/146: mkdir d3/d27/d28/d2e 0 2024-04-24T22:45:16.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/147: readlink d3/ld 0 2024-04-24T22:45:16.920 INFO:tasks.workunit.client.1.smithi192.stdout:0/148: write d3/f2c [290346,25120] 0 2024-04-24T22:45:16.921 INFO:tasks.workunit.client.1.smithi192.stdout:9/77: rename d3/c13 to d3/c18 0 2024-04-24T22:45:16.922 INFO:tasks.workunit.client.1.smithi192.stdout:9/78: chown d3/dd 39365410 1 2024-04-24T22:45:16.923 INFO:tasks.workunit.client.0.smithi063.stdout:6/135: creat db/d1d/d20/d29/f2e x:0 0 0 2024-04-24T22:45:16.923 INFO:tasks.workunit.client.0.smithi063.stdout:6/136: readlink db/d1c/l1e 0 2024-04-24T22:45:16.924 INFO:tasks.workunit.client.0.smithi063.stdout:3/152: mknod da/d18/c34 0 2024-04-24T22:45:16.925 INFO:tasks.workunit.client.0.smithi063.stdout:3/153: dread - da/f15 zero size 2024-04-24T22:45:16.926 INFO:tasks.workunit.client.1.smithi192.stdout:9/79: dread d3/dd/de/f16 [0,4194304] 0 2024-04-24T22:45:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:16 smithi192 ceph-mon[33991]: pgmap v6: 65 pgs: 65 active+clean; 281 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 193 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-24T22:45:16.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/149: rename d4/d29/f1d to d4/d9/f2c 0 2024-04-24T22:45:16.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/154: dread da/f1a [0,4194304] 0 2024-04-24T22:45:16.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/155: chown l9 12 1 2024-04-24T22:45:16.933 INFO:tasks.workunit.client.1.smithi192.stdout:0/149: symlink d3/l2f 0 2024-04-24T22:45:16.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/150: dread d4/f13 [0,4194304] 0 2024-04-24T22:45:16.937 INFO:tasks.workunit.client.1.smithi192.stdout:1/95: dwrite d9/f1a [0,4194304] 0 2024-04-24T22:45:16.937 INFO:tasks.workunit.client.1.smithi192.stdout:1/96: chown d9/f12 135382904 1 2024-04-24T22:45:16.939 INFO:tasks.workunit.client.0.smithi063.stdout:7/122: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:16.942 INFO:tasks.workunit.client.0.smithi063.stdout:9/131: truncate d6/d8/db/f10 1127808 0 2024-04-24T22:45:16.948 INFO:tasks.workunit.client.0.smithi063.stdout:1/133: mknod d3/da/dc/d11/c2f 0 2024-04-24T22:45:16.949 INFO:tasks.workunit.client.1.smithi192.stdout:0/150: dread d3/f8 [0,4194304] 0 2024-04-24T22:45:16.950 INFO:tasks.workunit.client.0.smithi063.stdout:2/128: mknod d0/d2/d3/d1c/c26 0 2024-04-24T22:45:16.950 INFO:tasks.workunit.client.0.smithi063.stdout:2/129: fsync d0/d2/f1b 0 2024-04-24T22:45:16.951 INFO:tasks.workunit.client.0.smithi063.stdout:2/130: readlink d0/l23 0 2024-04-24T22:45:16.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/144: symlink d5/d24/d2c/l34 0 2024-04-24T22:45:16.958 INFO:tasks.workunit.client.0.smithi063.stdout:6/137: fsync db/f1a 0 2024-04-24T22:45:16.959 INFO:tasks.workunit.client.1.smithi192.stdout:3/61: getdents d2/d4/d8 0 2024-04-24T22:45:16.959 INFO:tasks.workunit.client.1.smithi192.stdout:8/51: dwrite f6 [0,4194304] 0 2024-04-24T22:45:16.960 INFO:tasks.workunit.client.0.smithi063.stdout:2/131: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:16.960 INFO:tasks.workunit.client.1.smithi192.stdout:3/62: chown d2/d4/d7/f9 71044 1 2024-04-24T22:45:16.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/132: write d0/d2/f1e [501639,29453] 0 2024-04-24T22:45:16.963 INFO:tasks.workunit.client.1.smithi192.stdout:0/151: truncate d3/f8 3421826 0 2024-04-24T22:45:16.963 INFO:tasks.workunit.client.0.smithi063.stdout:8/166: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:16.966 INFO:tasks.workunit.client.0.smithi063.stdout:0/139: getdents d6/dd/d20/d21/d25 0 2024-04-24T22:45:16.969 INFO:tasks.workunit.client.0.smithi063.stdout:9/132: symlink d6/d8/d1f/d20/d22/l2c 0 2024-04-24T22:45:16.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/63: mkdir d2/d4/d12 0 2024-04-24T22:45:16.973 INFO:tasks.workunit.client.1.smithi192.stdout:2/151: rename d4/f28 to d4/d9/f2d 0 2024-04-24T22:45:16.974 INFO:tasks.workunit.client.0.smithi063.stdout:1/134: dread d3/f17 [0,4194304] 0 2024-04-24T22:45:16.976 INFO:tasks.workunit.client.1.smithi192.stdout:0/152: creat d3/d27/f30 x:0 0 0 2024-04-24T22:45:16.977 INFO:tasks.workunit.client.1.smithi192.stdout:0/153: chown d3/cf 102904384 1 2024-04-24T22:45:16.978 INFO:tasks.workunit.client.1.smithi192.stdout:8/52: symlink db/dc/ld 0 2024-04-24T22:45:16.979 INFO:tasks.workunit.client.1.smithi192.stdout:0/154: dread d3/f23 [0,4194304] 0 2024-04-24T22:45:16.983 INFO:tasks.workunit.client.1.smithi192.stdout:0/155: dread d3/d27/d28/f1d [0,4194304] 0 2024-04-24T22:45:16.987 INFO:tasks.workunit.client.1.smithi192.stdout:0/156: dread d3/d27/d28/f19 [0,4194304] 0 2024-04-24T22:45:16.988 INFO:tasks.workunit.client.1.smithi192.stdout:8/53: dwrite f8 [0,4194304] 0 2024-04-24T22:45:16.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/138: mknod db/d1d/d20/d29/c2f 0 2024-04-24T22:45:16.996 INFO:tasks.workunit.client.0.smithi063.stdout:6/139: write db/d1d/d20/d29/f2e [820700,120243] 0 2024-04-24T22:45:16.998 INFO:tasks.workunit.client.1.smithi192.stdout:9/80: truncate d3/f4 1558846 0 2024-04-24T22:45:17.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/156: write da/d18/f19 [820839,78091] 0 2024-04-24T22:45:17.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/157: dread - da/f21 zero size 2024-04-24T22:45:17.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/158: write da/d18/f27 [3224436,31284] 0 2024-04-24T22:45:17.006 INFO:tasks.workunit.client.1.smithi192.stdout:9/81: dwrite d3/dd/de/f12 [0,4194304] 0 2024-04-24T22:45:17.006 INFO:tasks.workunit.client.1.smithi192.stdout:3/64: unlink d2/f3 0 2024-04-24T22:45:17.008 INFO:tasks.workunit.client.0.smithi063.stdout:2/133: fsync d0/dd/f11 0 2024-04-24T22:45:17.012 INFO:tasks.workunit.client.1.smithi192.stdout:9/82: dread d3/f9 [0,4194304] 0 2024-04-24T22:45:17.012 INFO:tasks.workunit.client.0.smithi063.stdout:8/167: mknod d1/d21/c37 0 2024-04-24T22:45:17.015 INFO:tasks.workunit.client.0.smithi063.stdout:0/140: readlink d6/dd/l1f 0 2024-04-24T22:45:17.015 INFO:tasks.workunit.client.1.smithi192.stdout:8/54: fsync f8 0 2024-04-24T22:45:17.021 INFO:tasks.workunit.client.1.smithi192.stdout:1/97: rename c6 to d9/d1c/c22 0 2024-04-24T22:45:17.023 INFO:tasks.workunit.client.1.smithi192.stdout:1/98: write d9/d15/f1b [31587,39536] 0 2024-04-24T22:45:17.024 INFO:tasks.workunit.client.1.smithi192.stdout:8/55: dwrite f3 [4194304,4194304] 0 2024-04-24T22:45:17.025 INFO:tasks.workunit.client.0.smithi063.stdout:1/135: creat d3/da/dc/d1f/f30 x:0 0 0 2024-04-24T22:45:17.028 INFO:tasks.workunit.client.0.smithi063.stdout:5/145: dwrite d5/f9 [0,4194304] 0 2024-04-24T22:45:17.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/140: fsync db/f1a 0 2024-04-24T22:45:17.035 INFO:tasks.workunit.client.1.smithi192.stdout:1/99: dwrite d9/fa [0,4194304] 0 2024-04-24T22:45:17.035 INFO:tasks.workunit.client.0.smithi063.stdout:4/125: sync 2024-04-24T22:45:17.036 INFO:tasks.workunit.client.0.smithi063.stdout:6/141: chown db/d1d/d20/f23 1 1 2024-04-24T22:45:17.042 INFO:tasks.workunit.client.1.smithi192.stdout:9/83: creat d3/dd/de/f19 x:0 0 0 2024-04-24T22:45:17.045 INFO:tasks.workunit.client.0.smithi063.stdout:3/159: rename da/l20 to da/d18/l35 0 2024-04-24T22:45:17.045 INFO:tasks.workunit.client.0.smithi063.stdout:3/160: chown da/d18 54376 1 2024-04-24T22:45:17.047 INFO:tasks.workunit.client.0.smithi063.stdout:3/161: write da/d18/f27 [786114,60215] 0 2024-04-24T22:45:17.047 INFO:tasks.workunit.client.0.smithi063.stdout:3/162: stat da/ce 0 2024-04-24T22:45:17.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/163: dread - da/f15 zero size 2024-04-24T22:45:17.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/164: fdatasync da/f2e 0 2024-04-24T22:45:17.050 INFO:tasks.workunit.client.0.smithi063.stdout:2/134: stat d0/c24 0 2024-04-24T22:45:17.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:16 smithi063 ceph-mon[26912]: pgmap v6: 65 pgs: 65 active+clean; 281 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 193 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-24T22:45:17.053 INFO:tasks.workunit.client.0.smithi063.stdout:8/168: mkdir d1/d11/d32/d38 0 2024-04-24T22:45:17.054 INFO:tasks.workunit.client.1.smithi192.stdout:8/56: rmdir db/dc 39 2024-04-24T22:45:17.055 INFO:tasks.workunit.client.0.smithi063.stdout:2/135: dwrite d0/d2/d3/d7/f13 [0,4194304] 0 2024-04-24T22:45:17.055 INFO:tasks.workunit.client.0.smithi063.stdout:0/141: creat d6/dd/d20/d21/f36 x:0 0 0 2024-04-24T22:45:17.057 INFO:tasks.workunit.client.0.smithi063.stdout:2/136: read - d0/d2/f1d zero size 2024-04-24T22:45:17.062 INFO:tasks.workunit.client.0.smithi063.stdout:4/126: creat d2/d19/f2b x:0 0 0 2024-04-24T22:45:17.062 INFO:tasks.workunit.client.0.smithi063.stdout:4/127: write d2/d8/f16 [124362,76230] 0 2024-04-24T22:45:17.066 INFO:tasks.workunit.client.0.smithi063.stdout:6/142: rename db/c18 to db/c30 0 2024-04-24T22:45:17.067 INFO:tasks.workunit.client.0.smithi063.stdout:6/143: dread - db/f25 zero size 2024-04-24T22:45:17.067 INFO:tasks.workunit.client.1.smithi192.stdout:0/157: truncate d3/d27/d28/f19 315701 0 2024-04-24T22:45:17.074 INFO:tasks.workunit.client.0.smithi063.stdout:3/165: unlink da/f12 0 2024-04-24T22:45:17.078 INFO:tasks.workunit.client.1.smithi192.stdout:1/100: symlink d9/dd/d21/l23 0 2024-04-24T22:45:17.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/142: rmdir d6 39 2024-04-24T22:45:17.082 INFO:tasks.workunit.client.1.smithi192.stdout:1/101: dread d9/dd/ff [0,4194304] 0 2024-04-24T22:45:17.083 INFO:tasks.workunit.client.1.smithi192.stdout:1/102: stat d9/f1a 0 2024-04-24T22:45:17.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/133: link d6/d8/l16 d6/d8/d1f/d20/d2a/l2d 0 2024-04-24T22:45:17.084 INFO:tasks.workunit.client.1.smithi192.stdout:9/84: dwrite f2 [0,4194304] 0 2024-04-24T22:45:17.084 INFO:tasks.workunit.client.1.smithi192.stdout:1/103: fdatasync d9/d15/f1f 0 2024-04-24T22:45:17.085 INFO:tasks.workunit.client.1.smithi192.stdout:0/158: mkdir d3/d27/d28/d31 0 2024-04-24T22:45:17.085 INFO:tasks.workunit.client.1.smithi192.stdout:6/50: sync 2024-04-24T22:45:17.085 INFO:tasks.workunit.client.1.smithi192.stdout:9/85: write d3/f8 [2132526,125271] 0 2024-04-24T22:45:17.088 INFO:tasks.workunit.client.1.smithi192.stdout:2/152: symlink d4/d9/l2e 0 2024-04-24T22:45:17.089 INFO:tasks.workunit.client.1.smithi192.stdout:9/86: fdatasync d3/f8 0 2024-04-24T22:45:17.089 INFO:tasks.workunit.client.1.smithi192.stdout:1/104: dread d9/f1a [0,4194304] 0 2024-04-24T22:45:17.090 INFO:tasks.workunit.client.1.smithi192.stdout:1/105: write d9/fa [4374514,117486] 0 2024-04-24T22:45:17.093 INFO:tasks.workunit.client.1.smithi192.stdout:8/57: symlink db/dc/le 0 2024-04-24T22:45:17.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/128: truncate d2/f26 1757698 0 2024-04-24T22:45:17.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/58: read f9 [70750,75932] 0 2024-04-24T22:45:17.096 INFO:tasks.workunit.client.1.smithi192.stdout:3/65: getdents d2/d4 0 2024-04-24T22:45:17.096 INFO:tasks.workunit.client.1.smithi192.stdout:3/66: dread - d2/f11 zero size 2024-04-24T22:45:17.097 INFO:tasks.workunit.client.0.smithi063.stdout:6/144: symlink db/l31 0 2024-04-24T22:45:17.099 INFO:tasks.workunit.client.0.smithi063.stdout:4/129: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:17.101 INFO:tasks.workunit.client.0.smithi063.stdout:4/130: write d2/f13 [1861971,105197] 0 2024-04-24T22:45:17.106 INFO:tasks.workunit.client.0.smithi063.stdout:9/134: symlink d6/d8/l2e 0 2024-04-24T22:45:17.107 INFO:tasks.workunit.client.1.smithi192.stdout:5/42: sync 2024-04-24T22:45:17.107 INFO:tasks.workunit.client.1.smithi192.stdout:7/75: sync 2024-04-24T22:45:17.109 INFO:tasks.workunit.client.1.smithi192.stdout:5/43: write d0/f1 [2865943,79220] 0 2024-04-24T22:45:17.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/159: rename d3/c5 to d3/d27/d28/c32 0 2024-04-24T22:45:17.114 INFO:tasks.workunit.client.1.smithi192.stdout:7/76: dread f1 [0,4194304] 0 2024-04-24T22:45:17.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/44: dread d0/fb [0,4194304] 0 2024-04-24T22:45:17.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/45: write d0/f1 [3305294,50817] 0 2024-04-24T22:45:17.117 INFO:tasks.workunit.client.0.smithi063.stdout:3/166: rmdir da/d18 39 2024-04-24T22:45:17.123 INFO:tasks.workunit.client.0.smithi063.stdout:8/169: creat d1/d2/f39 x:0 0 0 2024-04-24T22:45:17.124 INFO:tasks.workunit.client.0.smithi063.stdout:6/145: dread db/ff [0,4194304] 0 2024-04-24T22:45:17.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/153: dread d4/f19 [0,4194304] 0 2024-04-24T22:45:17.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/51: dwrite d0/d8/fc [0,4194304] 0 2024-04-24T22:45:17.126 INFO:tasks.workunit.client.0.smithi063.stdout:7/123: sync 2024-04-24T22:45:17.129 INFO:tasks.workunit.client.0.smithi063.stdout:4/131: rmdir d2/d8 39 2024-04-24T22:45:17.131 INFO:tasks.workunit.client.0.smithi063.stdout:0/143: symlink d6/da/d2b/l37 0 2024-04-24T22:45:17.131 INFO:tasks.workunit.client.1.smithi192.stdout:8/59: rmdir db 39 2024-04-24T22:45:17.133 INFO:tasks.workunit.client.1.smithi192.stdout:3/67: creat d2/f13 x:0 0 0 2024-04-24T22:45:17.133 INFO:tasks.workunit.client.1.smithi192.stdout:3/68: write d2/f11 [576743,42097] 0 2024-04-24T22:45:17.134 INFO:tasks.workunit.client.0.smithi063.stdout:5/146: getdents d5/d11/d23 0 2024-04-24T22:45:17.135 INFO:tasks.workunit.client.1.smithi192.stdout:3/69: write d2/d4/d8/fa [2704885,62693] 0 2024-04-24T22:45:17.135 INFO:tasks.workunit.client.0.smithi063.stdout:5/147: chown d5/d11/d18/f20 69387 1 2024-04-24T22:45:17.136 INFO:tasks.workunit.client.1.smithi192.stdout:3/70: stat d2 0 2024-04-24T22:45:17.136 INFO:tasks.workunit.client.0.smithi063.stdout:5/148: chown d5/d11/f2f 6007 1 2024-04-24T22:45:17.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/170: creat d1/d21/f3a x:0 0 0 2024-04-24T22:45:17.138 INFO:tasks.workunit.client.0.smithi063.stdout:8/171: chown d1/d11/l13 235558539 1 2024-04-24T22:45:17.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/146: symlink db/d1d/d20/d29/l32 0 2024-04-24T22:45:17.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/149: dread d5/d11/f15 [0,4194304] 0 2024-04-24T22:45:17.141 INFO:tasks.workunit.client.1.smithi192.stdout:0/160: rename d3/d27/d28/l1a to d3/d27/l33 0 2024-04-24T22:45:17.142 INFO:tasks.workunit.client.0.smithi063.stdout:5/150: fsync d5/fd 0 2024-04-24T22:45:17.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/151: read - d5/d24/f32 zero size 2024-04-24T22:45:17.144 INFO:tasks.workunit.client.0.smithi063.stdout:6/147: dread f3 [0,4194304] 0 2024-04-24T22:45:17.145 INFO:tasks.workunit.client.0.smithi063.stdout:5/152: chown l2 54657641 1 2024-04-24T22:45:17.146 INFO:tasks.workunit.client.0.smithi063.stdout:8/172: dwrite d1/d2/d17/f27 [0,4194304] 0 2024-04-24T22:45:17.147 INFO:tasks.workunit.client.0.smithi063.stdout:8/173: chown d1/d11/f29 1 1 2024-04-24T22:45:17.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/136: sync 2024-04-24T22:45:17.150 INFO:tasks.workunit.client.0.smithi063.stdout:2/137: sync 2024-04-24T22:45:17.155 INFO:tasks.workunit.client.1.smithi192.stdout:2/154: read - d4/d29/f2b zero size 2024-04-24T22:45:17.160 INFO:tasks.workunit.client.0.smithi063.stdout:9/135: mkdir d6/d2f 0 2024-04-24T22:45:17.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/155: dread d4/d9/fb [0,4194304] 0 2024-04-24T22:45:17.161 INFO:tasks.workunit.client.0.smithi063.stdout:1/137: readlink d3/l8 0 2024-04-24T22:45:17.163 INFO:tasks.workunit.client.0.smithi063.stdout:0/144: write d6/dd/d20/d24/f2e [4778994,86046] 0 2024-04-24T22:45:17.163 INFO:tasks.workunit.client.0.smithi063.stdout:7/124: mknod d0/d2f/c30 0 2024-04-24T22:45:17.163 INFO:tasks.workunit.client.0.smithi063.stdout:9/136: fdatasync d6/d8/d1f/d20/d22/f25 0 2024-04-24T22:45:17.165 INFO:tasks.workunit.client.0.smithi063.stdout:1/138: chown d3/da/dc/f1c 886746 1 2024-04-24T22:45:17.166 INFO:tasks.workunit.client.0.smithi063.stdout:7/125: write d0/d4/de/f1a [4965358,65816] 0 2024-04-24T22:45:17.166 INFO:tasks.workunit.client.0.smithi063.stdout:6/148: dread db/f10 [0,4194304] 0 2024-04-24T22:45:17.168 INFO:tasks.workunit.client.0.smithi063.stdout:1/139: chown d3/da/dc/d1f/d21 71510196 1 2024-04-24T22:45:17.168 INFO:tasks.workunit.client.0.smithi063.stdout:0/145: read d6/da/f10 [604660,86552] 0 2024-04-24T22:45:17.168 INFO:tasks.workunit.client.0.smithi063.stdout:5/153: mknod d5/d1d/c35 0 2024-04-24T22:45:17.169 INFO:tasks.workunit.client.0.smithi063.stdout:8/174: rmdir d1/d22 39 2024-04-24T22:45:17.172 INFO:tasks.workunit.client.1.smithi192.stdout:0/161: stat d3/d27/l33 0 2024-04-24T22:45:17.173 INFO:tasks.workunit.client.0.smithi063.stdout:7/126: dwrite d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:17.178 INFO:tasks.workunit.client.1.smithi192.stdout:1/106: creat d9/f24 x:0 0 0 2024-04-24T22:45:17.179 INFO:tasks.workunit.client.1.smithi192.stdout:5/46: truncate d0/fb 2055242 0 2024-04-24T22:45:17.179 INFO:tasks.workunit.client.1.smithi192.stdout:6/52: symlink d0/l15 0 2024-04-24T22:45:17.185 INFO:tasks.workunit.client.1.smithi192.stdout:0/162: creat d3/d27/d28/f34 x:0 0 0 2024-04-24T22:45:17.187 INFO:tasks.workunit.client.1.smithi192.stdout:1/107: unlink l7 0 2024-04-24T22:45:17.192 INFO:tasks.workunit.client.1.smithi192.stdout:2/156: mknod d4/c2f 0 2024-04-24T22:45:17.192 INFO:tasks.workunit.client.1.smithi192.stdout:8/60: symlink db/lf 0 2024-04-24T22:45:17.195 INFO:tasks.workunit.client.1.smithi192.stdout:7/77: rename d2/db/ff to d2/f17 0 2024-04-24T22:45:17.198 INFO:tasks.workunit.client.1.smithi192.stdout:1/108: symlink d9/d15/l25 0 2024-04-24T22:45:17.206 INFO:tasks.workunit.client.1.smithi192.stdout:2/157: write d4/d9/d10/f1e [1148600,21419] 0 2024-04-24T22:45:17.208 INFO:tasks.workunit.client.1.smithi192.stdout:3/71: getdents d2/d4/d7 0 2024-04-24T22:45:17.213 INFO:tasks.workunit.client.1.smithi192.stdout:0/163: creat d3/d27/d28/d2e/f35 x:0 0 0 2024-04-24T22:45:17.213 INFO:tasks.workunit.client.1.smithi192.stdout:7/78: dread d2/f5 [0,4194304] 0 2024-04-24T22:45:17.214 INFO:tasks.workunit.client.1.smithi192.stdout:7/79: chown d2/db/f13 28655763 1 2024-04-24T22:45:17.216 INFO:tasks.workunit.client.1.smithi192.stdout:1/109: unlink d9/f24 0 2024-04-24T22:45:17.222 INFO:tasks.workunit.client.1.smithi192.stdout:0/164: dwrite d3/d27/d28/f34 [0,4194304] 0 2024-04-24T22:45:17.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/110: dwrite d9/d15/f1f [0,4194304] 0 2024-04-24T22:45:17.231 INFO:tasks.workunit.client.1.smithi192.stdout:3/72: rename d2/f11 to d2/d4/f14 0 2024-04-24T22:45:17.231 INFO:tasks.workunit.client.1.smithi192.stdout:6/53: write d0/d8/f14 [1215075,18926] 0 2024-04-24T22:45:17.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/73: stat d2/d4/d8/fa 0 2024-04-24T22:45:17.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/137: sync 2024-04-24T22:45:17.238 INFO:tasks.workunit.client.1.smithi192.stdout:7/80: unlink d2/f16 0 2024-04-24T22:45:17.244 INFO:tasks.workunit.client.1.smithi192.stdout:5/47: dread d0/d6/fd [0,4194304] 0 2024-04-24T22:45:17.244 INFO:tasks.workunit.client.1.smithi192.stdout:5/48: rename d0 to d0/df 22 2024-04-24T22:45:17.244 INFO:tasks.workunit.client.1.smithi192.stdout:0/165: symlink d3/d27/l36 0 2024-04-24T22:45:17.246 INFO:tasks.workunit.client.1.smithi192.stdout:5/49: write d0/f1 [2618719,57566] 0 2024-04-24T22:45:17.246 INFO:tasks.workunit.client.1.smithi192.stdout:2/158: symlink d4/l30 0 2024-04-24T22:45:17.253 INFO:tasks.workunit.client.1.smithi192.stdout:3/74: creat d2/d4/f15 x:0 0 0 2024-04-24T22:45:17.254 INFO:tasks.workunit.client.1.smithi192.stdout:6/54: dread d0/d8/fa [0,4194304] 0 2024-04-24T22:45:17.255 INFO:tasks.workunit.client.1.smithi192.stdout:6/55: readlink d0/d8/l13 0 2024-04-24T22:45:17.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/166: rmdir d3/d27/d28 39 2024-04-24T22:45:17.268 INFO:tasks.workunit.client.1.smithi192.stdout:3/75: creat d2/d4/f16 x:0 0 0 2024-04-24T22:45:17.269 INFO:tasks.workunit.client.1.smithi192.stdout:3/76: read - d2/d4/d7/f9 zero size 2024-04-24T22:45:17.269 INFO:tasks.workunit.client.1.smithi192.stdout:3/77: dread - d2/f13 zero size 2024-04-24T22:45:17.270 INFO:tasks.workunit.client.1.smithi192.stdout:6/56: mknod d0/d8/c16 0 2024-04-24T22:45:17.271 INFO:tasks.workunit.client.1.smithi192.stdout:6/57: chown d0/l15 4 1 2024-04-24T22:45:17.277 INFO:tasks.workunit.client.1.smithi192.stdout:5/50: link d0/fc d0/f10 0 2024-04-24T22:45:17.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/58: creat d0/d8/f17 x:0 0 0 2024-04-24T22:45:17.286 INFO:tasks.workunit.client.1.smithi192.stdout:2/159: rename f0 to d4/f31 0 2024-04-24T22:45:17.289 INFO:tasks.workunit.client.1.smithi192.stdout:5/51: unlink d0/f1 0 2024-04-24T22:45:17.290 INFO:tasks.workunit.client.1.smithi192.stdout:0/167: dwrite d3/f23 [0,4194304] 0 2024-04-24T22:45:17.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/52: read d0/d6/fd [999069,124266] 0 2024-04-24T22:45:17.291 INFO:tasks.workunit.client.1.smithi192.stdout:1/111: getdents d9/dd 0 2024-04-24T22:45:17.293 INFO:tasks.workunit.client.1.smithi192.stdout:5/53: rename d0/d6 to d0/d6/d11 22 2024-04-24T22:45:17.297 INFO:tasks.workunit.client.1.smithi192.stdout:3/78: link d2/d4/f15 d2/d4/d7/f17 0 2024-04-24T22:45:17.297 INFO:tasks.workunit.client.1.smithi192.stdout:1/112: dread d9/f12 [0,4194304] 0 2024-04-24T22:45:17.298 INFO:tasks.workunit.client.1.smithi192.stdout:6/59: mkdir d0/d8/d18 0 2024-04-24T22:45:17.299 INFO:tasks.workunit.client.1.smithi192.stdout:7/81: getdents d2/db 0 2024-04-24T22:45:17.301 INFO:tasks.workunit.client.1.smithi192.stdout:2/160: chown d4/f13 102 1 2024-04-24T22:45:17.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/79: mknod d2/d4/d8/c18 0 2024-04-24T22:45:17.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/80: chown d2/d4/d7 510 1 2024-04-24T22:45:17.313 INFO:tasks.workunit.client.1.smithi192.stdout:1/113: mknod d9/dd/d21/c26 0 2024-04-24T22:45:17.315 INFO:tasks.workunit.client.1.smithi192.stdout:1/114: truncate d9/d15/f1b 92270 0 2024-04-24T22:45:17.315 INFO:tasks.workunit.client.1.smithi192.stdout:5/54: rmdir d0 39 2024-04-24T22:45:17.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/168: mkdir d3/d27/d28/d2e/d37 0 2024-04-24T22:45:17.326 INFO:tasks.workunit.client.1.smithi192.stdout:0/169: dread d3/f2c [0,4194304] 0 2024-04-24T22:45:17.328 INFO:tasks.workunit.client.1.smithi192.stdout:3/81: write d2/d4/f14 [625717,80475] 0 2024-04-24T22:45:17.329 INFO:tasks.workunit.client.1.smithi192.stdout:2/161: dwrite d4/f27 [0,4194304] 0 2024-04-24T22:45:17.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/115: mknod d9/d1c/c27 0 2024-04-24T22:45:17.331 INFO:tasks.workunit.client.1.smithi192.stdout:3/82: dread - d2/d4/f16 zero size 2024-04-24T22:45:17.331 INFO:tasks.workunit.client.1.smithi192.stdout:3/83: dread - d2/d4/f16 zero size 2024-04-24T22:45:17.332 INFO:tasks.workunit.client.1.smithi192.stdout:5/55: chown d0/f10 381322 1 2024-04-24T22:45:17.337 INFO:tasks.workunit.client.1.smithi192.stdout:7/82: mknod d2/c18 0 2024-04-24T22:45:17.346 INFO:tasks.workunit.client.1.smithi192.stdout:6/60: dwrite d0/d8/fa [0,4194304] 0 2024-04-24T22:45:17.346 INFO:tasks.workunit.client.1.smithi192.stdout:2/162: write d4/d29/f2b [132013,46227] 0 2024-04-24T22:45:17.347 INFO:tasks.workunit.client.1.smithi192.stdout:1/116: rmdir d9/dd/d21 39 2024-04-24T22:45:17.349 INFO:tasks.workunit.client.1.smithi192.stdout:1/117: dread d9/dd/ff [0,4194304] 0 2024-04-24T22:45:17.350 INFO:tasks.workunit.client.1.smithi192.stdout:3/84: creat d2/d4/d8/f19 x:0 0 0 2024-04-24T22:45:17.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/56: fdatasync d0/fc 0 2024-04-24T22:45:17.358 INFO:tasks.workunit.client.1.smithi192.stdout:0/170: symlink d3/d27/d28/d31/l38 0 2024-04-24T22:45:17.359 INFO:tasks.workunit.client.1.smithi192.stdout:4/52: sync 2024-04-24T22:45:17.369 INFO:tasks.workunit.client.1.smithi192.stdout:5/57: truncate d0/fb 1144805 0 2024-04-24T22:45:17.371 INFO:tasks.workunit.client.1.smithi192.stdout:0/171: dread d3/d27/d28/f1d [0,4194304] 0 2024-04-24T22:45:17.374 INFO:tasks.workunit.client.1.smithi192.stdout:2/163: symlink d4/d9/d10/l32 0 2024-04-24T22:45:17.376 INFO:tasks.workunit.client.1.smithi192.stdout:2/164: write d4/d9/fb [1964393,123304] 0 2024-04-24T22:45:17.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/118: rename d9/dd/d21/c26 to d9/d15/c28 0 2024-04-24T22:45:17.379 INFO:tasks.workunit.client.1.smithi192.stdout:1/119: write d9/fa [218152,21269] 0 2024-04-24T22:45:17.379 INFO:tasks.workunit.client.1.smithi192.stdout:3/85: mkdir d2/d4/d12/d1a 0 2024-04-24T22:45:17.383 INFO:tasks.workunit.client.1.smithi192.stdout:0/172: creat d3/f39 x:0 0 0 2024-04-24T22:45:17.384 INFO:tasks.workunit.client.1.smithi192.stdout:6/61: creat d0/f19 x:0 0 0 2024-04-24T22:45:17.391 INFO:tasks.workunit.client.1.smithi192.stdout:7/83: link d2/db/ld d2/l19 0 2024-04-24T22:45:17.392 INFO:tasks.workunit.client.1.smithi192.stdout:0/173: creat d3/d27/d28/d31/f3a x:0 0 0 2024-04-24T22:45:17.396 INFO:tasks.workunit.client.1.smithi192.stdout:5/58: dread d0/d6/fd [0,4194304] 0 2024-04-24T22:45:17.397 INFO:tasks.workunit.client.1.smithi192.stdout:5/59: readlink d0/l8 0 2024-04-24T22:45:17.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/60: chown d0/fc 889 1 2024-04-24T22:45:17.398 INFO:tasks.workunit.client.1.smithi192.stdout:2/165: fdatasync d4/d9/f2c 0 2024-04-24T22:45:17.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/61: stat d0/d6 0 2024-04-24T22:45:17.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/120: unlink d9/lb 0 2024-04-24T22:45:17.403 INFO:tasks.workunit.client.1.smithi192.stdout:3/86: link d2/d4/f16 d2/d4/d12/f1b 0 2024-04-24T22:45:17.404 INFO:tasks.workunit.client.1.smithi192.stdout:3/87: read - d2/d4/d7/fd zero size 2024-04-24T22:45:17.417 INFO:tasks.workunit.client.1.smithi192.stdout:5/62: mknod d0/d6/c12 0 2024-04-24T22:45:17.419 INFO:tasks.workunit.client.1.smithi192.stdout:1/121: rmdir d9/dd 39 2024-04-24T22:45:17.422 INFO:tasks.workunit.client.1.smithi192.stdout:3/88: creat d2/d4/d12/f1c x:0 0 0 2024-04-24T22:45:17.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/174: mkdir d3/d27/d28/d2e/d37/d3b 0 2024-04-24T22:45:17.427 INFO:tasks.workunit.client.1.smithi192.stdout:2/166: mknod d4/d9/c33 0 2024-04-24T22:45:17.428 INFO:tasks.workunit.client.1.smithi192.stdout:2/167: fsync d4/d9/fb 0 2024-04-24T22:45:17.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/122: stat d9/dd/d21/l23 0 2024-04-24T22:45:17.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/168: dread d4/d9/d10/f23 [0,4194304] 0 2024-04-24T22:45:17.432 INFO:tasks.workunit.client.1.smithi192.stdout:7/84: creat d2/f1a x:0 0 0 2024-04-24T22:45:17.433 INFO:tasks.workunit.client.1.smithi192.stdout:7/85: stat d2/db 0 2024-04-24T22:45:17.434 INFO:tasks.workunit.client.1.smithi192.stdout:6/62: link d0/l15 d0/l1a 0 2024-04-24T22:45:17.435 INFO:tasks.workunit.client.1.smithi192.stdout:6/63: chown d0/cf 65 1 2024-04-24T22:45:17.435 INFO:tasks.workunit.client.1.smithi192.stdout:5/63: dwrite d0/fe [0,4194304] 0 2024-04-24T22:45:17.436 INFO:tasks.workunit.client.1.smithi192.stdout:6/64: chown d0/d8/f14 34490 1 2024-04-24T22:45:17.437 INFO:tasks.workunit.client.1.smithi192.stdout:5/64: chown d0/fb 0 1 2024-04-24T22:45:17.438 INFO:tasks.workunit.client.1.smithi192.stdout:3/89: mkdir d2/d4/d12/d1a/d1d 0 2024-04-24T22:45:17.446 INFO:tasks.workunit.client.1.smithi192.stdout:5/65: mkdir d0/d13 0 2024-04-24T22:45:17.448 INFO:tasks.workunit.client.1.smithi192.stdout:6/65: symlink d0/d8/l1b 0 2024-04-24T22:45:17.454 INFO:tasks.workunit.client.1.smithi192.stdout:2/169: unlink d4/fe 0 2024-04-24T22:45:17.456 INFO:tasks.workunit.client.1.smithi192.stdout:6/66: dwrite d0/d8/fa [0,4194304] 0 2024-04-24T22:45:17.457 INFO:tasks.workunit.client.1.smithi192.stdout:6/67: chown d0/d8/fe 44 1 2024-04-24T22:45:17.460 INFO:tasks.workunit.client.1.smithi192.stdout:7/86: getdents d2/db 0 2024-04-24T22:45:17.468 INFO:tasks.workunit.client.1.smithi192.stdout:7/87: dwrite d2/db/f10 [0,4194304] 0 2024-04-24T22:45:17.469 INFO:tasks.workunit.client.1.smithi192.stdout:2/170: fdatasync d4/d9/f18 0 2024-04-24T22:45:17.470 INFO:tasks.workunit.client.1.smithi192.stdout:0/175: rename d3/d27/d28/d2e/d37 to d3/d27/d3c 0 2024-04-24T22:45:17.472 INFO:tasks.workunit.client.1.smithi192.stdout:0/176: chown d3/ld 764 1 2024-04-24T22:45:17.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/123: creat d9/dd/f29 x:0 0 0 2024-04-24T22:45:17.477 INFO:tasks.workunit.client.1.smithi192.stdout:7/88: dread d2/db/f10 [0,4194304] 0 2024-04-24T22:45:17.478 INFO:tasks.workunit.client.1.smithi192.stdout:6/68: symlink d0/d8/l1c 0 2024-04-24T22:45:17.479 INFO:tasks.workunit.client.1.smithi192.stdout:6/69: readlink d0/d8/l13 0 2024-04-24T22:45:17.481 INFO:tasks.workunit.client.1.smithi192.stdout:6/70: chown d0/cf 0 1 2024-04-24T22:45:17.484 INFO:tasks.workunit.client.1.smithi192.stdout:7/89: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:17.486 INFO:tasks.workunit.client.1.smithi192.stdout:2/171: read d4/f19 [2514836,82353] 0 2024-04-24T22:45:17.493 INFO:tasks.workunit.client.1.smithi192.stdout:2/172: dread d4/d9/d10/f1e [0,4194304] 0 2024-04-24T22:45:17.494 INFO:tasks.workunit.client.1.smithi192.stdout:0/177: rename d3/f23 to d3/d25/f3d 0 2024-04-24T22:45:17.500 INFO:tasks.workunit.client.1.smithi192.stdout:7/90: unlink d2/db/le 0 2024-04-24T22:45:17.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/91: chown d2/db/f15 42 1 2024-04-24T22:45:17.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/92: readlink d2/l8 0 2024-04-24T22:45:17.509 INFO:tasks.workunit.client.1.smithi192.stdout:0/178: unlink d3/le 0 2024-04-24T22:45:17.512 INFO:tasks.workunit.client.1.smithi192.stdout:2/173: creat d4/d29/f34 x:0 0 0 2024-04-24T22:45:17.512 INFO:tasks.workunit.client.1.smithi192.stdout:3/90: dwrite d2/d4/f5 [0,4194304] 0 2024-04-24T22:45:17.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/91: read - d2/d4/d12/f1b zero size 2024-04-24T22:45:17.514 INFO:tasks.workunit.client.1.smithi192.stdout:2/174: truncate d4/d29/f34 769069 0 2024-04-24T22:45:17.517 INFO:tasks.workunit.client.1.smithi192.stdout:7/93: symlink d2/db/l1b 0 2024-04-24T22:45:17.522 INFO:tasks.workunit.client.1.smithi192.stdout:5/66: dwrite d0/f10 [0,4194304] 0 2024-04-24T22:45:17.523 INFO:tasks.workunit.client.1.smithi192.stdout:0/179: rename d3/d27/d28/d2e/f35 to d3/d27/d28/f3e 0 2024-04-24T22:45:17.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/94: dread d2/db/f10 [0,4194304] 0 2024-04-24T22:45:17.529 INFO:tasks.workunit.client.1.smithi192.stdout:0/180: dread d3/d25/f3d [0,4194304] 0 2024-04-24T22:45:17.534 INFO:tasks.workunit.client.1.smithi192.stdout:7/95: dwrite d2/db/f15 [0,4194304] 0 2024-04-24T22:45:17.539 INFO:tasks.workunit.client.1.smithi192.stdout:7/96: dread d2/db/f13 [0,4194304] 0 2024-04-24T22:45:17.541 INFO:tasks.workunit.client.1.smithi192.stdout:9/87: sync 2024-04-24T22:45:17.541 INFO:tasks.workunit.client.1.smithi192.stdout:8/61: sync 2024-04-24T22:45:17.542 INFO:tasks.workunit.client.1.smithi192.stdout:8/62: write f5 [2252191,35035] 0 2024-04-24T22:45:17.543 INFO:tasks.workunit.client.1.smithi192.stdout:8/63: write f6 [1089825,93402] 0 2024-04-24T22:45:17.547 INFO:tasks.workunit.client.1.smithi192.stdout:9/88: dread d3/fa [0,4194304] 0 2024-04-24T22:45:17.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/67: truncate d0/d6/fd 13950 0 2024-04-24T22:45:17.553 INFO:tasks.workunit.client.1.smithi192.stdout:8/64: dwrite f8 [0,4194304] 0 2024-04-24T22:45:17.555 INFO:tasks.workunit.client.1.smithi192.stdout:9/89: dread f2 [0,4194304] 0 2024-04-24T22:45:17.556 INFO:tasks.workunit.client.1.smithi192.stdout:0/181: mkdir d3/d27/d28/d2e/d3f 0 2024-04-24T22:45:17.570 INFO:tasks.workunit.client.1.smithi192.stdout:7/97: mknod d2/db/c1c 0 2024-04-24T22:45:17.571 INFO:tasks.workunit.client.1.smithi192.stdout:7/98: chown d2/f1a 52936 1 2024-04-24T22:45:17.572 INFO:tasks.workunit.client.1.smithi192.stdout:3/92: write d2/d4/f16 [517762,91732] 0 2024-04-24T22:45:17.575 INFO:tasks.workunit.client.1.smithi192.stdout:3/93: truncate d2/d4/d7/fd 676794 0 2024-04-24T22:45:17.577 INFO:tasks.workunit.client.1.smithi192.stdout:1/124: truncate d9/f16 1362254 0 2024-04-24T22:45:17.580 INFO:tasks.workunit.client.1.smithi192.stdout:9/90: creat d3/dd/de/f1a x:0 0 0 2024-04-24T22:45:17.581 INFO:tasks.workunit.client.1.smithi192.stdout:8/65: dread f9 [0,4194304] 0 2024-04-24T22:45:17.582 INFO:tasks.workunit.client.1.smithi192.stdout:0/182: mkdir d3/d40 0 2024-04-24T22:45:17.584 INFO:tasks.workunit.client.1.smithi192.stdout:2/175: creat d4/d9/f35 x:0 0 0 2024-04-24T22:45:17.585 INFO:tasks.workunit.client.1.smithi192.stdout:1/125: dwrite d9/dd/f19 [0,4194304] 0 2024-04-24T22:45:17.586 INFO:tasks.workunit.client.1.smithi192.stdout:2/176: dread d4/d9/f18 [0,4194304] 0 2024-04-24T22:45:17.591 INFO:tasks.workunit.client.1.smithi192.stdout:7/99: chown d2/db/ld 7 1 2024-04-24T22:45:17.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/100: truncate d2/f17 640699 0 2024-04-24T22:45:17.597 INFO:tasks.workunit.client.1.smithi192.stdout:3/94: rmdir d2 39 2024-04-24T22:45:17.610 INFO:tasks.workunit.client.1.smithi192.stdout:9/91: dwrite d3/fa [4194304,4194304] 0 2024-04-24T22:45:17.612 INFO:tasks.workunit.client.1.smithi192.stdout:1/126: unlink d9/d15/l25 0 2024-04-24T22:45:17.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/101: rmdir d2/db 39 2024-04-24T22:45:17.623 INFO:tasks.workunit.client.1.smithi192.stdout:5/68: chown d0/fb 20622 1 2024-04-24T22:45:17.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/183: mknod d3/d27/d28/d2e/d3f/c41 0 2024-04-24T22:45:17.630 INFO:tasks.workunit.client.1.smithi192.stdout:9/92: symlink d3/dd/de/l1b 0 2024-04-24T22:45:17.638 INFO:tasks.workunit.client.1.smithi192.stdout:3/95: unlink d2/d4/d8/f19 0 2024-04-24T22:45:17.641 INFO:tasks.workunit.client.1.smithi192.stdout:5/69: rename d0/fe to d0/d13/f14 0 2024-04-24T22:45:17.643 INFO:tasks.workunit.client.1.smithi192.stdout:8/66: link f5 db/f10 0 2024-04-24T22:45:17.644 INFO:tasks.workunit.client.1.smithi192.stdout:2/177: write d4/d9/f2c [889018,39878] 0 2024-04-24T22:45:17.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/178: write d4/d9/f18 [656294,50978] 0 2024-04-24T22:45:17.651 INFO:tasks.workunit.client.1.smithi192.stdout:8/67: dwrite f3 [4194304,4194304] 0 2024-04-24T22:45:17.653 INFO:tasks.workunit.client.1.smithi192.stdout:8/68: truncate f8 4652475 0 2024-04-24T22:45:17.653 INFO:tasks.workunit.client.1.smithi192.stdout:3/96: creat d2/f1e x:0 0 0 2024-04-24T22:45:17.654 INFO:tasks.workunit.client.1.smithi192.stdout:8/69: chown f9 392139 1 2024-04-24T22:45:17.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/97: stat d2/d4/d12 0 2024-04-24T22:45:17.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/102: dwrite d2/db/f14 [0,4194304] 0 2024-04-24T22:45:17.660 INFO:tasks.workunit.client.1.smithi192.stdout:5/70: creat d0/f15 x:0 0 0 2024-04-24T22:45:17.665 INFO:tasks.workunit.client.1.smithi192.stdout:0/184: fdatasync d3/f8 0 2024-04-24T22:45:17.669 INFO:tasks.workunit.client.1.smithi192.stdout:2/179: fdatasync d4/d9/f2d 0 2024-04-24T22:45:17.669 INFO:tasks.workunit.client.1.smithi192.stdout:0/185: dread d3/d25/f3d [0,4194304] 0 2024-04-24T22:45:17.670 INFO:tasks.workunit.client.1.smithi192.stdout:0/186: read - d3/d27/d28/f20 zero size 2024-04-24T22:45:17.677 INFO:tasks.workunit.client.1.smithi192.stdout:0/187: dwrite d3/f14 [0,4194304] 0 2024-04-24T22:45:17.677 INFO:tasks.workunit.client.1.smithi192.stdout:8/70: creat db/dc/f11 x:0 0 0 2024-04-24T22:45:17.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/98: symlink d2/d4/d8/l1f 0 2024-04-24T22:45:17.681 INFO:tasks.workunit.client.1.smithi192.stdout:3/99: readlink d2/d4/d8/l1f 0 2024-04-24T22:45:17.682 INFO:tasks.workunit.client.1.smithi192.stdout:4/53: sync 2024-04-24T22:45:17.684 INFO:tasks.workunit.client.1.smithi192.stdout:3/100: dread d2/d4/d7/fd [0,4194304] 0 2024-04-24T22:45:17.685 INFO:tasks.workunit.client.1.smithi192.stdout:3/101: readlink d2/d4/d8/l1f 0 2024-04-24T22:45:17.686 INFO:tasks.workunit.client.1.smithi192.stdout:5/71: chown d0/d6/fd 22 1 2024-04-24T22:45:17.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/102: write d2/d4/d12/f1b [1617550,114627] 0 2024-04-24T22:45:17.690 INFO:tasks.workunit.client.1.smithi192.stdout:5/72: dread d0/d13/f14 [0,4194304] 0 2024-04-24T22:45:17.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/93: link d3/dd/c15 d3/dd/de/c1c 0 2024-04-24T22:45:17.693 INFO:tasks.workunit.client.1.smithi192.stdout:2/180: chown d4/d9/cd 368625 1 2024-04-24T22:45:17.694 INFO:tasks.workunit.client.1.smithi192.stdout:9/94: dread d3/dd/de/f16 [0,4194304] 0 2024-04-24T22:45:17.697 INFO:tasks.workunit.client.1.smithi192.stdout:1/127: link d9/c11 d9/dd/d21/c2a 0 2024-04-24T22:45:17.698 INFO:tasks.workunit.client.1.smithi192.stdout:1/128: chown d9/dd/d21 2648 1 2024-04-24T22:45:17.702 INFO:tasks.workunit.client.1.smithi192.stdout:0/188: dread d3/d27/d28/f19 [0,4194304] 0 2024-04-24T22:45:17.704 INFO:tasks.workunit.client.1.smithi192.stdout:2/181: dwrite d4/d29/f26 [0,4194304] 0 2024-04-24T22:45:17.708 INFO:tasks.workunit.client.1.smithi192.stdout:9/95: dwrite d3/fb [0,4194304] 0 2024-04-24T22:45:17.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/103: symlink d2/d4/d12/l20 0 2024-04-24T22:45:17.728 INFO:tasks.workunit.client.0.smithi063.stdout:4/132: link f0 d2/d8/f2c 0 2024-04-24T22:45:17.732 INFO:tasks.workunit.client.1.smithi192.stdout:0/189: creat d3/d27/d28/f42 x:0 0 0 2024-04-24T22:45:17.735 INFO:tasks.workunit.client.1.smithi192.stdout:7/103: rename d2/l4 to d2/l1d 0 2024-04-24T22:45:17.742 INFO:tasks.workunit.client.0.smithi063.stdout:5/154: mkdir d5/d24/d36 0 2024-04-24T22:45:17.743 INFO:tasks.workunit.client.1.smithi192.stdout:3/104: dread d2/d4/d7/fd [0,4194304] 0 2024-04-24T22:45:17.744 INFO:tasks.workunit.client.0.smithi063.stdout:2/138: write d0/d2/d3/d7/fc [1756572,84301] 0 2024-04-24T22:45:17.746 INFO:tasks.workunit.client.0.smithi063.stdout:2/139: write d0/dd/f22 [1006637,63201] 0 2024-04-24T22:45:17.746 INFO:tasks.workunit.client.0.smithi063.stdout:0/146: mknod d6/dd/d20/d24/c38 0 2024-04-24T22:45:17.747 INFO:tasks.workunit.client.1.smithi192.stdout:5/73: dwrite d0/d6/fd [0,4194304] 0 2024-04-24T22:45:17.750 INFO:tasks.workunit.client.0.smithi063.stdout:3/167: dwrite da/f28 [0,4194304] 0 2024-04-24T22:45:17.751 INFO:tasks.workunit.client.0.smithi063.stdout:7/127: mkdir d0/d4/de/d17/d28/d31 0 2024-04-24T22:45:17.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/54: write d2/db/d10/f11 [767287,15991] 0 2024-04-24T22:45:17.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/71: sync 2024-04-24T22:45:17.756 INFO:tasks.workunit.client.1.smithi192.stdout:5/74: dwrite d0/f10 [0,4194304] 0 2024-04-24T22:45:17.759 INFO:tasks.workunit.client.1.smithi192.stdout:6/72: read - d0/f19 zero size 2024-04-24T22:45:17.762 INFO:tasks.workunit.client.0.smithi063.stdout:0/147: dwrite d6/da/dc/f17 [0,4194304] 0 2024-04-24T22:45:17.765 INFO:tasks.workunit.client.1.smithi192.stdout:2/182: truncate d4/d9/fb 2195954 0 2024-04-24T22:45:17.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/73: dread d0/d8/fa [0,4194304] 0 2024-04-24T22:45:17.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/148: fsync d6/dd/d20/d24/f2e 0 2024-04-24T22:45:17.766 INFO:tasks.workunit.client.1.smithi192.stdout:9/96: creat d3/dd/de/d17/f1d x:0 0 0 2024-04-24T22:45:17.767 INFO:tasks.workunit.client.0.smithi063.stdout:1/140: dwrite d3/f2c [0,4194304] 0 2024-04-24T22:45:17.769 INFO:tasks.workunit.client.1.smithi192.stdout:9/97: stat d3/dd/de/f1a 0 2024-04-24T22:45:17.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/133: dread d2/fd [0,4194304] 0 2024-04-24T22:45:17.770 INFO:tasks.workunit.client.0.smithi063.stdout:6/149: link db/f2a db/d1d/d2d/f33 0 2024-04-24T22:45:17.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/149: dread - d6/da/dc/f1e zero size 2024-04-24T22:45:17.775 INFO:tasks.workunit.client.1.smithi192.stdout:1/129: creat d9/dd/f2b x:0 0 0 2024-04-24T22:45:17.776 INFO:tasks.workunit.client.0.smithi063.stdout:5/155: mknod d5/d11/d18/c37 0 2024-04-24T22:45:17.776 INFO:tasks.workunit.client.0.smithi063.stdout:0/150: write d6/da/dc/f13 [1959354,95648] 0 2024-04-24T22:45:17.776 INFO:tasks.workunit.client.0.smithi063.stdout:8/175: mkdir d1/d22/d33/d3b 0 2024-04-24T22:45:17.778 INFO:tasks.workunit.client.1.smithi192.stdout:3/105: rename d2/d4/d12/l20 to d2/d4/l21 0 2024-04-24T22:45:17.780 INFO:tasks.workunit.client.0.smithi063.stdout:3/168: readlink da/lb 0 2024-04-24T22:45:17.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/71: getdents db/dc 0 2024-04-24T22:45:17.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/72: chown db/dc/ld 54009837 1 2024-04-24T22:45:17.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/106: truncate d2/d4/d12/f1b 1885026 0 2024-04-24T22:45:17.782 INFO:tasks.workunit.client.0.smithi063.stdout:3/169: chown da/lb 24 1 2024-04-24T22:45:17.782 INFO:tasks.workunit.client.1.smithi192.stdout:8/73: chown f5 223450 1 2024-04-24T22:45:17.784 INFO:tasks.workunit.client.1.smithi192.stdout:4/55: symlink d2/db/d10/l1a 0 2024-04-24T22:45:17.784 INFO:tasks.workunit.client.1.smithi192.stdout:4/56: stat d2/db/d10/c15 0 2024-04-24T22:45:17.784 INFO:tasks.workunit.client.1.smithi192.stdout:7/104: mknod d2/c1e 0 2024-04-24T22:45:17.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/75: symlink d0/l16 0 2024-04-24T22:45:17.789 INFO:tasks.workunit.client.0.smithi063.stdout:1/141: write d3/f28 [5777753,65713] 0 2024-04-24T22:45:17.791 INFO:tasks.workunit.client.1.smithi192.stdout:6/74: mknod d0/d8/c1d 0 2024-04-24T22:45:17.792 INFO:tasks.workunit.client.1.smithi192.stdout:6/75: write d0/d8/fc [2259443,122282] 0 2024-04-24T22:45:17.792 INFO:tasks.workunit.client.1.smithi192.stdout:9/98: symlink d3/dd/de/l1e 0 2024-04-24T22:45:17.794 INFO:tasks.workunit.client.0.smithi063.stdout:6/150: symlink db/d1d/d20/d29/l34 0 2024-04-24T22:45:17.795 INFO:tasks.workunit.client.0.smithi063.stdout:6/151: read f8 [259040,47496] 0 2024-04-24T22:45:17.796 INFO:tasks.workunit.client.0.smithi063.stdout:6/152: dread - db/d1d/d2d/f33 zero size 2024-04-24T22:45:17.797 INFO:tasks.workunit.client.0.smithi063.stdout:0/151: rmdir d6/da/dc 39 2024-04-24T22:45:17.798 INFO:tasks.workunit.client.1.smithi192.stdout:6/76: dread d0/d8/fc [0,4194304] 0 2024-04-24T22:45:17.799 INFO:tasks.workunit.client.1.smithi192.stdout:0/190: creat d3/d27/f43 x:0 0 0 2024-04-24T22:45:17.800 INFO:tasks.workunit.client.1.smithi192.stdout:6/77: chown d0/d8/fe 364704 1 2024-04-24T22:45:17.801 INFO:tasks.workunit.client.1.smithi192.stdout:1/130: dread d9/f14 [0,4194304] 0 2024-04-24T22:45:17.801 INFO:tasks.workunit.client.0.smithi063.stdout:5/156: rename d5/d11/d18/l25 to d5/d11/d23/d2d/l38 0 2024-04-24T22:45:17.801 INFO:tasks.workunit.client.0.smithi063.stdout:5/157: chown d5/d11 21659 1 2024-04-24T22:45:17.805 INFO:tasks.workunit.client.0.smithi063.stdout:8/176: mkdir d1/d11/d3c 0 2024-04-24T22:45:17.807 INFO:tasks.workunit.client.1.smithi192.stdout:3/107: rename d2/d4/d8 to d2/d4/d12/d1a/d22 0 2024-04-24T22:45:17.809 INFO:tasks.workunit.client.1.smithi192.stdout:6/78: dread d0/d8/f14 [0,4194304] 0 2024-04-24T22:45:17.810 INFO:tasks.workunit.client.0.smithi063.stdout:2/140: truncate d0/d2/f1b 400354 0 2024-04-24T22:45:17.811 INFO:tasks.workunit.client.1.smithi192.stdout:6/79: write d0/d8/f17 [449459,78219] 0 2024-04-24T22:45:17.813 INFO:tasks.workunit.client.1.smithi192.stdout:7/105: readlink d2/l3 0 2024-04-24T22:45:17.815 INFO:tasks.workunit.client.0.smithi063.stdout:9/138: creat d6/f30 x:0 0 0 2024-04-24T22:45:17.816 INFO:tasks.workunit.client.1.smithi192.stdout:3/108: dwrite d2/d4/d7/f9 [0,4194304] 0 2024-04-24T22:45:17.817 INFO:tasks.workunit.client.1.smithi192.stdout:3/109: chown d2/d4/d7/fd 196693 1 2024-04-24T22:45:17.819 INFO:tasks.workunit.client.0.smithi063.stdout:5/158: dread f1 [0,4194304] 0 2024-04-24T22:45:17.819 INFO:tasks.workunit.client.1.smithi192.stdout:5/76: mkdir d0/d17 0 2024-04-24T22:45:17.819 INFO:tasks.workunit.client.1.smithi192.stdout:5/77: truncate d0/f15 676465 0 2024-04-24T22:45:17.822 INFO:tasks.workunit.client.1.smithi192.stdout:2/183: mkdir d4/d9/d36 0 2024-04-24T22:45:17.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/134: mknod d2/c2d 0 2024-04-24T22:45:17.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/139: dwrite d6/d8/d1f/d20/f21 [0,4194304] 0 2024-04-24T22:45:17.826 INFO:tasks.workunit.client.0.smithi063.stdout:4/135: fdatasync d2/d8/f10 0 2024-04-24T22:45:17.826 INFO:tasks.workunit.client.0.smithi063.stdout:4/136: chown d2/d8/f10 2037250 1 2024-04-24T22:45:17.827 INFO:tasks.workunit.client.0.smithi063.stdout:4/137: chown d2/d19/f20 2684 1 2024-04-24T22:45:17.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/138: write d2/d19/f25 [192278,112025] 0 2024-04-24T22:45:17.831 INFO:tasks.workunit.client.0.smithi063.stdout:4/139: write d2/f13 [1153479,54669] 0 2024-04-24T22:45:17.833 INFO:tasks.workunit.client.1.smithi192.stdout:9/99: symlink d3/dd/de/l1f 0 2024-04-24T22:45:17.837 INFO:tasks.workunit.client.0.smithi063.stdout:4/140: dread d2/f6 [0,4194304] 0 2024-04-24T22:45:17.837 INFO:tasks.workunit.client.0.smithi063.stdout:0/152: mknod d6/dd/d20/d21/d25/d31/c39 0 2024-04-24T22:45:17.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/153: fsync d6/f7 0 2024-04-24T22:45:17.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/141: chown d2/d8/f1a 15192 1 2024-04-24T22:45:17.843 INFO:tasks.workunit.client.1.smithi192.stdout:0/191: truncate d3/f2c 1310422 0 2024-04-24T22:45:17.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/177: creat d1/d2/d17/f3d x:0 0 0 2024-04-24T22:45:17.848 INFO:tasks.workunit.client.1.smithi192.stdout:8/74: dwrite f7 [0,4194304] 0 2024-04-24T22:45:17.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/131: fsync d9/f1a 0 2024-04-24T22:45:17.856 INFO:tasks.workunit.client.1.smithi192.stdout:0/192: dwrite d3/f14 [4194304,4194304] 0 2024-04-24T22:45:17.857 INFO:tasks.workunit.client.1.smithi192.stdout:0/193: fdatasync d3/d27/d28/f20 0 2024-04-24T22:45:17.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/128: link d0/c24 d0/d4/de/d17/d28/d31/c32 0 2024-04-24T22:45:17.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/129: stat d0/d2f 0 2024-04-24T22:45:17.859 INFO:tasks.workunit.client.1.smithi192.stdout:0/194: dread - d3/f39 zero size 2024-04-24T22:45:17.860 INFO:tasks.workunit.client.0.smithi063.stdout:5/159: creat d5/d11/d18/f39 x:0 0 0 2024-04-24T22:45:17.861 INFO:tasks.workunit.client.1.smithi192.stdout:7/106: symlink d2/db/l1f 0 2024-04-24T22:45:17.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/80: rename d0/d8/c1d to d0/d8/c1e 0 2024-04-24T22:45:17.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/153: dwrite db/f2a [0,4194304] 0 2024-04-24T22:45:17.867 INFO:tasks.workunit.client.0.smithi063.stdout:7/130: dread d0/d4/de/f21 [0,4194304] 0 2024-04-24T22:45:17.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/184: dread - d4/f2a zero size 2024-04-24T22:45:17.871 INFO:tasks.workunit.client.1.smithi192.stdout:2/185: read d4/d29/f26 [3204854,72971] 0 2024-04-24T22:45:17.879 INFO:tasks.workunit.client.1.smithi192.stdout:8/75: rename db/lf to db/dc/l12 0 2024-04-24T22:45:17.879 INFO:tasks.workunit.client.1.smithi192.stdout:8/76: read f5 [1576724,21350] 0 2024-04-24T22:45:17.885 INFO:tasks.workunit.client.0.smithi063.stdout:3/170: creat da/f36 x:0 0 0 2024-04-24T22:45:17.889 INFO:tasks.workunit.client.1.smithi192.stdout:7/107: creat d2/db/f20 x:0 0 0 2024-04-24T22:45:17.892 INFO:tasks.workunit.client.0.smithi063.stdout:5/160: unlink d5/f28 0 2024-04-24T22:45:17.893 INFO:tasks.workunit.client.1.smithi192.stdout:3/110: mknod d2/d4/d12/d1a/d1d/c23 0 2024-04-24T22:45:17.894 INFO:tasks.workunit.client.0.smithi063.stdout:6/154: unlink db/d1d/d20/f28 0 2024-04-24T22:45:17.895 INFO:tasks.workunit.client.0.smithi063.stdout:9/140: getdents d6/d8/db/d2b 0 2024-04-24T22:45:17.897 INFO:tasks.workunit.client.0.smithi063.stdout:0/154: mkdir d6/d3a 0 2024-04-24T22:45:17.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/57: link d2/l8 d2/db/l1b 0 2024-04-24T22:45:17.911 INFO:tasks.workunit.client.1.smithi192.stdout:0/195: mknod d3/d27/d3c/d3b/c44 0 2024-04-24T22:45:17.914 INFO:tasks.workunit.client.1.smithi192.stdout:7/108: mkdir d2/db/d21 0 2024-04-24T22:45:17.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/141: mkdir d6/d8/d1f/d20/d22/d31 0 2024-04-24T22:45:17.916 INFO:tasks.workunit.client.1.smithi192.stdout:7/109: write d2/fa [874762,4961] 0 2024-04-24T22:45:17.917 INFO:tasks.workunit.client.0.smithi063.stdout:1/142: sync 2024-04-24T22:45:17.918 INFO:tasks.workunit.client.0.smithi063.stdout:6/155: dwrite db/f10 [0,4194304] 0 2024-04-24T22:45:17.920 INFO:tasks.workunit.client.0.smithi063.stdout:0/155: creat d6/dd/d20/f3b x:0 0 0 2024-04-24T22:45:17.920 INFO:tasks.workunit.client.0.smithi063.stdout:0/156: chown d6/dd 39485099 1 2024-04-24T22:45:17.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/81: dwrite d0/d8/fa [0,4194304] 0 2024-04-24T22:45:17.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/157: write d6/da/f29 [121469,21705] 0 2024-04-24T22:45:17.922 INFO:tasks.workunit.client.1.smithi192.stdout:6/82: readlink d0/d8/l13 0 2024-04-24T22:45:17.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/111: creat d2/d4/d12/d1a/d22/f24 x:0 0 0 2024-04-24T22:45:17.928 INFO:tasks.workunit.client.0.smithi063.stdout:2/141: getdents d0/dd 0 2024-04-24T22:45:17.928 INFO:tasks.workunit.client.0.smithi063.stdout:2/142: dread - d0/d2/f1d zero size 2024-04-24T22:45:17.930 INFO:tasks.workunit.client.0.smithi063.stdout:3/171: symlink da/l37 0 2024-04-24T22:45:17.932 INFO:tasks.workunit.client.1.smithi192.stdout:2/186: mkdir d4/d9/d37 0 2024-04-24T22:45:17.934 INFO:tasks.workunit.client.0.smithi063.stdout:2/143: dread d0/dd/f11 [0,4194304] 0 2024-04-24T22:45:17.938 INFO:tasks.workunit.client.1.smithi192.stdout:8/77: rename db/f10 to db/dc/f13 0 2024-04-24T22:45:17.942 INFO:tasks.workunit.client.0.smithi063.stdout:1/143: fsync d3/f7 0 2024-04-24T22:45:17.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/144: dread d3/f17 [0,4194304] 0 2024-04-24T22:45:17.946 INFO:tasks.workunit.client.1.smithi192.stdout:8/78: dread f7 [0,4194304] 0 2024-04-24T22:45:17.947 INFO:tasks.workunit.client.1.smithi192.stdout:8/79: chown db/dc/le 9646351 1 2024-04-24T22:45:17.953 INFO:tasks.workunit.client.0.smithi063.stdout:6/156: mknod db/d1c/c35 0 2024-04-24T22:45:17.957 INFO:tasks.workunit.client.0.smithi063.stdout:6/157: dread db/d1d/d2d/f33 [0,4194304] 0 2024-04-24T22:45:17.957 INFO:tasks.workunit.client.0.smithi063.stdout:6/158: chown db/c11 0 1 2024-04-24T22:45:18.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/142: sync 2024-04-24T22:45:18.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/143: readlink d2/l18 0 2024-04-24T22:45:18.006 INFO:tasks.workunit.client.0.smithi063.stdout:8/178: rename d1/d2/d3/ca to d1/c3e 0 2024-04-24T22:45:18.007 INFO:tasks.workunit.client.0.smithi063.stdout:8/179: chown d1/d21 554836 1 2024-04-24T22:45:18.008 INFO:tasks.workunit.client.0.smithi063.stdout:8/180: write d1/f2c [213514,15463] 0 2024-04-24T22:45:18.008 INFO:tasks.workunit.client.1.smithi192.stdout:5/78: getdents d0/d13 0 2024-04-24T22:45:18.009 INFO:tasks.workunit.client.0.smithi063.stdout:2/144: mkdir d0/d2/d27 0 2024-04-24T22:45:18.009 INFO:tasks.workunit.client.0.smithi063.stdout:8/181: fsync d1/f2c 0 2024-04-24T22:45:18.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/142: mknod d6/d8/db/d1c/c32 0 2024-04-24T22:45:18.012 INFO:tasks.workunit.client.1.smithi192.stdout:1/132: getdents d9/d1c 0 2024-04-24T22:45:18.013 INFO:tasks.workunit.client.0.smithi063.stdout:8/182: dread d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:18.013 INFO:tasks.workunit.client.1.smithi192.stdout:1/133: stat d9 0 2024-04-24T22:45:18.014 INFO:tasks.workunit.client.0.smithi063.stdout:5/161: truncate d5/f16 3333 0 2024-04-24T22:45:18.014 INFO:tasks.workunit.client.1.smithi192.stdout:5/79: dwrite d0/f10 [0,4194304] 0 2024-04-24T22:45:18.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/134: chown d9/f12 954100 1 2024-04-24T22:45:18.016 INFO:tasks.workunit.client.1.smithi192.stdout:4/58: rename d2/d9/l14 to d2/d9/l1c 0 2024-04-24T22:45:18.016 INFO:tasks.workunit.client.1.smithi192.stdout:4/59: dread - d2/f17 zero size 2024-04-24T22:45:18.019 INFO:tasks.workunit.client.0.smithi063.stdout:8/183: dwrite d1/d22/f2e [0,4194304] 0 2024-04-24T22:45:18.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/110: creat d2/db/d21/f22 x:0 0 0 2024-04-24T22:45:18.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/80: dread d0/d13/f14 [0,4194304] 0 2024-04-24T22:45:18.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/83: mknod d0/c1f 0 2024-04-24T22:45:18.024 INFO:tasks.workunit.client.0.smithi063.stdout:1/145: fsync d3/f17 0 2024-04-24T22:45:18.028 INFO:tasks.workunit.client.1.smithi192.stdout:3/112: link d2/d4/d7/f9 d2/d4/d12/d1a/d1d/f25 0 2024-04-24T22:45:18.030 INFO:tasks.workunit.client.1.smithi192.stdout:3/113: rename d2/d4/d12 to d2/d4/d12/d1a/d22/d26 22 2024-04-24T22:45:18.042 INFO:tasks.workunit.client.0.smithi063.stdout:7/131: rename d0/d4/de/c2c to d0/d4/de/d17/c33 0 2024-04-24T22:45:18.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/172: rename da/d18/c1f to da/d18/c38 0 2024-04-24T22:45:18.049 INFO:tasks.workunit.client.1.smithi192.stdout:4/60: mknod d2/db/d10/c1d 0 2024-04-24T22:45:18.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/114: dwrite d2/d4/d12/d1a/d22/f24 [0,4194304] 0 2024-04-24T22:45:18.055 INFO:tasks.workunit.client.0.smithi063.stdout:6/159: dwrite db/f1a [0,4194304] 0 2024-04-24T22:45:18.056 INFO:tasks.workunit.client.0.smithi063.stdout:6/160: chown db/d1d/d2d/f33 1 1 2024-04-24T22:45:18.057 INFO:tasks.workunit.client.1.smithi192.stdout:2/187: write d4/f31 [851818,26039] 0 2024-04-24T22:45:18.058 INFO:tasks.workunit.client.1.smithi192.stdout:6/84: creat d0/d8/f20 x:0 0 0 2024-04-24T22:45:18.062 INFO:tasks.workunit.client.1.smithi192.stdout:8/80: creat db/f14 x:0 0 0 2024-04-24T22:45:18.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/144: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:18.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/188: dread d4/d9/d10/f11 [0,4194304] 0 2024-04-24T22:45:18.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/189: readlink d4/d9/d10/l25 0 2024-04-24T22:45:18.071 INFO:tasks.workunit.client.0.smithi063.stdout:2/145: write d0/dd/f11 [4540298,32465] 0 2024-04-24T22:45:18.077 INFO:tasks.workunit.client.1.smithi192.stdout:7/111: dwrite d2/f5 [0,4194304] 0 2024-04-24T22:45:18.077 INFO:tasks.workunit.client.0.smithi063.stdout:0/158: rename d6/l8 to d6/da/dc/d26/l3c 0 2024-04-24T22:45:18.078 INFO:tasks.workunit.client.1.smithi192.stdout:7/112: chown f1 0 1 2024-04-24T22:45:18.083 INFO:tasks.workunit.client.1.smithi192.stdout:3/115: write d2/d4/d12/d1a/d1d/f25 [2885393,68285] 0 2024-04-24T22:45:18.085 INFO:tasks.workunit.client.1.smithi192.stdout:3/116: write d2/d4/f16 [596101,32023] 0 2024-04-24T22:45:18.086 INFO:tasks.workunit.client.1.smithi192.stdout:1/135: rename d9/f12 to d9/f2c 0 2024-04-24T22:45:18.087 INFO:tasks.workunit.client.0.smithi063.stdout:6/161: unlink db/f12 0 2024-04-24T22:45:18.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/190: fdatasync d4/d9/d10/f23 0 2024-04-24T22:45:18.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/143: creat d6/d8/d1f/d20/d22/d31/f33 x:0 0 0 2024-04-24T22:45:18.093 INFO:tasks.workunit.client.1.smithi192.stdout:5/81: truncate d0/f10 2860661 0 2024-04-24T22:45:18.094 INFO:tasks.workunit.client.0.smithi063.stdout:7/132: write d0/d4/de/f21 [2853268,52726] 0 2024-04-24T22:45:18.094 INFO:tasks.workunit.client.1.smithi192.stdout:5/82: dread d0/f15 [0,4194304] 0 2024-04-24T22:45:18.097 INFO:tasks.workunit.client.0.smithi063.stdout:9/144: dwrite d6/d8/d1f/d20/f21 [0,4194304] 0 2024-04-24T22:45:18.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/145: write d6/f30 [844553,27752] 0 2024-04-24T22:45:18.102 INFO:tasks.workunit.client.0.smithi063.stdout:4/145: dread d2/f26 [0,4194304] 0 2024-04-24T22:45:18.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/85: symlink d0/l21 0 2024-04-24T22:45:18.102 INFO:tasks.workunit.client.0.smithi063.stdout:4/146: readlink d2/l1b 0 2024-04-24T22:45:18.103 INFO:tasks.workunit.client.0.smithi063.stdout:2/146: mknod d0/d2/d3/d7/c28 0 2024-04-24T22:45:18.104 INFO:tasks.workunit.client.0.smithi063.stdout:2/147: fdatasync d0/d2/f1d 0 2024-04-24T22:45:18.109 INFO:tasks.workunit.client.0.smithi063.stdout:0/159: chown d6/da/dc/c1d 406381766 1 2024-04-24T22:45:18.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/160: dread - d6/dd/d20/f3b zero size 2024-04-24T22:45:18.110 INFO:tasks.workunit.client.1.smithi192.stdout:3/117: mkdir d2/d27 0 2024-04-24T22:45:18.111 INFO:tasks.workunit.client.0.smithi063.stdout:0/161: readlink d6/l9 0 2024-04-24T22:45:18.112 INFO:tasks.workunit.client.0.smithi063.stdout:3/173: mkdir da/d39 0 2024-04-24T22:45:18.112 INFO:tasks.workunit.client.1.smithi192.stdout:3/118: dread d2/d4/d7/fd [0,4194304] 0 2024-04-24T22:45:18.113 INFO:tasks.workunit.client.0.smithi063.stdout:3/174: chown da/d18/f27 49 1 2024-04-24T22:45:18.113 INFO:tasks.workunit.client.1.smithi192.stdout:3/119: chown d2/d4/d12/d1a/d1d 100 1 2024-04-24T22:45:18.115 INFO:tasks.workunit.client.0.smithi063.stdout:4/147: dread f1 [0,4194304] 0 2024-04-24T22:45:18.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/61: rmdir d2/db/dc 0 2024-04-24T22:45:18.117 INFO:tasks.workunit.client.1.smithi192.stdout:4/62: readlink d2/db/d10/l1a 0 2024-04-24T22:45:18.117 INFO:tasks.workunit.client.1.smithi192.stdout:4/63: fsync d2/f17 0 2024-04-24T22:45:18.120 INFO:tasks.workunit.client.1.smithi192.stdout:3/120: dwrite d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:18.122 INFO:tasks.workunit.client.0.smithi063.stdout:8/184: rmdir d1/d22/d33/d3b 0 2024-04-24T22:45:18.122 INFO:tasks.workunit.client.0.smithi063.stdout:4/148: dwrite d2/d19/f20 [0,4194304] 0 2024-04-24T22:45:18.123 INFO:tasks.workunit.client.0.smithi063.stdout:4/149: chown d2/c12 8336899 1 2024-04-24T22:45:18.123 INFO:tasks.workunit.client.0.smithi063.stdout:4/150: read d2/f13 [425680,53027] 0 2024-04-24T22:45:18.123 INFO:tasks.workunit.client.1.smithi192.stdout:6/86: mkdir d0/d8/d22 0 2024-04-24T22:45:18.125 INFO:tasks.workunit.client.1.smithi192.stdout:6/87: chown d0/c5 301801315 1 2024-04-24T22:45:18.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/88: fsync d0/f19 0 2024-04-24T22:45:18.129 INFO:tasks.workunit.client.1.smithi192.stdout:8/81: link f8 db/f15 0 2024-04-24T22:45:18.131 INFO:tasks.workunit.client.0.smithi063.stdout:5/162: rename d5/f16 to d5/d11/d18/f3a 0 2024-04-24T22:45:18.133 INFO:tasks.workunit.client.1.smithi192.stdout:6/89: dread d0/d8/fe [0,4194304] 0 2024-04-24T22:45:18.134 INFO:tasks.workunit.client.1.smithi192.stdout:8/82: dread f3 [0,4194304] 0 2024-04-24T22:45:18.134 INFO:tasks.workunit.client.0.smithi063.stdout:3/175: mkdir da/d18/d3a 0 2024-04-24T22:45:18.135 INFO:tasks.workunit.client.0.smithi063.stdout:6/162: creat db/d1d/d2d/f36 x:0 0 0 2024-04-24T22:45:18.138 INFO:tasks.workunit.client.1.smithi192.stdout:4/64: mkdir d2/d9/d1e 0 2024-04-24T22:45:18.138 INFO:tasks.workunit.client.1.smithi192.stdout:4/65: chown d2/db/d10/l1a 125871 1 2024-04-24T22:45:18.139 INFO:tasks.workunit.client.0.smithi063.stdout:7/133: symlink d0/d26/d2e/l34 0 2024-04-24T22:45:18.140 INFO:tasks.workunit.client.1.smithi192.stdout:7/113: rename d2/f5 to d2/f23 0 2024-04-24T22:45:18.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/136: truncate d9/f13 1141268 0 2024-04-24T22:45:18.145 INFO:tasks.workunit.client.1.smithi192.stdout:7/114: dwrite d2/db/f20 [0,4194304] 0 2024-04-24T22:45:18.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/146: rename d3/da/dc/d1f/c26 to d3/c31 0 2024-04-24T22:45:18.152 INFO:tasks.workunit.client.1.smithi192.stdout:5/83: rmdir d0/d17 0 2024-04-24T22:45:18.153 INFO:tasks.workunit.client.0.smithi063.stdout:2/148: write d0/d2/d3/d7/f19 [1390626,70686] 0 2024-04-24T22:45:18.155 INFO:tasks.workunit.client.0.smithi063.stdout:1/147: dwrite d3/da/dc/d1f/f30 [0,4194304] 0 2024-04-24T22:45:18.156 INFO:tasks.workunit.client.0.smithi063.stdout:6/163: rename db/f25 to db/f37 0 2024-04-24T22:45:18.156 INFO:tasks.workunit.client.0.smithi063.stdout:6/164: fsync db/f10 0 2024-04-24T22:45:18.158 INFO:tasks.workunit.client.0.smithi063.stdout:7/134: mkdir d0/d4/de/d17/d35 0 2024-04-24T22:45:18.159 INFO:tasks.workunit.client.1.smithi192.stdout:3/121: mkdir d2/d27/d28 0 2024-04-24T22:45:18.159 INFO:tasks.workunit.client.0.smithi063.stdout:7/135: chown d0/d26/d2e 78 1 2024-04-24T22:45:18.160 INFO:tasks.workunit.client.1.smithi192.stdout:3/122: chown d2/d4/d12/f1c 2032017543 1 2024-04-24T22:45:18.161 INFO:tasks.workunit.client.1.smithi192.stdout:3/123: write d2/f1e [69588,20156] 0 2024-04-24T22:45:18.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/137: rename d9/dd/d21 to d9/d15/d2d 0 2024-04-24T22:45:18.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/138: stat d9/dd/c1e 0 2024-04-24T22:45:18.173 INFO:tasks.workunit.client.0.smithi063.stdout:8/185: rename d1/d2/d17/f27 to d1/d11/d32/d38/f3f 0 2024-04-24T22:45:18.173 INFO:tasks.workunit.client.0.smithi063.stdout:5/163: mkdir d5/d30/d3b 0 2024-04-24T22:45:18.174 INFO:tasks.workunit.client.0.smithi063.stdout:0/162: creat d6/da/f3d x:0 0 0 2024-04-24T22:45:18.174 INFO:tasks.workunit.client.1.smithi192.stdout:8/83: symlink db/l16 0 2024-04-24T22:45:18.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/90: write d0/d8/f14 [1962520,124436] 0 2024-04-24T22:45:18.177 INFO:tasks.workunit.client.1.smithi192.stdout:5/84: rename d0/fb to d0/f18 0 2024-04-24T22:45:18.184 INFO:tasks.workunit.client.1.smithi192.stdout:5/85: dread d0/d13/f14 [0,4194304] 0 2024-04-24T22:45:18.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/124: rename d2/d4/d12/d1a/d22/ff to d2/f29 0 2024-04-24T22:45:18.185 INFO:tasks.workunit.client.1.smithi192.stdout:3/125: write d2/d4/d7/f9 [2009988,12002] 0 2024-04-24T22:45:18.185 INFO:tasks.workunit.client.1.smithi192.stdout:1/139: chown d9/f16 10 1 2024-04-24T22:45:18.187 INFO:tasks.workunit.client.1.smithi192.stdout:1/140: truncate d9/dd/ff 1681855 0 2024-04-24T22:45:18.188 INFO:tasks.workunit.client.1.smithi192.stdout:8/84: creat db/dc/f17 x:0 0 0 2024-04-24T22:45:18.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/196: sync 2024-04-24T22:45:18.189 INFO:tasks.workunit.client.1.smithi192.stdout:9/100: sync 2024-04-24T22:45:18.195 INFO:tasks.workunit.client.1.smithi192.stdout:4/66: creat d2/f1f x:0 0 0 2024-04-24T22:45:18.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/67: write d2/f17 [325641,18199] 0 2024-04-24T22:45:18.199 INFO:tasks.workunit.client.1.smithi192.stdout:6/91: dwrite d0/d8/fc [4194304,4194304] 0 2024-04-24T22:45:18.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/126: rename d2/d4/f15 to d2/d4/d12/d1a/d22/f2a 0 2024-04-24T22:45:18.202 INFO:tasks.workunit.client.1.smithi192.stdout:1/141: creat d9/d15/f2e x:0 0 0 2024-04-24T22:45:18.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/86: dwrite d0/f18 [4194304,4194304] 0 2024-04-24T22:45:18.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/87: read d0/f18 [4063411,74773] 0 2024-04-24T22:45:18.210 INFO:tasks.workunit.client.1.smithi192.stdout:9/101: fsync f2 0 2024-04-24T22:45:18.210 INFO:tasks.workunit.client.1.smithi192.stdout:9/102: stat d3/dd/de/l1e 0 2024-04-24T22:45:18.213 INFO:tasks.workunit.client.1.smithi192.stdout:4/68: mkdir d2/db/d20 0 2024-04-24T22:45:18.217 INFO:tasks.workunit.client.1.smithi192.stdout:4/69: write d2/db/d10/f11 [672909,90711] 0 2024-04-24T22:45:18.219 INFO:tasks.workunit.client.1.smithi192.stdout:0/197: dwrite d3/d27/d28/f3e [0,4194304] 0 2024-04-24T22:45:18.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/127: rmdir d2/d4/d12/d1a 39 2024-04-24T22:45:18.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/115: getdents d2 0 2024-04-24T22:45:18.225 INFO:tasks.workunit.client.1.smithi192.stdout:1/142: mkdir d9/d1c/d2f 0 2024-04-24T22:45:18.226 INFO:tasks.workunit.client.1.smithi192.stdout:1/143: dread - d9/dd/f2b zero size 2024-04-24T22:45:18.238 INFO:tasks.workunit.client.1.smithi192.stdout:1/144: dwrite d9/dd/f2b [0,4194304] 0 2024-04-24T22:45:18.238 INFO:tasks.workunit.client.1.smithi192.stdout:5/88: unlink d0/l16 0 2024-04-24T22:45:18.238 INFO:tasks.workunit.client.1.smithi192.stdout:8/85: dwrite db/f15 [0,4194304] 0 2024-04-24T22:45:18.239 INFO:tasks.workunit.client.1.smithi192.stdout:9/103: mknod d3/dd/de/c20 0 2024-04-24T22:45:18.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/92: mknod d0/d8/d22/c23 0 2024-04-24T22:45:18.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/93: readlink d0/l21 0 2024-04-24T22:45:18.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/94: write d0/d8/fe [8021326,6594] 0 2024-04-24T22:45:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/70: chown d2/c16 7632065 1 2024-04-24T22:45:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/191: sync 2024-04-24T22:45:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:6/95: write d0/d8/f17 [1086946,118364] 0 2024-04-24T22:45:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:0/198: fdatasync d3/d27/d28/f19 0 2024-04-24T22:45:18.253 INFO:tasks.workunit.client.1.smithi192.stdout:3/128: stat d2/d4/d12/d1a/d22/f24 0 2024-04-24T22:45:18.254 INFO:tasks.workunit.client.1.smithi192.stdout:0/199: write d3/d27/d28/f20 [437481,9495] 0 2024-04-24T22:45:18.257 INFO:tasks.workunit.client.1.smithi192.stdout:1/145: unlink d9/f16 0 2024-04-24T22:45:18.258 INFO:tasks.workunit.client.1.smithi192.stdout:4/71: dwrite d2/f1f [0,4194304] 0 2024-04-24T22:45:18.258 INFO:tasks.workunit.client.1.smithi192.stdout:0/200: chown d3/l13 28539 1 2024-04-24T22:45:18.259 INFO:tasks.workunit.client.1.smithi192.stdout:0/201: read - d3/d27/f43 zero size 2024-04-24T22:45:18.259 INFO:tasks.workunit.client.1.smithi192.stdout:5/89: stat d0/f10 0 2024-04-24T22:45:18.262 INFO:tasks.workunit.client.1.smithi192.stdout:8/86: truncate f3 8902889 0 2024-04-24T22:45:18.264 INFO:tasks.workunit.client.1.smithi192.stdout:9/104: creat d3/f21 x:0 0 0 2024-04-24T22:45:18.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/129: mknod d2/d4/d12/d1a/d22/c2b 0 2024-04-24T22:45:18.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/202: unlink d3/d27/d28/f42 0 2024-04-24T22:45:18.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/87: creat db/dc/f18 x:0 0 0 2024-04-24T22:45:18.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/88: truncate f5 4419681 0 2024-04-24T22:45:18.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/105: symlink d3/dd/de/d17/l22 0 2024-04-24T22:45:18.292 INFO:tasks.workunit.client.1.smithi192.stdout:3/130: creat d2/d4/d7/f2c x:0 0 0 2024-04-24T22:45:18.292 INFO:tasks.workunit.client.1.smithi192.stdout:7/116: truncate d2/db/f15 1399945 0 2024-04-24T22:45:18.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/72: mkdir d2/d9/d1e/d21 0 2024-04-24T22:45:18.295 INFO:tasks.workunit.client.1.smithi192.stdout:2/192: dwrite d4/d9/d10/f11 [4194304,4194304] 0 2024-04-24T22:45:18.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/117: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:18.296 INFO:tasks.workunit.client.1.smithi192.stdout:8/89: dwrite f8 [0,4194304] 0 2024-04-24T22:45:18.298 INFO:tasks.workunit.client.1.smithi192.stdout:4/73: write d2/db/d10/f11 [1530613,45278] 0 2024-04-24T22:45:18.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/146: sync 2024-04-24T22:45:18.302 INFO:tasks.workunit.client.1.smithi192.stdout:7/118: dread f1 [0,4194304] 0 2024-04-24T22:45:18.303 INFO:tasks.workunit.client.1.smithi192.stdout:7/119: fsync d2/f1a 0 2024-04-24T22:45:18.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/203: write d3/d25/f3d [2460266,121422] 0 2024-04-24T22:45:18.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/204: chown d3/d27/d28/c2d 1 1 2024-04-24T22:45:18.306 INFO:tasks.workunit.client.1.smithi192.stdout:9/106: creat d3/dd/de/d17/f23 x:0 0 0 2024-04-24T22:45:18.307 INFO:tasks.workunit.client.1.smithi192.stdout:9/107: read d3/fb [3096090,70842] 0 2024-04-24T22:45:18.308 INFO:tasks.workunit.client.1.smithi192.stdout:3/131: mkdir d2/d2d 0 2024-04-24T22:45:18.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/132: dread d2/d4/d7/f9 [0,4194304] 0 2024-04-24T22:45:18.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/146: link d9/d1c/c27 d9/d1c/d2f/c30 0 2024-04-24T22:45:18.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/147: write d9/dd/f2b [3383716,22799] 0 2024-04-24T22:45:18.318 INFO:tasks.workunit.client.1.smithi192.stdout:2/193: truncate d4/f13 3137673 0 2024-04-24T22:45:18.318 INFO:tasks.workunit.client.1.smithi192.stdout:2/194: chown d4/d9/c33 3196799 1 2024-04-24T22:45:18.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/120: unlink d2/db/f14 0 2024-04-24T22:45:18.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/205: truncate d3/d27/d28/f19 1071157 0 2024-04-24T22:45:18.330 INFO:tasks.workunit.client.1.smithi192.stdout:9/108: mkdir d3/d24 0 2024-04-24T22:45:18.331 INFO:tasks.workunit.client.1.smithi192.stdout:9/109: write d3/dd/de/d17/f1d [52453,26957] 0 2024-04-24T22:45:18.337 INFO:tasks.workunit.client.1.smithi192.stdout:1/148: rename d9/d1c to d9/d15/d2d/d31 0 2024-04-24T22:45:18.339 INFO:tasks.workunit.client.1.smithi192.stdout:8/90: symlink db/l19 0 2024-04-24T22:45:18.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/74: mknod d2/d9/d1e/d21/c22 0 2024-04-24T22:45:18.344 INFO:tasks.workunit.client.0.smithi063.stdout:6/165: sync 2024-04-24T22:45:18.344 INFO:tasks.workunit.client.1.smithi192.stdout:5/90: getdents d0/d6 0 2024-04-24T22:45:18.345 INFO:tasks.workunit.client.1.smithi192.stdout:1/149: dwrite d9/dd/f29 [0,4194304] 0 2024-04-24T22:45:18.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/91: dread f6 [0,4194304] 0 2024-04-24T22:45:18.349 INFO:tasks.workunit.client.1.smithi192.stdout:7/121: rename d2/db/f13 to d2/db/f24 0 2024-04-24T22:45:18.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/206: unlink d3/d27/d28/l18 0 2024-04-24T22:45:18.356 INFO:tasks.workunit.client.1.smithi192.stdout:7/122: dwrite d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:18.356 INFO:tasks.workunit.client.1.smithi192.stdout:7/123: fdatasync f0 0 2024-04-24T22:45:18.357 INFO:tasks.workunit.client.1.smithi192.stdout:9/110: mknod d3/dd/c25 0 2024-04-24T22:45:18.363 INFO:tasks.workunit.client.1.smithi192.stdout:3/133: link d2/d4/d12/d1a/d22/fa d2/d4/d7/f2e 0 2024-04-24T22:45:18.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/111: dwrite d3/dd/de/d17/f1d [0,4194304] 0 2024-04-24T22:45:18.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/112: stat f2 0 2024-04-24T22:45:18.374 INFO:tasks.workunit.client.1.smithi192.stdout:2/195: truncate d4/d9/d10/f23 137140 0 2024-04-24T22:45:18.374 INFO:tasks.workunit.client.1.smithi192.stdout:9/113: dwrite d3/f21 [0,4194304] 0 2024-04-24T22:45:18.377 INFO:tasks.workunit.client.1.smithi192.stdout:4/75: rename d2/db/f18 to d2/d9/d1e/d21/f23 0 2024-04-24T22:45:18.380 INFO:tasks.workunit.client.1.smithi192.stdout:5/91: write d0/f18 [7034806,99250] 0 2024-04-24T22:45:18.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/92: write d0/f18 [8554651,117635] 0 2024-04-24T22:45:18.386 INFO:tasks.workunit.client.1.smithi192.stdout:8/92: fdatasync db/dc/f13 0 2024-04-24T22:45:18.389 INFO:tasks.workunit.client.1.smithi192.stdout:0/207: creat d3/f45 x:0 0 0 2024-04-24T22:45:18.400 INFO:tasks.workunit.client.1.smithi192.stdout:3/134: dwrite d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:18.410 INFO:tasks.workunit.client.1.smithi192.stdout:9/114: creat d3/dd/de/d17/f26 x:0 0 0 2024-04-24T22:45:18.417 INFO:tasks.workunit.client.1.smithi192.stdout:5/93: rename d0/d6 to d0/d13/d19 0 2024-04-24T22:45:18.422 INFO:tasks.workunit.client.1.smithi192.stdout:8/93: dread f6 [0,4194304] 0 2024-04-24T22:45:18.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/94: dread f5 [4194304,4194304] 0 2024-04-24T22:45:18.424 INFO:tasks.workunit.client.1.smithi192.stdout:0/208: creat d3/d25/f46 x:0 0 0 2024-04-24T22:45:18.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/209: dread - d3/f39 zero size 2024-04-24T22:45:18.428 INFO:tasks.workunit.client.1.smithi192.stdout:7/124: dread d2/db/f24 [0,4194304] 0 2024-04-24T22:45:18.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/196: symlink d4/d9/d10/l38 0 2024-04-24T22:45:18.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/197: stat d4/d9/d10/l25 0 2024-04-24T22:45:18.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/198: write d4/d29/f2b [629435,83717] 0 2024-04-24T22:45:18.435 INFO:tasks.workunit.client.1.smithi192.stdout:4/76: getdents d2/db/d20 0 2024-04-24T22:45:18.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/94: rename d0/f15 to d0/d13/d19/f1a 0 2024-04-24T22:45:18.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/210: rename d3/f8 to d3/f47 0 2024-04-24T22:45:18.447 INFO:tasks.workunit.client.1.smithi192.stdout:1/150: truncate d9/dd/ff 1405531 0 2024-04-24T22:45:18.448 INFO:tasks.workunit.client.1.smithi192.stdout:1/151: write d9/dd/f29 [3760574,17080] 0 2024-04-24T22:45:18.448 INFO:tasks.workunit.client.1.smithi192.stdout:1/152: read - d9/d15/f2e zero size 2024-04-24T22:45:18.449 INFO:tasks.workunit.client.1.smithi192.stdout:7/125: mkdir d2/db/d25 0 2024-04-24T22:45:18.450 INFO:tasks.workunit.client.1.smithi192.stdout:7/126: write d2/fa [4819178,69555] 0 2024-04-24T22:45:18.454 INFO:tasks.workunit.client.1.smithi192.stdout:3/135: write d2/d4/d7/f9 [331597,80605] 0 2024-04-24T22:45:18.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/77: creat d2/d9/d1e/f24 x:0 0 0 2024-04-24T22:45:18.459 INFO:tasks.workunit.client.1.smithi192.stdout:4/78: fdatasync d2/f17 0 2024-04-24T22:45:18.460 INFO:tasks.workunit.client.1.smithi192.stdout:5/95: mkdir d0/d1b 0 2024-04-24T22:45:18.466 INFO:tasks.workunit.client.1.smithi192.stdout:5/96: dwrite d0/f18 [4194304,4194304] 0 2024-04-24T22:45:18.468 INFO:tasks.workunit.client.1.smithi192.stdout:0/211: readlink d3/d27/d28/l21 0 2024-04-24T22:45:18.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/153: creat d9/d15/d2d/d31/f32 x:0 0 0 2024-04-24T22:45:18.474 INFO:tasks.workunit.client.1.smithi192.stdout:9/115: truncate d3/dd/de/f16 2646875 0 2024-04-24T22:45:18.479 INFO:tasks.workunit.client.1.smithi192.stdout:3/136: write d2/d4/d7/fd [583145,66831] 0 2024-04-24T22:45:18.481 INFO:tasks.workunit.client.1.smithi192.stdout:7/127: dwrite d2/f17 [0,4194304] 0 2024-04-24T22:45:18.485 INFO:tasks.workunit.client.1.smithi192.stdout:8/95: truncate f8 1725949 0 2024-04-24T22:45:18.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/97: truncate d0/f10 3468285 0 2024-04-24T22:45:18.490 INFO:tasks.workunit.client.1.smithi192.stdout:6/96: sync 2024-04-24T22:45:18.491 INFO:tasks.workunit.client.1.smithi192.stdout:0/212: rename d3/d25/f2a to d3/f48 0 2024-04-24T22:45:18.493 INFO:tasks.workunit.client.1.smithi192.stdout:5/98: dwrite d0/d13/d19/fd [0,4194304] 0 2024-04-24T22:45:18.498 INFO:tasks.workunit.client.1.smithi192.stdout:7/128: mknod d2/c26 0 2024-04-24T22:45:18.520 INFO:tasks.workunit.client.1.smithi192.stdout:2/199: getdents d4/d9/d10 0 2024-04-24T22:45:18.521 INFO:tasks.workunit.client.1.smithi192.stdout:2/200: chown d4/d9/d10/f23 5617 1 2024-04-24T22:45:18.531 INFO:tasks.workunit.client.1.smithi192.stdout:6/97: unlink d0/d8/c1e 0 2024-04-24T22:45:18.531 INFO:tasks.workunit.client.1.smithi192.stdout:6/98: chown d0/d8/f17 1191482891 1 2024-04-24T22:45:18.532 INFO:tasks.workunit.client.1.smithi192.stdout:0/213: mknod d3/d27/c49 0 2024-04-24T22:45:18.533 INFO:tasks.workunit.client.1.smithi192.stdout:0/214: stat d3/f14 0 2024-04-24T22:45:18.538 INFO:tasks.workunit.client.1.smithi192.stdout:5/99: mkdir d0/d1b/d1c 0 2024-04-24T22:45:18.542 INFO:tasks.workunit.client.1.smithi192.stdout:4/79: truncate d2/f3 999793 0 2024-04-24T22:45:18.543 INFO:tasks.workunit.client.1.smithi192.stdout:4/80: truncate d2/db/d10/f11 2462016 0 2024-04-24T22:45:18.547 INFO:tasks.workunit.client.1.smithi192.stdout:8/96: write db/f15 [2178136,111207] 0 2024-04-24T22:45:18.547 INFO:tasks.workunit.client.1.smithi192.stdout:8/97: read - db/f14 zero size 2024-04-24T22:45:18.548 INFO:tasks.workunit.client.1.smithi192.stdout:7/129: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:18.549 INFO:tasks.workunit.client.1.smithi192.stdout:6/99: creat d0/d8/d22/f24 x:0 0 0 2024-04-24T22:45:18.551 INFO:tasks.workunit.client.1.smithi192.stdout:0/215: mkdir d3/d27/d28/d2e/d4a 0 2024-04-24T22:45:18.553 INFO:tasks.workunit.client.1.smithi192.stdout:1/154: link d9/c17 d9/d15/c33 0 2024-04-24T22:45:18.554 INFO:tasks.workunit.client.1.smithi192.stdout:9/116: getdents d3/dd 0 2024-04-24T22:45:18.555 INFO:tasks.workunit.client.1.smithi192.stdout:9/117: stat d3/f9 0 2024-04-24T22:45:18.556 INFO:tasks.workunit.client.1.smithi192.stdout:5/100: read d0/d13/d19/f1a [50803,98378] 0 2024-04-24T22:45:18.556 INFO:tasks.workunit.client.1.smithi192.stdout:1/155: write d9/d15/d2d/d31/f32 [407330,28072] 0 2024-04-24T22:45:18.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/137: getdents d2/d4/d12 0 2024-04-24T22:45:18.559 INFO:tasks.workunit.client.1.smithi192.stdout:2/201: symlink d4/l39 0 2024-04-24T22:45:18.559 INFO:tasks.workunit.client.1.smithi192.stdout:6/100: dwrite d0/d8/f17 [0,4194304] 0 2024-04-24T22:45:18.560 INFO:tasks.workunit.client.1.smithi192.stdout:3/138: read d2/d4/d7/f2e [745692,126916] 0 2024-04-24T22:45:18.561 INFO:tasks.workunit.client.1.smithi192.stdout:1/156: write d9/dd/f2b [4266538,39910] 0 2024-04-24T22:45:18.563 INFO:tasks.workunit.client.1.smithi192.stdout:4/81: rename d2/d9/d1e/d21 to d2/d9/d25 0 2024-04-24T22:45:18.564 INFO:tasks.workunit.client.1.smithi192.stdout:8/98: mkdir db/dc/d1a 0 2024-04-24T22:45:18.570 INFO:tasks.workunit.client.1.smithi192.stdout:2/202: dwrite d4/d9/f2d [0,4194304] 0 2024-04-24T22:45:18.571 INFO:tasks.workunit.client.1.smithi192.stdout:5/101: fsync d0/d13/d19/f1a 0 2024-04-24T22:45:18.574 INFO:tasks.workunit.client.1.smithi192.stdout:7/130: dread d2/db/f24 [0,4194304] 0 2024-04-24T22:45:18.576 INFO:tasks.workunit.client.1.smithi192.stdout:9/118: rename d3/dd/de/d17/f26 to d3/dd/de/d17/f27 0 2024-04-24T22:45:18.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/101: rename d0/d8/c16 to d0/d8/d22/c25 0 2024-04-24T22:45:18.591 INFO:tasks.workunit.client.1.smithi192.stdout:1/157: mknod d9/d15/d2d/c34 0 2024-04-24T22:45:18.591 INFO:tasks.workunit.client.1.smithi192.stdout:8/99: chown f3 28 1 2024-04-24T22:45:18.592 INFO:tasks.workunit.client.1.smithi192.stdout:0/216: creat d3/d27/d28/d2e/d4a/f4b x:0 0 0 2024-04-24T22:45:18.598 INFO:tasks.workunit.client.1.smithi192.stdout:5/102: creat d0/d13/d19/f1d x:0 0 0 2024-04-24T22:45:18.607 INFO:tasks.workunit.client.1.smithi192.stdout:3/139: mknod d2/d27/d28/c2f 0 2024-04-24T22:45:18.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/100: creat db/dc/f1b x:0 0 0 2024-04-24T22:45:18.615 INFO:tasks.workunit.client.1.smithi192.stdout:0/217: creat d3/d27/d28/d2e/f4c x:0 0 0 2024-04-24T22:45:18.616 INFO:tasks.workunit.client.1.smithi192.stdout:0/218: write d3/f39 [70602,100194] 0 2024-04-24T22:45:18.617 INFO:tasks.workunit.client.1.smithi192.stdout:2/203: symlink d4/d9/d37/l3a 0 2024-04-24T22:45:18.617 INFO:tasks.workunit.client.1.smithi192.stdout:2/204: readlink d4/d9/d10/l25 0 2024-04-24T22:45:18.617 INFO:tasks.workunit.client.1.smithi192.stdout:0/219: truncate d3/d27/f43 111768 0 2024-04-24T22:45:18.619 INFO:tasks.workunit.client.1.smithi192.stdout:5/103: unlink d0/d13/f14 0 2024-04-24T22:45:18.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/131: link d2/f6 d2/db/d25/f27 0 2024-04-24T22:45:18.622 INFO:tasks.workunit.client.1.smithi192.stdout:9/119: mknod d3/d24/c28 0 2024-04-24T22:45:18.624 INFO:tasks.workunit.client.1.smithi192.stdout:3/140: creat d2/d27/d28/f30 x:0 0 0 2024-04-24T22:45:18.627 INFO:tasks.workunit.client.1.smithi192.stdout:1/158: mkdir d9/d15/d2d/d31/d2f/d35 0 2024-04-24T22:45:18.627 INFO:tasks.workunit.client.1.smithi192.stdout:7/132: dwrite d2/db/f20 [0,4194304] 0 2024-04-24T22:45:18.628 INFO:tasks.workunit.client.1.smithi192.stdout:8/101: read f5 [3621353,58835] 0 2024-04-24T22:45:18.628 INFO:tasks.workunit.client.1.smithi192.stdout:8/102: read f9 [2958,68606] 0 2024-04-24T22:45:18.629 INFO:tasks.workunit.client.1.smithi192.stdout:8/103: stat f5 0 2024-04-24T22:45:18.632 INFO:tasks.workunit.client.1.smithi192.stdout:3/141: dwrite d2/d4/f5 [4194304,4194304] 0 2024-04-24T22:45:18.635 INFO:tasks.workunit.client.1.smithi192.stdout:2/205: creat d4/d9/d37/f3b x:0 0 0 2024-04-24T22:45:18.635 INFO:tasks.workunit.client.1.smithi192.stdout:5/104: rmdir d0/d1b 39 2024-04-24T22:45:18.640 INFO:tasks.workunit.client.1.smithi192.stdout:0/220: dread d3/f2c [0,4194304] 0 2024-04-24T22:45:18.641 INFO:tasks.workunit.client.1.smithi192.stdout:0/221: chown d3/f2c 7475734 1 2024-04-24T22:45:18.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/120: mknod d3/dd/c29 0 2024-04-24T22:45:18.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/133: write d2/f6 [994144,41436] 0 2024-04-24T22:45:18.658 INFO:tasks.workunit.client.1.smithi192.stdout:3/142: unlink d2/d4/f5 0 2024-04-24T22:45:18.658 INFO:tasks.workunit.client.1.smithi192.stdout:0/222: fsync d3/d27/d28/d2e/f4c 0 2024-04-24T22:45:18.665 INFO:tasks.workunit.client.1.smithi192.stdout:0/223: dwrite d3/d27/f43 [0,4194304] 0 2024-04-24T22:45:18.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/121: symlink d3/dd/l2a 0 2024-04-24T22:45:18.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/102: getdents d0/d8 0 2024-04-24T22:45:18.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/103: fdatasync d0/d8/fa 0 2024-04-24T22:45:18.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/159: symlink d9/dd/l36 0 2024-04-24T22:45:18.676 INFO:tasks.workunit.client.1.smithi192.stdout:7/134: creat d2/db/d21/f28 x:0 0 0 2024-04-24T22:45:18.677 INFO:tasks.workunit.client.1.smithi192.stdout:7/135: readlink d2/db/lc 0 2024-04-24T22:45:18.680 INFO:tasks.workunit.client.1.smithi192.stdout:6/104: dwrite d0/d8/fa [0,4194304] 0 2024-04-24T22:45:18.681 INFO:tasks.workunit.client.1.smithi192.stdout:6/105: truncate d0/d8/fe 9078166 0 2024-04-24T22:45:18.683 INFO:tasks.workunit.client.0.smithi063.stdout:7/136: mknod d0/d4/de/d17/c36 0 2024-04-24T22:45:18.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/151: creat d2/f2e x:0 0 0 2024-04-24T22:45:18.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/143: symlink d2/d27/d28/l31 0 2024-04-24T22:45:18.686 INFO:tasks.workunit.client.0.smithi063.stdout:4/152: write d2/f1c [509664,14168] 0 2024-04-24T22:45:18.691 INFO:tasks.workunit.client.0.smithi063.stdout:0/163: rmdir d6 39 2024-04-24T22:45:18.694 INFO:tasks.workunit.client.0.smithi063.stdout:5/164: creat d5/d11/f3c x:0 0 0 2024-04-24T22:45:18.697 INFO:tasks.workunit.client.1.smithi192.stdout:0/224: dread d3/d27/d28/f19 [0,4194304] 0 2024-04-24T22:45:18.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/176: creat da/f3b x:0 0 0 2024-04-24T22:45:18.698 INFO:tasks.workunit.client.1.smithi192.stdout:9/122: chown d3/c18 550 1 2024-04-24T22:45:18.699 INFO:tasks.workunit.client.0.smithi063.stdout:2/149: creat d0/d2/d27/f29 x:0 0 0 2024-04-24T22:45:18.699 INFO:tasks.workunit.client.0.smithi063.stdout:2/150: fsync d0/d2/d3/f25 0 2024-04-24T22:45:18.701 INFO:tasks.workunit.client.0.smithi063.stdout:1/148: unlink d3/c5 0 2024-04-24T22:45:18.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/160: creat d9/d15/d2d/f37 x:0 0 0 2024-04-24T22:45:18.704 INFO:tasks.workunit.client.1.smithi192.stdout:7/136: mknod d2/db/d21/c29 0 2024-04-24T22:45:18.706 INFO:tasks.workunit.client.0.smithi063.stdout:6/166: rmdir db/d1d/d20/d29 39 2024-04-24T22:45:18.706 INFO:tasks.workunit.client.0.smithi063.stdout:6/167: chown f3 423375 1 2024-04-24T22:45:18.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/168: chown db/f10 52 1 2024-04-24T22:45:18.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/104: rmdir db/dc/d1a 0 2024-04-24T22:45:18.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/105: chown f5 694395 1 2024-04-24T22:45:18.709 INFO:tasks.workunit.client.1.smithi192.stdout:8/106: dread - db/dc/f17 zero size 2024-04-24T22:45:18.710 INFO:tasks.workunit.client.0.smithi063.stdout:7/137: creat d0/d2f/f37 x:0 0 0 2024-04-24T22:45:18.712 INFO:tasks.workunit.client.0.smithi063.stdout:4/153: unlink d2/c17 0 2024-04-24T22:45:18.714 INFO:tasks.workunit.client.1.smithi192.stdout:9/123: dread d3/dd/de/f12 [0,4194304] 0 2024-04-24T22:45:18.715 INFO:tasks.workunit.client.1.smithi192.stdout:2/206: creat d4/d9/f3c x:0 0 0 2024-04-24T22:45:18.716 INFO:tasks.workunit.client.0.smithi063.stdout:8/186: symlink d1/d11/d3c/l40 0 2024-04-24T22:45:18.717 INFO:tasks.workunit.client.1.smithi192.stdout:5/105: symlink d0/d1b/d1c/l1e 0 2024-04-24T22:45:18.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/164: dwrite d6/dd/d20/f3b [0,4194304] 0 2024-04-24T22:45:18.719 INFO:tasks.workunit.client.0.smithi063.stdout:5/165: rename d5/d11/f1a to d5/d24/f3d 0 2024-04-24T22:45:18.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/177: write da/f31 [649530,22506] 0 2024-04-24T22:45:18.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/82: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:18.725 INFO:tasks.workunit.client.0.smithi063.stdout:5/166: dwrite d5/f9 [0,4194304] 0 2024-04-24T22:45:18.725 INFO:tasks.workunit.client.1.smithi192.stdout:4/83: truncate d2/d9/d25/f23 35438 0 2024-04-24T22:45:18.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/147: creat d6/d8/db/d2b/f34 x:0 0 0 2024-04-24T22:45:18.733 INFO:tasks.workunit.client.1.smithi192.stdout:7/137: mkdir d2/db/d2a 0 2024-04-24T22:45:18.734 INFO:tasks.workunit.client.0.smithi063.stdout:5/167: dwrite d5/d11/f2f [0,4194304] 0 2024-04-24T22:45:18.734 INFO:tasks.workunit.client.0.smithi063.stdout:6/169: fdatasync f8 0 2024-04-24T22:45:18.734 INFO:tasks.workunit.client.0.smithi063.stdout:5/168: readlink l2 0 2024-04-24T22:45:18.737 INFO:tasks.workunit.client.1.smithi192.stdout:8/107: creat db/dc/f1c x:0 0 0 2024-04-24T22:45:18.739 INFO:tasks.workunit.client.0.smithi063.stdout:7/138: chown d0/d4/c1d 323488 1 2024-04-24T22:45:18.741 INFO:tasks.workunit.client.1.smithi192.stdout:6/106: creat d0/d8/d18/f26 x:0 0 0 2024-04-24T22:45:18.747 INFO:tasks.workunit.client.1.smithi192.stdout:7/138: dwrite d2/f6 [4194304,4194304] 0 2024-04-24T22:45:18.750 INFO:tasks.workunit.client.1.smithi192.stdout:9/124: unlink d3/f9 0 2024-04-24T22:45:18.755 INFO:tasks.workunit.client.1.smithi192.stdout:2/207: symlink d4/d29/l3d 0 2024-04-24T22:45:18.758 INFO:tasks.workunit.client.1.smithi192.stdout:5/106: mkdir d0/d13/d1f 0 2024-04-24T22:45:18.761 INFO:tasks.workunit.client.1.smithi192.stdout:0/225: link d3/d27/d28/f20 d3/d27/d28/d2e/d4a/f4d 0 2024-04-24T22:45:18.762 INFO:tasks.workunit.client.1.smithi192.stdout:5/107: dread d0/f18 [4194304,4194304] 0 2024-04-24T22:45:18.763 INFO:tasks.workunit.client.0.smithi063.stdout:1/149: write d3/f17 [1016232,17465] 0 2024-04-24T22:45:18.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/165: dwrite d6/da/dc/f1e [0,4194304] 0 2024-04-24T22:45:18.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/166: dread d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:18.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/187: write d1/d22/f31 [1285135,14487] 0 2024-04-24T22:45:18.813 INFO:tasks.workunit.client.0.smithi063.stdout:4/154: dwrite d2/fd [4194304,4194304] 0 2024-04-24T22:45:18.868 INFO:tasks.workunit.client.0.smithi063.stdout:2/151: sync 2024-04-24T22:45:18.877 INFO:tasks.workunit.client.1.smithi192.stdout:6/107: unlink d0/d8/l1c 0 2024-04-24T22:45:18.877 INFO:tasks.workunit.client.1.smithi192.stdout:6/108: read - d0/f19 zero size 2024-04-24T22:45:18.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/169: rename d5/d11/d18/f39 to d5/d24/d2c/f3e 0 2024-04-24T22:45:18.885 INFO:tasks.workunit.client.1.smithi192.stdout:6/109: dwrite d0/d8/f14 [0,4194304] 0 2024-04-24T22:45:18.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/139: mknod d0/d26/c38 0 2024-04-24T22:45:18.885 INFO:tasks.workunit.client.0.smithi063.stdout:5/170: dread d5/d11/f2f [0,4194304] 0 2024-04-24T22:45:18.886 INFO:tasks.workunit.client.1.smithi192.stdout:6/110: dread - d0/d8/d22/f24 zero size 2024-04-24T22:45:18.886 INFO:tasks.workunit.client.0.smithi063.stdout:5/171: chown d5/d24 109887 1 2024-04-24T22:45:18.896 INFO:tasks.workunit.client.1.smithi192.stdout:5/108: creat d0/d1b/d1c/f20 x:0 0 0 2024-04-24T22:45:18.897 INFO:tasks.workunit.client.1.smithi192.stdout:5/109: truncate d0/d13/d19/f1d 531411 0 2024-04-24T22:45:18.902 INFO:tasks.workunit.client.1.smithi192.stdout:4/84: symlink d2/l26 0 2024-04-24T22:45:18.903 INFO:tasks.workunit.client.1.smithi192.stdout:4/85: chown d2/db/d10/c1d 1 1 2024-04-24T22:45:18.904 INFO:tasks.workunit.client.1.smithi192.stdout:4/86: dread - d2/d9/d1e/f24 zero size 2024-04-24T22:45:18.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/161: creat d9/f38 x:0 0 0 2024-04-24T22:45:18.904 INFO:tasks.workunit.client.1.smithi192.stdout:4/87: chown d2/f1f 129636617 1 2024-04-24T22:45:18.905 INFO:tasks.workunit.client.0.smithi063.stdout:0/167: mknod d6/da/dc/d26/c3e 0 2024-04-24T22:45:18.906 INFO:tasks.workunit.client.0.smithi063.stdout:3/178: creat da/d39/f3c x:0 0 0 2024-04-24T22:45:18.906 INFO:tasks.workunit.client.0.smithi063.stdout:3/179: stat da/f36 0 2024-04-24T22:45:18.907 INFO:tasks.workunit.client.0.smithi063.stdout:9/148: creat d6/d2f/f35 x:0 0 0 2024-04-24T22:45:18.909 INFO:tasks.workunit.client.1.smithi192.stdout:3/144: getdents d2 0 2024-04-24T22:45:18.911 INFO:tasks.workunit.client.1.smithi192.stdout:3/145: read d2/d4/d7/fd [438081,95030] 0 2024-04-24T22:45:18.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/125: link d3/f8 d3/dd/de/d17/f2b 0 2024-04-24T22:45:18.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/208: getdents d4/d9/d36 0 2024-04-24T22:45:18.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/188: creat d1/d2/d17/f41 x:0 0 0 2024-04-24T22:45:18.913 INFO:tasks.workunit.client.1.smithi192.stdout:3/146: truncate d2/d4/d12/d1a/d22/f2a 113446 0 2024-04-24T22:45:18.913 INFO:tasks.workunit.client.0.smithi063.stdout:9/149: dwrite d6/d8/db/d2b/f34 [0,4194304] 0 2024-04-24T22:45:18.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/150: readlink d6/d8/l16 0 2024-04-24T22:45:18.914 INFO:tasks.workunit.client.1.smithi192.stdout:6/111: rename d0/d8/d22/f24 to d0/d8/d22/f27 0 2024-04-24T22:45:18.914 INFO:tasks.workunit.client.1.smithi192.stdout:3/147: readlink d2/d4/d12/d1a/d22/l1f 0 2024-04-24T22:45:18.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/151: fdatasync d6/d8/d1f/d20/f21 0 2024-04-24T22:45:18.915 INFO:tasks.workunit.client.1.smithi192.stdout:6/112: write d0/d8/fa [285461,12146] 0 2024-04-24T22:45:18.917 INFO:tasks.workunit.client.1.smithi192.stdout:2/209: dread d4/d9/f18 [0,4194304] 0 2024-04-24T22:45:18.926 INFO:tasks.workunit.client.1.smithi192.stdout:6/113: dwrite d0/d8/d18/f26 [0,4194304] 0 2024-04-24T22:45:18.926 INFO:tasks.workunit.client.1.smithi192.stdout:8/108: creat db/f1d x:0 0 0 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: pgmap v7: 65 pgs: 65 active+clean; 542 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2.1 MiB/s rd, 59 MiB/s wr, 178 op/s 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: Standby manager daemon smithi063.upcnvd started 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/crt"}]: dispatch 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/key"}]: dispatch 2024-04-24T22:45:18.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:18 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:45:18.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/109: read - db/f1d zero size 2024-04-24T22:45:18.930 INFO:tasks.workunit.client.1.smithi192.stdout:1/162: creat d9/d15/d2d/d31/f39 x:0 0 0 2024-04-24T22:45:18.932 INFO:tasks.workunit.client.0.smithi063.stdout:7/140: symlink d0/l39 0 2024-04-24T22:45:18.932 INFO:tasks.workunit.client.1.smithi192.stdout:8/110: chown db/dc/f1c 11322 1 2024-04-24T22:45:18.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/88: creat d2/db/d10/f27 x:0 0 0 2024-04-24T22:45:18.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/110: dwrite d0/f18 [0,4194304] 0 2024-04-24T22:45:18.938 INFO:tasks.workunit.client.0.smithi063.stdout:1/150: read d3/f17 [371551,36438] 0 2024-04-24T22:45:18.939 INFO:tasks.workunit.client.0.smithi063.stdout:5/172: dread f1 [0,4194304] 0 2024-04-24T22:45:18.939 INFO:tasks.workunit.client.1.smithi192.stdout:6/114: dwrite d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:18.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/173: read d5/d11/d18/f20 [1487607,99776] 0 2024-04-24T22:45:18.945 INFO:tasks.workunit.client.1.smithi192.stdout:3/148: rmdir d2/d27 39 2024-04-24T22:45:18.946 INFO:tasks.workunit.client.0.smithi063.stdout:0/168: creat d6/dd/f3f x:0 0 0 2024-04-24T22:45:18.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/169: write d6/f7 [8507263,34006] 0 2024-04-24T22:45:18.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/170: read d6/da/f14 [8207490,43123] 0 2024-04-24T22:45:18.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/210: creat d4/d9/d37/f3e x:0 0 0 2024-04-24T22:45:18.949 INFO:tasks.workunit.client.0.smithi063.stdout:0/171: stat d6/da 0 2024-04-24T22:45:18.950 INFO:tasks.workunit.client.0.smithi063.stdout:5/174: dread f0 [0,4194304] 0 2024-04-24T22:45:18.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/172: dread - d6/dd/d20/d21/d25/d31/f35 zero size 2024-04-24T22:45:18.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/180: dread da/f2c [0,4194304] 0 2024-04-24T22:45:18.953 INFO:tasks.workunit.client.0.smithi063.stdout:3/181: chown l9 278 1 2024-04-24T22:45:18.953 INFO:tasks.workunit.client.0.smithi063.stdout:0/173: dread - d6/dd/d20/d21/d25/d31/f35 zero size 2024-04-24T22:45:18.956 INFO:tasks.workunit.client.0.smithi063.stdout:2/152: link d0/d2/d3/d7/f13 d0/d2/d3/d1c/f2a 0 2024-04-24T22:45:18.957 INFO:tasks.workunit.client.0.smithi063.stdout:9/152: unlink d6/c27 0 2024-04-24T22:45:18.959 INFO:tasks.workunit.client.0.smithi063.stdout:6/170: creat db/d1d/d20/f38 x:0 0 0 2024-04-24T22:45:18.961 INFO:tasks.workunit.client.1.smithi192.stdout:1/163: creat d9/d15/d2d/d31/d2f/f3a x:0 0 0 2024-04-24T22:45:18.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/155: rename d2/d8/c24 to d2/d19/c2f 0 2024-04-24T22:45:18.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/156: chown d2/d19/f1f 21 1 2024-04-24T22:45:18.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/157: fdatasync d2/fb 0 2024-04-24T22:45:18.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/89: write d2/d9/d25/f23 [431562,68060] 0 2024-04-24T22:45:18.966 INFO:tasks.workunit.client.1.smithi192.stdout:8/111: rename db/dc/f11 to db/dc/f1e 0 2024-04-24T22:45:18.966 INFO:tasks.workunit.client.1.smithi192.stdout:4/90: chown d2/db/d10/l1a 0 1 2024-04-24T22:45:18.970 INFO:tasks.workunit.client.1.smithi192.stdout:1/164: dwrite d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:18.972 INFO:tasks.workunit.client.1.smithi192.stdout:6/115: rmdir d0/d8/d22 39 2024-04-24T22:45:18.978 INFO:tasks.workunit.client.1.smithi192.stdout:6/116: dwrite d0/d8/fe [0,4194304] 0 2024-04-24T22:45:18.979 INFO:tasks.workunit.client.1.smithi192.stdout:3/149: creat d2/d4/d12/d1a/d22/f32 x:0 0 0 2024-04-24T22:45:18.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/189: unlink d1/d2/c23 0 2024-04-24T22:45:18.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/117: dread d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:18.985 INFO:tasks.workunit.client.1.smithi192.stdout:0/226: getdents d3/d27 0 2024-04-24T22:45:18.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/174: unlink d6/dd/d20/d24/f2e 0 2024-04-24T22:45:18.993 INFO:tasks.workunit.client.0.smithi063.stdout:2/153: mknod d0/dd/c2b 0 2024-04-24T22:45:18.994 INFO:tasks.workunit.client.0.smithi063.stdout:9/153: mkdir d6/d8/db/d2b/d36 0 2024-04-24T22:45:18.995 INFO:tasks.workunit.client.0.smithi063.stdout:9/154: write d6/fd [3147300,82582] 0 2024-04-24T22:45:18.997 INFO:tasks.workunit.client.0.smithi063.stdout:7/141: rename d0/d4/de/d17/f1e to d0/d4/f3a 0 2024-04-24T22:45:18.998 INFO:tasks.workunit.client.0.smithi063.stdout:9/155: dread d6/d8/d1f/d20/f21 [0,4194304] 0 2024-04-24T22:45:18.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/156: chown d6/l13 3885271 1 2024-04-24T22:45:19.001 INFO:tasks.workunit.client.1.smithi192.stdout:5/111: mknod d0/c21 0 2024-04-24T22:45:19.001 INFO:tasks.workunit.client.1.smithi192.stdout:5/112: rename d0/d13/d1f to d0/d13/d1f/d22 22 2024-04-24T22:45:19.002 INFO:tasks.workunit.client.0.smithi063.stdout:4/158: symlink d2/d8/l30 0 2024-04-24T22:45:19.002 INFO:tasks.workunit.client.1.smithi192.stdout:2/211: write d4/d9/d10/f1e [993637,19952] 0 2024-04-24T22:45:19.003 INFO:tasks.workunit.client.1.smithi192.stdout:2/212: readlink d4/d29/l3d 0 2024-04-24T22:45:19.004 INFO:tasks.workunit.client.1.smithi192.stdout:2/213: chown d4/d9/d10/c22 869794271 1 2024-04-24T22:45:19.004 INFO:tasks.workunit.client.1.smithi192.stdout:1/165: rmdir d9/d15/d2d 39 2024-04-24T22:45:19.006 INFO:tasks.workunit.client.0.smithi063.stdout:4/159: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:19.013 INFO:tasks.workunit.client.1.smithi192.stdout:0/227: creat d3/d27/d3c/f4e x:0 0 0 2024-04-24T22:45:19.014 INFO:tasks.workunit.client.1.smithi192.stdout:9/126: getdents d3/d24 0 2024-04-24T22:45:19.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/127: stat d3/dd/de/d17/f23 0 2024-04-24T22:45:19.019 INFO:tasks.workunit.client.0.smithi063.stdout:5/175: write d5/d11/d18/f3a [613254,97983] 0 2024-04-24T22:45:19.020 INFO:tasks.workunit.client.0.smithi063.stdout:2/154: symlink d0/l2c 0 2024-04-24T22:45:19.023 INFO:tasks.workunit.client.1.smithi192.stdout:1/166: dread - d9/d15/d2d/f37 zero size 2024-04-24T22:45:19.024 INFO:tasks.workunit.client.1.smithi192.stdout:1/167: write d9/d15/d2d/d31/f39 [188784,84177] 0 2024-04-24T22:45:19.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/171: rename db/d1d/c2c to db/d1d/d2d/c39 0 2024-04-24T22:45:19.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/175: dwrite d6/da/f14 [4194304,4194304] 0 2024-04-24T22:45:19.027 INFO:tasks.workunit.client.1.smithi192.stdout:0/228: symlink d3/l4f 0 2024-04-24T22:45:19.027 INFO:tasks.workunit.client.0.smithi063.stdout:4/160: fdatasync d2/f13 0 2024-04-24T22:45:19.028 INFO:tasks.workunit.client.1.smithi192.stdout:9/128: mkdir d3/d2c 0 2024-04-24T22:45:19.029 INFO:tasks.workunit.client.1.smithi192.stdout:9/129: chown d3/dd/de/f1a 233 1 2024-04-24T22:45:19.029 INFO:tasks.workunit.client.1.smithi192.stdout:8/112: link ca db/dc/c1f 0 2024-04-24T22:45:19.030 INFO:tasks.workunit.client.1.smithi192.stdout:8/113: chown db/dc/ld 5 1 2024-04-24T22:45:19.031 INFO:tasks.workunit.client.0.smithi063.stdout:4/161: dread d2/d19/f20 [0,4194304] 0 2024-04-24T22:45:19.031 INFO:tasks.workunit.client.1.smithi192.stdout:8/114: write db/dc/f18 [329295,66320] 0 2024-04-24T22:45:19.032 INFO:tasks.workunit.client.0.smithi063.stdout:3/182: creat da/f3d x:0 0 0 2024-04-24T22:45:19.032 INFO:tasks.workunit.client.1.smithi192.stdout:8/115: truncate db/f14 775810 0 2024-04-24T22:45:19.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/162: truncate d2/d19/f2b 23494 0 2024-04-24T22:45:19.034 INFO:tasks.workunit.client.1.smithi192.stdout:2/214: creat d4/d9/d36/f3f x:0 0 0 2024-04-24T22:45:19.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/150: creat d2/f33 x:0 0 0 2024-04-24T22:45:19.039 INFO:tasks.workunit.client.0.smithi063.stdout:7/142: symlink d0/d4/de/d17/d28/d31/l3b 0 2024-04-24T22:45:19.041 INFO:tasks.workunit.client.1.smithi192.stdout:0/229: rename d3/f45 to d3/d40/f50 0 2024-04-24T22:45:19.041 INFO:tasks.workunit.client.1.smithi192.stdout:0/230: read d3/d27/d28/f19 [146696,13873] 0 2024-04-24T22:45:19.042 INFO:tasks.workunit.client.0.smithi063.stdout:6/172: mknod db/c3a 0 2024-04-24T22:45:19.043 INFO:tasks.workunit.client.0.smithi063.stdout:8/190: rename d1/d2/d17/f41 to d1/d11/d32/d38/f42 0 2024-04-24T22:45:19.044 INFO:tasks.workunit.client.1.smithi192.stdout:4/91: getdents d2/d9/d1e 0 2024-04-24T22:45:19.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/176: creat d6/da/d2b/f40 x:0 0 0 2024-04-24T22:45:19.045 INFO:tasks.workunit.client.1.smithi192.stdout:8/116: symlink db/dc/l20 0 2024-04-24T22:45:19.046 INFO:tasks.workunit.client.0.smithi063.stdout:3/183: creat da/d18/f3e x:0 0 0 2024-04-24T22:45:19.047 INFO:tasks.workunit.client.1.smithi192.stdout:5/113: creat d0/f23 x:0 0 0 2024-04-24T22:45:19.048 INFO:tasks.workunit.client.0.smithi063.stdout:4/163: creat d2/d8/f31 x:0 0 0 2024-04-24T22:45:19.048 INFO:tasks.workunit.client.1.smithi192.stdout:2/215: creat d4/d9/d37/f40 x:0 0 0 2024-04-24T22:45:19.049 INFO:tasks.workunit.client.1.smithi192.stdout:2/216: readlink d4/d9/d10/l32 0 2024-04-24T22:45:19.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/176: creat d5/d11/d33/f3f x:0 0 0 2024-04-24T22:45:19.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/151: creat d2/d4/f34 x:0 0 0 2024-04-24T22:45:19.052 INFO:tasks.workunit.client.0.smithi063.stdout:7/143: creat d0/d4/de/d17/d28/f3c x:0 0 0 2024-04-24T22:45:19.053 INFO:tasks.workunit.client.0.smithi063.stdout:4/164: dwrite d2/d19/f20 [0,4194304] 0 2024-04-24T22:45:19.053 INFO:tasks.workunit.client.1.smithi192.stdout:9/130: getdents d3/d24 0 2024-04-24T22:45:19.054 INFO:tasks.workunit.client.0.smithi063.stdout:4/165: read d2/d8/f2c [4100291,44983] 0 2024-04-24T22:45:19.055 INFO:tasks.workunit.client.1.smithi192.stdout:2/217: dwrite d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:19.056 INFO:tasks.workunit.client.0.smithi063.stdout:1/151: rename d3/da/dc/d1f/f20 to d3/da/dc/d1f/d2b/f32 0 2024-04-24T22:45:19.063 INFO:tasks.workunit.client.1.smithi192.stdout:5/114: creat d0/d1b/f24 x:0 0 0 2024-04-24T22:45:19.064 INFO:tasks.workunit.client.1.smithi192.stdout:5/115: chown d0/d13/d19/fd 265674 1 2024-04-24T22:45:19.066 INFO:tasks.workunit.client.1.smithi192.stdout:3/152: truncate d2/d4/d7/fd 1667112 0 2024-04-24T22:45:19.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/144: creat d0/f3d x:0 0 0 2024-04-24T22:45:19.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/145: chown d0/c29 0 1 2024-04-24T22:45:19.077 INFO:tasks.workunit.client.0.smithi063.stdout:8/191: write d1/d11/f1b [1793718,96239] 0 2024-04-24T22:45:19.077 INFO:tasks.workunit.client.0.smithi063.stdout:8/192: fdatasync d1/d2/f1c 0 2024-04-24T22:45:19.078 INFO:tasks.workunit.client.1.smithi192.stdout:2/218: truncate d4/d9/d10/f23 764851 0 2024-04-24T22:45:19.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/166: rmdir d2/d19 39 2024-04-24T22:45:19.079 INFO:tasks.workunit.client.1.smithi192.stdout:0/231: dwrite d3/d27/d28/d2e/d4a/f4d [0,4194304] 0 2024-04-24T22:45:19.084 INFO:tasks.workunit.client.0.smithi063.stdout:5/177: rename l2 to d5/d11/l40 0 2024-04-24T22:45:19.085 INFO:tasks.workunit.client.1.smithi192.stdout:2/219: dread d4/f31 [0,4194304] 0 2024-04-24T22:45:19.085 INFO:tasks.workunit.client.1.smithi192.stdout:8/117: dwrite f5 [0,4194304] 0 2024-04-24T22:45:19.086 INFO:tasks.workunit.client.1.smithi192.stdout:2/220: chown d4/d9/f3c 28354772 1 2024-04-24T22:45:19.087 INFO:tasks.workunit.client.1.smithi192.stdout:5/116: creat d0/d1b/f25 x:0 0 0 2024-04-24T22:45:19.087 INFO:tasks.workunit.client.1.smithi192.stdout:8/118: write db/dc/f1c [69832,69724] 0 2024-04-24T22:45:19.088 INFO:tasks.workunit.client.0.smithi063.stdout:1/152: creat d3/da/dc/d11/f33 x:0 0 0 2024-04-24T22:45:19.089 INFO:tasks.workunit.client.1.smithi192.stdout:8/119: write f5 [2499603,18437] 0 2024-04-24T22:45:19.091 INFO:tasks.workunit.client.1.smithi192.stdout:4/92: truncate d2/f1f 3822692 0 2024-04-24T22:45:19.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/155: getdents d0/dd 0 2024-04-24T22:45:19.098 INFO:tasks.workunit.client.1.smithi192.stdout:8/120: dwrite db/dc/f1c [0,4194304] 0 2024-04-24T22:45:19.102 INFO:tasks.workunit.client.1.smithi192.stdout:8/121: dread db/f15 [0,4194304] 0 2024-04-24T22:45:19.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/117: symlink d0/d1b/l26 0 2024-04-24T22:45:19.108 INFO:tasks.workunit.client.0.smithi063.stdout:4/167: rmdir d2 39 2024-04-24T22:45:19.109 INFO:tasks.workunit.client.1.smithi192.stdout:8/122: dwrite f7 [0,4194304] 0 2024-04-24T22:45:19.109 INFO:tasks.workunit.client.1.smithi192.stdout:8/123: chown db/l16 4021092 1 2024-04-24T22:45:19.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/124: write db/f14 [1416370,1878] 0 2024-04-24T22:45:19.112 INFO:tasks.workunit.client.0.smithi063.stdout:5/178: rename d5/f1f to d5/d11/d18/f41 0 2024-04-24T22:45:19.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/153: mkdir d3/da/d34 0 2024-04-24T22:45:19.115 INFO:tasks.workunit.client.1.smithi192.stdout:4/93: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:19.117 INFO:tasks.workunit.client.1.smithi192.stdout:3/153: mknod d2/d27/d28/c35 0 2024-04-24T22:45:19.118 INFO:tasks.workunit.client.1.smithi192.stdout:3/154: truncate d2/d4/d12/f1c 347389 0 2024-04-24T22:45:19.119 INFO:tasks.workunit.client.1.smithi192.stdout:3/155: stat d2/d4/d12/f1b 0 2024-04-24T22:45:19.121 INFO:tasks.workunit.client.0.smithi063.stdout:2/156: mkdir d0/d2/d2d 0 2024-04-24T22:45:19.121 INFO:tasks.workunit.client.1.smithi192.stdout:9/131: link d3/dd/de/d17/f2b d3/d2c/f2d 0 2024-04-24T22:45:19.123 INFO:tasks.workunit.client.0.smithi063.stdout:6/173: link db/d1d/l24 db/d1d/d20/l3b 0 2024-04-24T22:45:19.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/193: mkdir d1/d43 0 2024-04-24T22:45:19.125 INFO:tasks.workunit.client.1.smithi192.stdout:9/132: dread d3/dd/de/d17/f1d [0,4194304] 0 2024-04-24T22:45:19.128 INFO:tasks.workunit.client.0.smithi063.stdout:4/168: chown d2/c21 2 1 2024-04-24T22:45:19.129 INFO:tasks.workunit.client.0.smithi063.stdout:8/194: dread d1/f5 [0,4194304] 0 2024-04-24T22:45:19.134 INFO:tasks.workunit.client.1.smithi192.stdout:5/118: rename d0/d13/d19/fd to d0/d1b/d1c/f27 0 2024-04-24T22:45:19.136 INFO:tasks.workunit.client.1.smithi192.stdout:8/125: truncate f3 9731201 0 2024-04-24T22:45:19.138 INFO:tasks.workunit.client.1.smithi192.stdout:3/156: creat d2/d27/d28/f36 x:0 0 0 2024-04-24T22:45:19.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/133: truncate d3/f8 4098928 0 2024-04-24T22:45:19.142 INFO:tasks.workunit.client.1.smithi192.stdout:0/232: link d3/d27/d28/f1d d3/d25/f51 0 2024-04-24T22:45:19.144 INFO:tasks.workunit.client.1.smithi192.stdout:2/221: creat d4/f41 x:0 0 0 2024-04-24T22:45:19.148 INFO:tasks.workunit.client.1.smithi192.stdout:0/233: rmdir d3/d25 39 2024-04-24T22:45:19.149 INFO:tasks.workunit.client.1.smithi192.stdout:0/234: readlink d3/d27/l36 0 2024-04-24T22:45:19.152 INFO:tasks.workunit.client.1.smithi192.stdout:9/134: creat d3/d2c/f2e x:0 0 0 2024-04-24T22:45:19.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/235: creat d3/d27/d28/d31/f52 x:0 0 0 2024-04-24T22:45:19.154 INFO:tasks.workunit.client.1.smithi192.stdout:9/135: write d3/dd/de/f1a [686108,117217] 0 2024-04-24T22:45:19.158 INFO:tasks.workunit.client.1.smithi192.stdout:8/126: getdents db/dc 0 2024-04-24T22:45:19.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/222: dread d4/f27 [0,4194304] 0 2024-04-24T22:45:19.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/136: fdatasync d3/dd/de/f12 0 2024-04-24T22:45:19.159 INFO:tasks.workunit.client.1.smithi192.stdout:8/127: readlink db/dc/l12 0 2024-04-24T22:45:19.160 INFO:tasks.workunit.client.1.smithi192.stdout:8/128: rmdir db 39 2024-04-24T22:45:19.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/137: creat d3/dd/de/f2f x:0 0 0 2024-04-24T22:45:19.167 INFO:tasks.workunit.client.1.smithi192.stdout:2/223: dwrite d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:19.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/129: rmdir db/dc 39 2024-04-24T22:45:19.169 INFO:tasks.workunit.client.1.smithi192.stdout:5/119: dwrite d0/fc [0,4194304] 0 2024-04-24T22:45:19.169 INFO:tasks.workunit.client.1.smithi192.stdout:2/224: truncate d4/d9/f3c 815807 0 2024-04-24T22:45:19.172 INFO:tasks.workunit.client.1.smithi192.stdout:5/120: symlink d0/d1b/l28 0 2024-04-24T22:45:19.177 INFO:tasks.workunit.client.1.smithi192.stdout:5/121: creat d0/f29 x:0 0 0 2024-04-24T22:45:19.178 INFO:tasks.workunit.client.1.smithi192.stdout:5/122: readlink d0/d1b/l26 0 2024-04-24T22:45:19.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/130: dwrite db/dc/f1e [0,4194304] 0 2024-04-24T22:45:19.180 INFO:tasks.workunit.client.0.smithi063.stdout:0/177: getdents d6/da/dc 0 2024-04-24T22:45:19.180 INFO:tasks.workunit.client.0.smithi063.stdout:7/146: link d0/l16 d0/d26/l3e 0 2024-04-24T22:45:19.181 INFO:tasks.workunit.client.1.smithi192.stdout:2/225: truncate d4/f27 1307573 0 2024-04-24T22:45:19.184 INFO:tasks.workunit.client.1.smithi192.stdout:8/131: dread f8 [0,4194304] 0 2024-04-24T22:45:19.186 INFO:tasks.workunit.client.0.smithi063.stdout:1/154: rename d3/f1e to d3/da/dc/d11/f35 0 2024-04-24T22:45:19.187 INFO:tasks.workunit.client.0.smithi063.stdout:8/195: write d1/d2/fe [2330971,6341] 0 2024-04-24T22:45:19.188 INFO:tasks.workunit.client.1.smithi192.stdout:5/123: creat d0/d13/d1f/f2a x:0 0 0 2024-04-24T22:45:19.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/226: mknod d4/d9/d10/c42 0 2024-04-24T22:45:19.192 INFO:tasks.workunit.client.0.smithi063.stdout:5/179: dwrite d5/d24/d2c/f3e [0,4194304] 0 2024-04-24T22:45:19.195 INFO:tasks.workunit.client.1.smithi192.stdout:2/227: dread d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:19.197 INFO:tasks.workunit.client.0.smithi063.stdout:6/174: dwrite db/ff [4194304,4194304] 0 2024-04-24T22:45:19.197 INFO:tasks.workunit.client.0.smithi063.stdout:5/180: dread - d5/d11/d18/f19 zero size 2024-04-24T22:45:19.198 INFO:tasks.workunit.client.0.smithi063.stdout:2/157: truncate d0/dd/f22 691754 0 2024-04-24T22:45:19.198 INFO:tasks.workunit.client.0.smithi063.stdout:7/147: dwrite d0/d4/de/f1a [4194304,4194304] 0 2024-04-24T22:45:19.202 INFO:tasks.workunit.client.0.smithi063.stdout:4/169: dwrite d2/f5 [0,4194304] 0 2024-04-24T22:45:19.202 INFO:tasks.workunit.client.0.smithi063.stdout:5/181: write d5/f14 [1328794,122413] 0 2024-04-24T22:45:19.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/175: write db/d1d/d20/f2b [5946742,80836] 0 2024-04-24T22:45:19.204 INFO:tasks.workunit.client.0.smithi063.stdout:1/155: chown d3/f7 13256 1 2024-04-24T22:45:19.209 INFO:tasks.workunit.client.0.smithi063.stdout:0/178: creat d6/d3a/f41 x:0 0 0 2024-04-24T22:45:19.210 INFO:tasks.workunit.client.1.smithi192.stdout:8/132: link db/dc/c1f db/dc/c21 0 2024-04-24T22:45:19.213 INFO:tasks.workunit.client.0.smithi063.stdout:2/158: dwrite d0/d2/d3/d7/f19 [0,4194304] 0 2024-04-24T22:45:19.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/148: unlink d0/fd 0 2024-04-24T22:45:19.215 INFO:tasks.workunit.client.1.smithi192.stdout:8/133: mkdir db/dc/d22 0 2024-04-24T22:45:19.215 INFO:tasks.workunit.client.0.smithi063.stdout:9/157: sync 2024-04-24T22:45:19.217 INFO:tasks.workunit.client.0.smithi063.stdout:6/176: unlink db/c17 0 2024-04-24T22:45:19.218 INFO:tasks.workunit.client.0.smithi063.stdout:2/159: mknod d0/d2/d3/d7/c2e 0 2024-04-24T22:45:19.219 INFO:tasks.workunit.client.0.smithi063.stdout:1/156: dwrite f1 [0,4194304] 0 2024-04-24T22:45:19.219 INFO:tasks.workunit.client.0.smithi063.stdout:5/182: creat d5/d30/d3b/f42 x:0 0 0 2024-04-24T22:45:19.220 INFO:tasks.workunit.client.0.smithi063.stdout:9/158: symlink d6/d2f/l37 0 2024-04-24T22:45:19.220 INFO:tasks.workunit.client.0.smithi063.stdout:0/179: truncate d6/dd/d20/d21/d25/d31/f35 910907 0 2024-04-24T22:45:19.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/196: getdents d1 0 2024-04-24T22:45:19.222 INFO:tasks.workunit.client.0.smithi063.stdout:6/177: chown db/d1d/d20/l3b 400175 1 2024-04-24T22:45:19.224 INFO:tasks.workunit.client.0.smithi063.stdout:5/183: rmdir d5/d30 39 2024-04-24T22:45:19.227 INFO:tasks.workunit.client.0.smithi063.stdout:1/157: link d3/fd d3/da/f36 0 2024-04-24T22:45:19.228 INFO:tasks.workunit.client.0.smithi063.stdout:0/180: mkdir d6/dd/d20/d24/d42 0 2024-04-24T22:45:19.229 INFO:tasks.workunit.client.0.smithi063.stdout:9/159: symlink d6/d8/db/d1c/l38 0 2024-04-24T22:45:19.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/178: rename db/d1d to db/d1d/d20/d29/d3c 22 2024-04-24T22:45:19.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/149: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:19.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/179: chown db/ff 25 1 2024-04-24T22:45:19.232 INFO:tasks.workunit.client.0.smithi063.stdout:7/150: chown d0/d4/de/f1a 3661 1 2024-04-24T22:45:19.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/151: dread - d0/d4/de/f25 zero size 2024-04-24T22:45:19.234 INFO:tasks.workunit.client.0.smithi063.stdout:2/160: dwrite d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:19.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/197: symlink d1/d2/d17/l44 0 2024-04-24T22:45:19.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/158: fsync d3/f17 0 2024-04-24T22:45:19.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/198: write d1/d2/f1c [2553902,47259] 0 2024-04-24T22:45:19.246 INFO:tasks.workunit.client.0.smithi063.stdout:8/199: dread - d1/f1f zero size 2024-04-24T22:45:19.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/200: write d1/d11/f29 [408200,26943] 0 2024-04-24T22:45:19.248 INFO:tasks.workunit.client.0.smithi063.stdout:0/181: creat d6/da/dc/d26/f43 x:0 0 0 2024-04-24T22:45:19.249 INFO:tasks.workunit.client.0.smithi063.stdout:6/180: fsync f3 0 2024-04-24T22:45:19.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/161: chown d0/d2/f1b 8548070 1 2024-04-24T22:45:19.251 INFO:tasks.workunit.client.0.smithi063.stdout:6/181: write db/d1d/d20/f38 [1016752,74434] 0 2024-04-24T22:45:19.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/152: mknod d0/d26/c3f 0 2024-04-24T22:45:19.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/184: symlink d5/d24/d36/l43 0 2024-04-24T22:45:19.253 INFO:tasks.workunit.client.0.smithi063.stdout:6/182: dread - db/d1d/d2d/f36 zero size 2024-04-24T22:45:19.255 INFO:tasks.workunit.client.0.smithi063.stdout:1/159: write d3/f2c [1569346,67367] 0 2024-04-24T22:45:19.255 INFO:tasks.workunit.client.0.smithi063.stdout:7/153: chown d0/d26/d2e/l34 7 1 2024-04-24T22:45:19.257 INFO:tasks.workunit.client.0.smithi063.stdout:6/183: write db/d1d/d20/f23 [1193157,67443] 0 2024-04-24T22:45:19.257 INFO:tasks.workunit.client.0.smithi063.stdout:7/154: stat d0/d4/de/d17/d28/d31 0 2024-04-24T22:45:19.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/155: write d0/d4/de/f25 [363413,69054] 0 2024-04-24T22:45:19.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/162: dwrite d0/d2/d3/d7/f19 [0,4194304] 0 2024-04-24T22:45:19.262 INFO:tasks.workunit.client.0.smithi063.stdout:0/182: rename d6/dd/d20/d24/c38 to d6/dd/d20/d24/c44 0 2024-04-24T22:45:19.264 INFO:tasks.workunit.client.1.smithi192.stdout:5/124: truncate d0/d1b/d1c/f27 7197989 0 2024-04-24T22:45:19.267 INFO:tasks.workunit.client.0.smithi063.stdout:5/185: mkdir d5/d24/d44 0 2024-04-24T22:45:19.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/160: readlink d3/l8 0 2024-04-24T22:45:19.271 INFO:tasks.workunit.client.0.smithi063.stdout:1/161: write d3/da/dc/d11/f33 [409875,11633] 0 2024-04-24T22:45:19.273 INFO:tasks.workunit.client.0.smithi063.stdout:3/184: sync 2024-04-24T22:45:19.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/156: mknod d0/d26/c40 0 2024-04-24T22:45:19.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/163: mkdir d0/d2/d2f 0 2024-04-24T22:45:19.275 INFO:tasks.workunit.client.0.smithi063.stdout:2/164: write d0/d2/f1d [960788,37459] 0 2024-04-24T22:45:19.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/201: rmdir d1/d43 0 2024-04-24T22:45:19.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/160: getdents d6/d8/db 0 2024-04-24T22:45:19.278 INFO:tasks.workunit.client.0.smithi063.stdout:1/162: creat d3/da/f37 x:0 0 0 2024-04-24T22:45:19.280 INFO:tasks.workunit.client.0.smithi063.stdout:6/184: rename db/d1d/d2d/c39 to db/d1d/c3d 0 2024-04-24T22:45:19.281 INFO:tasks.workunit.client.0.smithi063.stdout:3/185: rmdir da/d39 39 2024-04-24T22:45:19.281 INFO:tasks.workunit.client.0.smithi063.stdout:9/161: dread d6/f1b [0,4194304] 0 2024-04-24T22:45:19.282 INFO:tasks.workunit.client.0.smithi063.stdout:0/183: mknod d6/da/dc/d26/d33/c45 0 2024-04-24T22:45:19.282 INFO:tasks.workunit.client.0.smithi063.stdout:9/162: write d6/d8/f17 [6945766,14487] 0 2024-04-24T22:45:19.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/186: creat d5/d30/d3b/f45 x:0 0 0 2024-04-24T22:45:19.284 INFO:tasks.workunit.client.1.smithi192.stdout:2/228: write d4/d9/fb [2057286,96071] 0 2024-04-24T22:45:19.284 INFO:tasks.workunit.client.0.smithi063.stdout:9/163: read - d6/d8/d1f/f28 zero size 2024-04-24T22:45:19.284 INFO:tasks.workunit.client.0.smithi063.stdout:9/164: readlink d6/d8/db/d1c/l38 0 2024-04-24T22:45:19.286 INFO:tasks.workunit.client.0.smithi063.stdout:1/163: mkdir d3/da/dc/d1f/d2b/d38 0 2024-04-24T22:45:19.287 INFO:tasks.workunit.client.0.smithi063.stdout:4/170: write f0 [2236987,78189] 0 2024-04-24T22:45:19.287 INFO:tasks.workunit.client.0.smithi063.stdout:6/185: fdatasync f3 0 2024-04-24T22:45:19.288 INFO:tasks.workunit.client.1.smithi192.stdout:2/229: dread d4/d9/f18 [0,4194304] 0 2024-04-24T22:45:19.288 INFO:tasks.workunit.client.0.smithi063.stdout:6/186: write db/ff [4942448,93397] 0 2024-04-24T22:45:19.290 INFO:tasks.workunit.client.0.smithi063.stdout:7/157: rename d0/d4/l13 to d0/l41 0 2024-04-24T22:45:19.293 INFO:tasks.workunit.client.1.smithi192.stdout:2/230: chown d4/c8 1660 1 2024-04-24T22:45:19.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/158: dread d0/d4/de/f10 [0,4194304] 0 2024-04-24T22:45:19.296 INFO:tasks.workunit.client.1.smithi192.stdout:2/231: unlink d4/d9/f3c 0 2024-04-24T22:45:19.298 INFO:tasks.workunit.client.0.smithi063.stdout:9/165: symlink d6/d2f/l39 0 2024-04-24T22:45:19.303 INFO:tasks.workunit.client.0.smithi063.stdout:1/164: dread - d3/d2d/f2e zero size 2024-04-24T22:45:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: pgmap v7: 65 pgs: 65 active+clean; 542 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2.1 MiB/s rd, 59 MiB/s wr, 178 op/s 2024-04-24T22:45:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: Standby manager daemon smithi063.upcnvd started 2024-04-24T22:45:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/crt"}]: dispatch 2024-04-24T22:45:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:45:19.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/key"}]: dispatch 2024-04-24T22:45:19.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:18 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/382412412' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:45:19.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/232: mknod d4/d9/d10/c43 0 2024-04-24T22:45:19.307 INFO:tasks.workunit.client.0.smithi063.stdout:6/187: fdatasync db/f37 0 2024-04-24T22:45:19.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/165: rename d0/d2/d3/d7/c28 to d0/dd/c30 0 2024-04-24T22:45:19.309 INFO:tasks.workunit.client.0.smithi063.stdout:3/186: creat da/d18/d3a/f3f x:0 0 0 2024-04-24T22:45:19.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/166: write d0/d2/d3/f25 [173805,128321] 0 2024-04-24T22:45:19.310 INFO:tasks.workunit.client.1.smithi192.stdout:2/233: creat d4/d9/d37/f44 x:0 0 0 2024-04-24T22:45:19.315 INFO:tasks.workunit.client.0.smithi063.stdout:5/187: creat d5/d24/d44/f46 x:0 0 0 2024-04-24T22:45:19.316 INFO:tasks.workunit.client.1.smithi192.stdout:5/125: truncate d0/d13/d19/f1d 90324 0 2024-04-24T22:45:19.317 INFO:tasks.workunit.client.0.smithi063.stdout:7/159: creat d0/d4/de/d17/f42 x:0 0 0 2024-04-24T22:45:19.317 INFO:tasks.workunit.client.1.smithi192.stdout:2/234: dwrite d4/d9/d37/f3b [0,4194304] 0 2024-04-24T22:45:19.320 INFO:tasks.workunit.client.0.smithi063.stdout:5/188: dwrite f0 [0,4194304] 0 2024-04-24T22:45:19.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/235: mknod d4/d29/c45 0 2024-04-24T22:45:19.328 INFO:tasks.workunit.client.0.smithi063.stdout:1/165: rename d3/da/dc/d1f/d2b/d38 to d3/da/dc/d39 0 2024-04-24T22:45:19.334 INFO:tasks.workunit.client.1.smithi192.stdout:2/236: rename d4/d9/c1f to d4/d9/d36/c46 0 2024-04-24T22:45:19.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/237: creat d4/d9/d36/f47 x:0 0 0 2024-04-24T22:45:19.339 INFO:tasks.workunit.client.1.smithi192.stdout:2/238: read - d4/d9/d37/f3e zero size 2024-04-24T22:45:19.341 INFO:tasks.workunit.client.0.smithi063.stdout:8/202: fsync d1/d21/f3a 0 2024-04-24T22:45:19.344 INFO:tasks.workunit.client.0.smithi063.stdout:5/189: creat d5/d11/f47 x:0 0 0 2024-04-24T22:45:19.345 INFO:tasks.workunit.client.0.smithi063.stdout:6/188: mkdir db/d1d/d20/d3e 0 2024-04-24T22:45:19.345 INFO:tasks.workunit.client.0.smithi063.stdout:3/187: rmdir da/d18/d3a 39 2024-04-24T22:45:19.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/166: write d3/f19 [3612393,72102] 0 2024-04-24T22:45:19.347 INFO:tasks.workunit.client.0.smithi063.stdout:2/167: mkdir d0/d2/d2d/d31 0 2024-04-24T22:45:19.348 INFO:tasks.workunit.client.0.smithi063.stdout:0/184: truncate d6/da/f14 116101 0 2024-04-24T22:45:19.349 INFO:tasks.workunit.client.0.smithi063.stdout:2/168: write d0/d2/f1d [342926,121658] 0 2024-04-24T22:45:19.349 INFO:tasks.workunit.client.0.smithi063.stdout:9/166: link d6/ce d6/d8/d1f/c3a 0 2024-04-24T22:45:19.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/185: dread - d6/da/dc/d26/f43 zero size 2024-04-24T22:45:19.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/126: rmdir d0 39 2024-04-24T22:45:19.351 INFO:tasks.workunit.client.0.smithi063.stdout:0/186: chown d6/da/d2b/l37 108220 1 2024-04-24T22:45:19.353 INFO:tasks.workunit.client.0.smithi063.stdout:7/160: rmdir d0/d4/de/d17 39 2024-04-24T22:45:19.353 INFO:tasks.workunit.client.0.smithi063.stdout:0/187: write d6/dd/d20/d21/d25/d31/f35 [1593706,118601] 0 2024-04-24T22:45:19.354 INFO:tasks.workunit.client.0.smithi063.stdout:0/188: read d6/da/dc/f17 [421250,112187] 0 2024-04-24T22:45:19.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/167: creat d3/da/dc/f3a x:0 0 0 2024-04-24T22:45:19.360 INFO:tasks.workunit.client.1.smithi192.stdout:5/127: dwrite d0/d1b/d1c/f20 [0,4194304] 0 2024-04-24T22:45:19.361 INFO:tasks.workunit.client.1.smithi192.stdout:5/128: chown d0/d1b/f25 16722307 1 2024-04-24T22:45:19.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/129: write d0/d1b/d1c/f20 [298372,116816] 0 2024-04-24T22:45:19.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/130: chown d0/d1b/d1c 400 1 2024-04-24T22:45:19.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/131: fsync d0/f23 0 2024-04-24T22:45:19.364 INFO:tasks.workunit.client.0.smithi063.stdout:2/169: rename d0/dd/c1f to d0/dd/c32 0 2024-04-24T22:45:19.365 INFO:tasks.workunit.client.0.smithi063.stdout:9/167: creat d6/d8/d1f/d20/d22/f3b x:0 0 0 2024-04-24T22:45:19.365 INFO:tasks.workunit.client.0.smithi063.stdout:1/168: dwrite d3/da/dc/f1c [4194304,4194304] 0 2024-04-24T22:45:19.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/171: truncate d2/d19/f20 1645342 0 2024-04-24T22:45:19.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/170: write d0/d2/f1d [1685913,107922] 0 2024-04-24T22:45:19.369 INFO:tasks.workunit.client.0.smithi063.stdout:4/172: write d2/f1c [1264126,97164] 0 2024-04-24T22:45:19.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/203: mknod d1/d11/d32/d38/c45 0 2024-04-24T22:45:19.379 INFO:tasks.workunit.client.0.smithi063.stdout:3/188: mkdir da/d40 0 2024-04-24T22:45:19.380 INFO:tasks.workunit.client.0.smithi063.stdout:6/189: mkdir db/d1d/d20/d3f 0 2024-04-24T22:45:19.381 INFO:tasks.workunit.client.1.smithi192.stdout:5/132: dread d0/f18 [4194304,4194304] 0 2024-04-24T22:45:19.381 INFO:tasks.workunit.client.1.smithi192.stdout:5/133: dread - d0/f23 zero size 2024-04-24T22:45:19.383 INFO:tasks.workunit.client.0.smithi063.stdout:0/189: unlink d6/dd/d20/d21/c22 0 2024-04-24T22:45:19.491 INFO:tasks.workunit.client.0.smithi063.stdout:5/190: link f1 d5/d30/d3b/f48 0 2024-04-24T22:45:19.492 INFO:tasks.workunit.client.0.smithi063.stdout:2/171: creat d0/f33 x:0 0 0 2024-04-24T22:45:19.493 INFO:tasks.workunit.client.0.smithi063.stdout:1/169: symlink d3/da/dc/d1f/d21/l3b 0 2024-04-24T22:45:19.495 INFO:tasks.workunit.client.0.smithi063.stdout:2/172: dread d0/d2/d3/d7/f9 [0,4194304] 0 2024-04-24T22:45:19.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/173: dread d0/d2/d3/d7/f13 [0,4194304] 0 2024-04-24T22:45:19.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/174: stat d0/d2/d3/d1c 0 2024-04-24T22:45:19.576 INFO:tasks.workunit.client.1.smithi192.stdout:7/139: sync 2024-04-24T22:45:19.626 INFO:tasks.workunit.client.0.smithi063.stdout:3/189: readlink da/lf 0 2024-04-24T22:45:19.628 INFO:tasks.workunit.client.1.smithi192.stdout:7/140: creat d2/db/d2a/f2b x:0 0 0 2024-04-24T22:45:19.628 INFO:tasks.workunit.client.0.smithi063.stdout:6/190: chown db/d1c/c22 6423077 1 2024-04-24T22:45:19.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/134: write d0/f18 [4859583,9993] 0 2024-04-24T22:45:19.629 INFO:tasks.workunit.client.0.smithi063.stdout:0/190: symlink d6/dd/d20/d24/l46 0 2024-04-24T22:45:19.629 INFO:tasks.workunit.client.0.smithi063.stdout:6/191: readlink db/d1d/d20/l3b 0 2024-04-24T22:45:19.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/191: truncate d6/da/dc/d26/f43 285881 0 2024-04-24T22:45:19.631 INFO:tasks.workunit.client.0.smithi063.stdout:0/192: write d6/da/f29 [257629,52382] 0 2024-04-24T22:45:19.632 INFO:tasks.workunit.client.0.smithi063.stdout:9/168: write d6/f7 [1562054,121854] 0 2024-04-24T22:45:19.633 INFO:tasks.workunit.client.1.smithi192.stdout:5/135: mkdir d0/d1b/d1c/d2b 0 2024-04-24T22:45:19.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/136: fdatasync d0/f18 0 2024-04-24T22:45:19.636 INFO:tasks.workunit.client.0.smithi063.stdout:5/191: rename d5/d30 to d5/d24/d49 0 2024-04-24T22:45:19.636 INFO:tasks.workunit.client.0.smithi063.stdout:6/192: dread db/d1d/d20/f2b [4194304,4194304] 0 2024-04-24T22:45:19.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/137: dwrite d0/f23 [0,4194304] 0 2024-04-24T22:45:19.643 INFO:tasks.workunit.client.1.smithi192.stdout:5/138: chown d0/d13/d1f/f2a 14200 1 2024-04-24T22:45:19.649 INFO:tasks.workunit.client.0.smithi063.stdout:7/161: dwrite d0/d4/f11 [0,4194304] 0 2024-04-24T22:45:19.650 INFO:tasks.workunit.client.1.smithi192.stdout:5/139: mkdir d0/d1b/d1c/d2c 0 2024-04-24T22:45:19.659 INFO:tasks.workunit.client.1.smithi192.stdout:5/140: dwrite d0/d1b/f25 [0,4194304] 0 2024-04-24T22:45:19.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/141: mkdir d0/d13/d19/d2d 0 2024-04-24T22:45:19.670 INFO:tasks.workunit.client.1.smithi192.stdout:7/141: creat d2/f2c x:0 0 0 2024-04-24T22:45:19.671 INFO:tasks.workunit.client.0.smithi063.stdout:0/193: rmdir d6/dd/d20/d21/d25/d31 39 2024-04-24T22:45:19.673 INFO:tasks.workunit.client.1.smithi192.stdout:5/142: dwrite d0/f18 [4194304,4194304] 0 2024-04-24T22:45:19.677 INFO:tasks.workunit.client.0.smithi063.stdout:6/193: write db/d1d/d2d/f33 [1711718,72955] 0 2024-04-24T22:45:19.678 INFO:tasks.workunit.client.0.smithi063.stdout:7/162: fsync d0/d4/f11 0 2024-04-24T22:45:19.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/142: unlink d2/db/f24 0 2024-04-24T22:45:19.680 INFO:tasks.workunit.client.1.smithi192.stdout:7/143: read d2/db/f20 [5127653,92757] 0 2024-04-24T22:45:19.681 INFO:tasks.workunit.client.1.smithi192.stdout:5/143: mknod d0/d1b/c2e 0 2024-04-24T22:45:19.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/192: dwrite d5/d11/f2f [0,4194304] 0 2024-04-24T22:45:19.684 INFO:tasks.workunit.client.0.smithi063.stdout:5/193: stat d5/d11/d23 0 2024-04-24T22:45:19.687 INFO:tasks.workunit.client.0.smithi063.stdout:7/163: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:19.690 INFO:tasks.workunit.client.1.smithi192.stdout:5/144: rename d0/d13/d1f to d0/d1b/d1c/d2b/d2f 0 2024-04-24T22:45:19.691 INFO:tasks.workunit.client.1.smithi192.stdout:5/145: write d0/f10 [757562,3412] 0 2024-04-24T22:45:19.691 INFO:tasks.workunit.client.1.smithi192.stdout:5/146: readlink d0/l8 0 2024-04-24T22:45:19.699 INFO:tasks.workunit.client.0.smithi063.stdout:2/175: unlink d0/d2/d3/d7/f9 0 2024-04-24T22:45:19.700 INFO:tasks.workunit.client.0.smithi063.stdout:2/176: readlink d0/d2/d3/d1c/l20 0 2024-04-24T22:45:19.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/177: write d0/d2/d3/f25 [374299,41039] 0 2024-04-24T22:45:19.705 INFO:tasks.workunit.client.0.smithi063.stdout:0/194: chown d6/dd/c30 0 1 2024-04-24T22:45:19.714 INFO:tasks.workunit.client.0.smithi063.stdout:8/204: link d1/f1f d1/d2/d3/f46 0 2024-04-24T22:45:19.716 INFO:tasks.workunit.client.0.smithi063.stdout:9/169: link d6/d8/db/d1c/c32 d6/d8/db/d2b/c3c 0 2024-04-24T22:45:19.717 INFO:tasks.workunit.client.0.smithi063.stdout:7/164: dwrite d0/d4/de/d17/d28/f3c [0,4194304] 0 2024-04-24T22:45:19.718 INFO:tasks.workunit.client.0.smithi063.stdout:7/165: fsync d0/d4/f11 0 2024-04-24T22:45:19.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/205: creat d1/d11/f47 x:0 0 0 2024-04-24T22:45:19.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/170: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:19.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/206: dread - d1/d2/f39 zero size 2024-04-24T22:45:19.726 INFO:tasks.workunit.client.0.smithi063.stdout:6/194: creat db/d1d/d20/f40 x:0 0 0 2024-04-24T22:45:19.727 INFO:tasks.workunit.client.0.smithi063.stdout:3/190: rmdir da 39 2024-04-24T22:45:19.729 INFO:tasks.workunit.client.0.smithi063.stdout:8/207: dwrite d1/d2/f1c [0,4194304] 0 2024-04-24T22:45:19.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/171: truncate d6/f1b 436919 0 2024-04-24T22:45:19.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/172: chown d6/d8/d1f/d20/d2a 139376606 1 2024-04-24T22:45:19.733 INFO:tasks.workunit.client.0.smithi063.stdout:3/191: dread - da/f3b zero size 2024-04-24T22:45:19.736 INFO:tasks.workunit.client.0.smithi063.stdout:8/208: symlink d1/d11/d32/d38/l48 0 2024-04-24T22:45:19.738 INFO:tasks.workunit.client.0.smithi063.stdout:9/173: unlink d6/d8/ff 0 2024-04-24T22:45:19.740 INFO:tasks.workunit.client.0.smithi063.stdout:4/173: write d2/d8/fe [1235700,56303] 0 2024-04-24T22:45:19.741 INFO:tasks.workunit.client.0.smithi063.stdout:8/209: mkdir d1/d11/d49 0 2024-04-24T22:45:19.742 INFO:tasks.workunit.client.0.smithi063.stdout:4/174: dread d2/f26 [0,4194304] 0 2024-04-24T22:45:19.743 INFO:tasks.workunit.client.0.smithi063.stdout:4/175: read - d2/d19/f1f zero size 2024-04-24T22:45:19.745 INFO:tasks.workunit.client.0.smithi063.stdout:3/192: creat da/d39/f41 x:0 0 0 2024-04-24T22:45:19.747 INFO:tasks.workunit.client.0.smithi063.stdout:9/174: fdatasync d6/f1b 0 2024-04-24T22:45:19.750 INFO:tasks.workunit.client.0.smithi063.stdout:3/193: symlink da/d40/l42 0 2024-04-24T22:45:19.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/176: dwrite d2/d8/f1d [0,4194304] 0 2024-04-24T22:45:19.751 INFO:tasks.workunit.client.0.smithi063.stdout:8/210: dread d1/f5 [0,4194304] 0 2024-04-24T22:45:19.751 INFO:tasks.workunit.client.0.smithi063.stdout:4/177: dread - d2/d19/f1f zero size 2024-04-24T22:45:19.752 INFO:tasks.workunit.client.0.smithi063.stdout:9/175: creat d6/d8/db/d2b/f3d x:0 0 0 2024-04-24T22:45:19.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/194: stat da/fc 0 2024-04-24T22:45:19.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/195: stat da/f2c 0 2024-04-24T22:45:19.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/196: symlink da/l43 0 2024-04-24T22:45:19.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/197: stat da/lf 0 2024-04-24T22:45:19.761 INFO:tasks.workunit.client.0.smithi063.stdout:3/198: truncate da/f36 328220 0 2024-04-24T22:45:19.762 INFO:tasks.workunit.client.0.smithi063.stdout:9/176: link d6/d8/l16 d6/d8/d1f/d20/d22/l3e 0 2024-04-24T22:45:19.763 INFO:tasks.workunit.client.0.smithi063.stdout:3/199: chown da/ce 1 1 2024-04-24T22:45:19.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/200: write da/f2e [245295,924] 0 2024-04-24T22:45:19.768 INFO:tasks.workunit.client.0.smithi063.stdout:3/201: unlink da/d18/f3e 0 2024-04-24T22:45:19.768 INFO:tasks.workunit.client.0.smithi063.stdout:3/202: chown l9 145417192 1 2024-04-24T22:45:19.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/178: rename d2/f13 to d2/d8/f32 0 2024-04-24T22:45:19.771 INFO:tasks.workunit.client.0.smithi063.stdout:3/203: dread - da/d18/d3a/f3f zero size 2024-04-24T22:45:19.772 INFO:tasks.workunit.client.0.smithi063.stdout:4/179: mkdir d2/d19/d33 0 2024-04-24T22:45:19.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/170: creat d3/f3c x:0 0 0 2024-04-24T22:45:19.775 INFO:tasks.workunit.client.0.smithi063.stdout:1/171: dread d3/f17 [0,4194304] 0 2024-04-24T22:45:19.776 INFO:tasks.workunit.client.0.smithi063.stdout:4/180: mknod d2/c34 0 2024-04-24T22:45:19.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/147: rmdir d0/d1b/d1c/d2b/d2f 39 2024-04-24T22:45:19.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/148: readlink d0/d1b/l26 0 2024-04-24T22:45:19.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/172: creat d3/da/f3d x:0 0 0 2024-04-24T22:45:19.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/173: write f1 [4036694,62433] 0 2024-04-24T22:45:19.780 INFO:tasks.workunit.client.0.smithi063.stdout:3/204: rename da/d18/c38 to da/d18/c44 0 2024-04-24T22:45:19.781 INFO:tasks.workunit.client.1.smithi192.stdout:7/144: truncate f1 1974334 0 2024-04-24T22:45:19.782 INFO:tasks.workunit.client.0.smithi063.stdout:1/174: fsync f1 0 2024-04-24T22:45:19.782 INFO:tasks.workunit.client.0.smithi063.stdout:1/175: readlink d3/l8 0 2024-04-24T22:45:19.782 INFO:tasks.workunit.client.0.smithi063.stdout:1/176: readlink d3/da/l1a 0 2024-04-24T22:45:19.785 INFO:tasks.workunit.client.0.smithi063.stdout:1/177: dread d3/da/dc/d11/f33 [0,4194304] 0 2024-04-24T22:45:19.786 INFO:tasks.workunit.client.0.smithi063.stdout:3/205: creat da/d39/f45 x:0 0 0 2024-04-24T22:45:19.787 INFO:tasks.workunit.client.0.smithi063.stdout:3/206: chown da/d39 17689 1 2024-04-24T22:45:19.787 INFO:tasks.workunit.client.1.smithi192.stdout:5/149: creat d0/f30 x:0 0 0 2024-04-24T22:45:19.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/150: fsync d0/fc 0 2024-04-24T22:45:19.789 INFO:tasks.workunit.client.1.smithi192.stdout:5/151: stat d0/c3 0 2024-04-24T22:45:19.789 INFO:tasks.workunit.client.1.smithi192.stdout:7/145: dread d2/f23 [0,4194304] 0 2024-04-24T22:45:19.789 INFO:tasks.workunit.client.1.smithi192.stdout:5/152: chown d0/d1b/d1c/f27 3607773 1 2024-04-24T22:45:19.790 INFO:tasks.workunit.client.1.smithi192.stdout:5/153: write d0/fc [4179849,115675] 0 2024-04-24T22:45:19.790 INFO:tasks.workunit.client.0.smithi063.stdout:1/178: stat d3/c31 0 2024-04-24T22:45:19.792 INFO:tasks.workunit.client.0.smithi063.stdout:3/207: dwrite da/d18/d3a/f3f [0,4194304] 0 2024-04-24T22:45:19.794 INFO:tasks.workunit.client.1.smithi192.stdout:7/146: write d2/fa [4020459,57447] 0 2024-04-24T22:45:19.794 INFO:tasks.workunit.client.1.smithi192.stdout:5/154: dread d0/f18 [0,4194304] 0 2024-04-24T22:45:19.794 INFO:tasks.workunit.client.0.smithi063.stdout:4/181: creat d2/f35 x:0 0 0 2024-04-24T22:45:19.796 INFO:tasks.workunit.client.0.smithi063.stdout:8/211: fsync d1/f1f 0 2024-04-24T22:45:19.797 INFO:tasks.workunit.client.0.smithi063.stdout:4/182: write d2/d8/f31 [304611,130576] 0 2024-04-24T22:45:19.798 INFO:tasks.workunit.client.1.smithi192.stdout:6/118: sync 2024-04-24T22:45:19.799 INFO:tasks.workunit.client.0.smithi063.stdout:1/179: dwrite d3/da/f37 [0,4194304] 0 2024-04-24T22:45:19.800 INFO:tasks.workunit.client.0.smithi063.stdout:1/180: fsync d3/da/dc/f3a 0 2024-04-24T22:45:19.802 INFO:tasks.workunit.client.0.smithi063.stdout:3/208: stat da/c2b 0 2024-04-24T22:45:19.802 INFO:tasks.workunit.client.0.smithi063.stdout:5/194: truncate f0 1560499 0 2024-04-24T22:45:19.803 INFO:tasks.workunit.client.1.smithi192.stdout:6/119: dread d0/d8/fa [0,4194304] 0 2024-04-24T22:45:19.804 INFO:tasks.workunit.client.1.smithi192.stdout:6/120: truncate d0/f19 681097 0 2024-04-24T22:45:19.804 INFO:tasks.workunit.client.0.smithi063.stdout:4/183: dwrite d2/d8/f2c [0,4194304] 0 2024-04-24T22:45:19.806 INFO:tasks.workunit.client.0.smithi063.stdout:4/184: write d2/d8/f2c [2188210,110347] 0 2024-04-24T22:45:19.810 INFO:tasks.workunit.client.0.smithi063.stdout:6/195: unlink l5 0 2024-04-24T22:45:19.811 INFO:tasks.workunit.client.0.smithi063.stdout:3/209: dwrite da/d18/f2a [0,4194304] 0 2024-04-24T22:45:19.815 INFO:tasks.workunit.client.1.smithi192.stdout:6/121: dwrite d0/d8/fe [0,4194304] 0 2024-04-24T22:45:19.815 INFO:tasks.workunit.client.1.smithi192.stdout:5/155: dread d0/f18 [0,4194304] 0 2024-04-24T22:45:19.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/185: dread d2/f1c [0,4194304] 0 2024-04-24T22:45:19.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/210: read da/f36 [174899,100628] 0 2024-04-24T22:45:19.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/212: rename d1/d21/c37 to d1/d11/d49/c4a 0 2024-04-24T22:45:19.819 INFO:tasks.workunit.client.0.smithi063.stdout:4/186: dwrite d2/f1c [0,4194304] 0 2024-04-24T22:45:19.825 INFO:tasks.workunit.client.0.smithi063.stdout:9/177: dread d6/f1b [0,4194304] 0 2024-04-24T22:45:19.830 INFO:tasks.workunit.client.0.smithi063.stdout:9/178: truncate d6/d2f/f35 478391 0 2024-04-24T22:45:19.830 INFO:tasks.workunit.client.1.smithi192.stdout:6/122: dwrite d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:19.832 INFO:tasks.workunit.client.0.smithi063.stdout:7/166: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:19.834 INFO:tasks.workunit.client.1.smithi192.stdout:7/147: rename d2/db/d21/c29 to d2/c2d 0 2024-04-24T22:45:19.835 INFO:tasks.workunit.client.0.smithi063.stdout:7/167: fsync d0/f3d 0 2024-04-24T22:45:19.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/195: dwrite d5/d11/d18/f3a [0,4194304] 0 2024-04-24T22:45:19.837 INFO:tasks.workunit.client.0.smithi063.stdout:5/196: chown d5/d24/d2c/f3e 0 1 2024-04-24T22:45:19.842 INFO:tasks.workunit.client.0.smithi063.stdout:2/178: unlink d0/l23 0 2024-04-24T22:45:19.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/179: write d0/d2/d27/f29 [437095,8666] 0 2024-04-24T22:45:19.844 INFO:tasks.workunit.client.0.smithi063.stdout:1/181: rename d3/da/dc/d11/f33 to d3/da/dc/d1f/f3e 0 2024-04-24T22:45:19.845 INFO:tasks.workunit.client.1.smithi192.stdout:7/148: rename d2/f23 to d2/db/d2a/f2e 0 2024-04-24T22:45:19.846 INFO:tasks.workunit.client.0.smithi063.stdout:2/180: write d0/d2/f1e [997598,12483] 0 2024-04-24T22:45:19.847 INFO:tasks.workunit.client.0.smithi063.stdout:8/213: creat d1/d2/d17/f4b x:0 0 0 2024-04-24T22:45:19.848 INFO:tasks.workunit.client.1.smithi192.stdout:5/156: rmdir d0/d13/d19/d2d 0 2024-04-24T22:45:19.851 INFO:tasks.workunit.client.1.smithi192.stdout:6/123: mkdir d0/d28 0 2024-04-24T22:45:19.855 INFO:tasks.workunit.client.1.smithi192.stdout:7/149: symlink d2/db/d21/l2f 0 2024-04-24T22:45:19.856 INFO:tasks.workunit.client.1.smithi192.stdout:7/150: chown d2/c26 0 1 2024-04-24T22:45:19.861 INFO:tasks.workunit.client.0.smithi063.stdout:9/179: creat d6/d8/d1f/d20/f3f x:0 0 0 2024-04-24T22:45:19.862 INFO:tasks.workunit.client.0.smithi063.stdout:9/180: write d6/d8/d24/f23 [150749,108889] 0 2024-04-24T22:45:19.867 INFO:tasks.workunit.client.1.smithi192.stdout:6/124: mkdir d0/d8/d29 0 2024-04-24T22:45:19.868 INFO:tasks.workunit.client.1.smithi192.stdout:6/125: read d0/d8/fc [2131945,50437] 0 2024-04-24T22:45:19.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/168: creat d0/d4/de/d17/f43 x:0 0 0 2024-04-24T22:45:19.870 INFO:tasks.workunit.client.1.smithi192.stdout:7/151: unlink d2/l8 0 2024-04-24T22:45:19.872 INFO:tasks.workunit.client.1.smithi192.stdout:7/152: chown d2/db/d21/f28 1807894 1 2024-04-24T22:45:19.872 INFO:tasks.workunit.client.1.smithi192.stdout:6/126: dread d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:19.873 INFO:tasks.workunit.client.0.smithi063.stdout:5/197: mknod d5/d1d/c4a 0 2024-04-24T22:45:19.874 INFO:tasks.workunit.client.0.smithi063.stdout:3/211: fdatasync da/f14 0 2024-04-24T22:45:19.875 INFO:tasks.workunit.client.0.smithi063.stdout:3/212: truncate da/d39/f41 985487 0 2024-04-24T22:45:19.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/181: fdatasync d0/d2/d27/f29 0 2024-04-24T22:45:19.877 INFO:tasks.workunit.client.0.smithi063.stdout:3/213: write da/f28 [3230178,113530] 0 2024-04-24T22:45:19.879 INFO:tasks.workunit.client.0.smithi063.stdout:1/182: creat d3/da/dc/d1f/f3f x:0 0 0 2024-04-24T22:45:19.884 INFO:tasks.workunit.client.1.smithi192.stdout:7/153: dwrite d2/db/f10 [0,4194304] 0 2024-04-24T22:45:19.892 INFO:tasks.workunit.client.1.smithi192.stdout:7/154: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:19.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/169: symlink d0/d4/de/d17/l44 0 2024-04-24T22:45:19.897 INFO:tasks.workunit.client.1.smithi192.stdout:6/127: creat d0/d8/f2a x:0 0 0 2024-04-24T22:45:19.898 INFO:tasks.workunit.client.1.smithi192.stdout:7/155: dread d2/db/d2a/f2e [0,4194304] 0 2024-04-24T22:45:19.899 INFO:tasks.workunit.client.1.smithi192.stdout:7/156: stat d2/db/c1c 0 2024-04-24T22:45:19.903 INFO:tasks.workunit.client.0.smithi063.stdout:2/182: creat d0/d2/d27/f34 x:0 0 0 2024-04-24T22:45:19.906 INFO:tasks.workunit.client.0.smithi063.stdout:1/183: mknod d3/da/dc/d11/c40 0 2024-04-24T22:45:19.907 INFO:tasks.workunit.client.1.smithi192.stdout:7/157: dread d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:19.908 INFO:tasks.workunit.client.0.smithi063.stdout:8/214: truncate d1/d11/f15 281790 0 2024-04-24T22:45:19.909 INFO:tasks.workunit.client.0.smithi063.stdout:6/196: dwrite db/f1a [0,4194304] 0 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.0.smithi063.stdout:6/197: chown db/d1d/d2d/f33 1789924 1 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.0.smithi063.stdout:6/198: readlink db/l1f 0 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.1.smithi192.stdout:0/236: sync 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/157: sync 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.1.smithi192.stdout:1/168: sync 2024-04-24T22:45:19.910 INFO:tasks.workunit.client.1.smithi192.stdout:4/94: sync 2024-04-24T22:45:19.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/128: rename d0/d8/fa to d0/d8/f2b 0 2024-04-24T22:45:19.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/129: chown d0 157070036 1 2024-04-24T22:45:19.911 INFO:tasks.workunit.client.1.smithi192.stdout:4/95: stat d2/db/d10/c15 0 2024-04-24T22:45:19.911 INFO:tasks.workunit.client.0.smithi063.stdout:6/199: write db/f10 [3320411,48699] 0 2024-04-24T22:45:19.912 INFO:tasks.workunit.client.0.smithi063.stdout:9/181: creat d6/d8/db/d2b/d36/f40 x:0 0 0 2024-04-24T22:45:19.913 INFO:tasks.workunit.client.0.smithi063.stdout:6/200: truncate db/d1d/d20/f40 167833 0 2024-04-24T22:45:19.914 INFO:tasks.workunit.client.1.smithi192.stdout:7/158: fsync d2/f1a 0 2024-04-24T22:45:19.916 INFO:tasks.workunit.client.0.smithi063.stdout:7/170: creat d0/d26/f45 x:0 0 0 2024-04-24T22:45:19.917 INFO:tasks.workunit.client.0.smithi063.stdout:7/171: chown d0/d26/c38 758 1 2024-04-24T22:45:19.919 INFO:tasks.workunit.client.0.smithi063.stdout:5/198: rename d5/d11/l40 to d5/d24/d2c/l4b 0 2024-04-24T22:45:19.920 INFO:tasks.workunit.client.0.smithi063.stdout:7/172: dread d0/d4/de/f21 [0,4194304] 0 2024-04-24T22:45:19.923 INFO:tasks.workunit.client.1.smithi192.stdout:1/169: symlink d9/d15/d2d/l3b 0 2024-04-24T22:45:19.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/237: dread d3/d27/d28/f34 [0,4194304] 0 2024-04-24T22:45:19.928 INFO:tasks.workunit.client.1.smithi192.stdout:5/157: write d0/d13/d19/f1d [165908,126349] 0 2024-04-24T22:45:19.930 INFO:tasks.workunit.client.0.smithi063.stdout:0/195: sync 2024-04-24T22:45:19.931 INFO:tasks.workunit.client.1.smithi192.stdout:4/96: unlink d2/l8 0 2024-04-24T22:45:19.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/158: dread d0/f18 [4194304,4194304] 0 2024-04-24T22:45:19.935 INFO:tasks.workunit.client.1.smithi192.stdout:3/158: mknod d2/d2d/c37 0 2024-04-24T22:45:19.941 INFO:tasks.workunit.client.0.smithi063.stdout:5/199: truncate f1 1139800 0 2024-04-24T22:45:19.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/200: write d5/d11/d18/f19 [526164,112728] 0 2024-04-24T22:45:19.945 INFO:tasks.workunit.client.1.smithi192.stdout:6/130: mknod d0/d8/c2c 0 2024-04-24T22:45:19.946 INFO:tasks.workunit.client.0.smithi063.stdout:7/173: unlink d0/d4/de/d17/d28/f3c 0 2024-04-24T22:45:19.950 INFO:tasks.workunit.client.0.smithi063.stdout:4/187: truncate f0 2993116 0 2024-04-24T22:45:19.951 INFO:tasks.workunit.client.1.smithi192.stdout:4/97: mkdir d2/d9/d1e/d28 0 2024-04-24T22:45:19.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/196: readlink d6/da/dc/d26/l3c 0 2024-04-24T22:45:19.953 INFO:tasks.workunit.client.1.smithi192.stdout:8/134: sync 2024-04-24T22:45:19.953 INFO:tasks.workunit.client.1.smithi192.stdout:9/138: sync 2024-04-24T22:45:19.953 INFO:tasks.workunit.client.1.smithi192.stdout:2/239: sync 2024-04-24T22:45:19.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/98: truncate d2/db/d10/f27 100245 0 2024-04-24T22:45:19.954 INFO:tasks.workunit.client.1.smithi192.stdout:9/139: chown d3/dd/de/l1e 3 1 2024-04-24T22:45:19.954 INFO:tasks.workunit.client.0.smithi063.stdout:6/201: mknod db/d1d/d20/d3f/c41 0 2024-04-24T22:45:19.955 INFO:tasks.workunit.client.0.smithi063.stdout:6/202: write db/d1d/d2d/f36 [247028,2529] 0 2024-04-24T22:45:19.956 INFO:tasks.workunit.client.0.smithi063.stdout:6/203: fdatasync db/d1d/d20/d29/f2e 0 2024-04-24T22:45:19.956 INFO:tasks.workunit.client.1.smithi192.stdout:8/135: dread f8 [0,4194304] 0 2024-04-24T22:45:19.957 INFO:tasks.workunit.client.1.smithi192.stdout:8/136: read db/dc/f18 [211329,17974] 0 2024-04-24T22:45:19.959 INFO:tasks.workunit.client.0.smithi063.stdout:1/184: rename d3/f28 to d3/da/dc/d11/f41 0 2024-04-24T22:45:19.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/185: read d3/f2c [80380,37199] 0 2024-04-24T22:45:19.960 INFO:tasks.workunit.client.0.smithi063.stdout:3/214: getdents da/d18/d3a 0 2024-04-24T22:45:19.962 INFO:tasks.workunit.client.0.smithi063.stdout:3/215: truncate da/f3d 229617 0 2024-04-24T22:45:19.964 INFO:tasks.workunit.client.1.smithi192.stdout:2/240: dread d4/d9/fb [0,4194304] 0 2024-04-24T22:45:19.965 INFO:tasks.workunit.client.0.smithi063.stdout:5/201: mknod d5/d11/d23/d2d/c4c 0 2024-04-24T22:45:19.966 INFO:tasks.workunit.client.0.smithi063.stdout:5/202: dread - d5/d11/d23/f27 zero size 2024-04-24T22:45:19.966 INFO:tasks.workunit.client.0.smithi063.stdout:1/186: dwrite d3/d2d/f2e [0,4194304] 0 2024-04-24T22:45:19.967 INFO:tasks.workunit.client.0.smithi063.stdout:5/203: stat d5/d1d/c4a 0 2024-04-24T22:45:19.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/216: dread da/d18/f2a [0,4194304] 0 2024-04-24T22:45:19.969 INFO:tasks.workunit.client.0.smithi063.stdout:4/188: truncate d2/d8/f11 4252654 0 2024-04-24T22:45:19.970 INFO:tasks.workunit.client.1.smithi192.stdout:6/131: dread d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:19.971 INFO:tasks.workunit.client.0.smithi063.stdout:4/189: write d2/d19/f25 [347260,19773] 0 2024-04-24T22:45:19.974 INFO:tasks.workunit.client.0.smithi063.stdout:0/197: symlink d6/dd/d20/d21/l47 0 2024-04-24T22:45:19.975 INFO:tasks.workunit.client.1.smithi192.stdout:9/140: creat d3/dd/f30 x:0 0 0 2024-04-24T22:45:19.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/204: stat db/d1d/c3d 0 2024-04-24T22:45:19.978 INFO:tasks.workunit.client.0.smithi063.stdout:9/182: write d6/f7 [734386,78755] 0 2024-04-24T22:45:19.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/215: dwrite d1/f1f [0,4194304] 0 2024-04-24T22:45:19.984 INFO:tasks.workunit.client.1.smithi192.stdout:2/241: chown d4/f13 2 1 2024-04-24T22:45:19.985 INFO:tasks.workunit.client.1.smithi192.stdout:1/170: creat d9/f3c x:0 0 0 2024-04-24T22:45:19.987 INFO:tasks.workunit.client.0.smithi063.stdout:1/187: mkdir d3/da/dc/d1f/d21/d42 0 2024-04-24T22:45:19.987 INFO:tasks.workunit.client.0.smithi063.stdout:1/188: read f1 [3834141,110322] 0 2024-04-24T22:45:19.988 INFO:tasks.workunit.client.0.smithi063.stdout:1/189: dread - d3/da/dc/d1f/f3f zero size 2024-04-24T22:45:19.989 INFO:tasks.workunit.client.0.smithi063.stdout:1/190: stat d3/da/dc/c14 0 2024-04-24T22:45:19.993 INFO:tasks.workunit.client.1.smithi192.stdout:9/141: creat d3/f31 x:0 0 0 2024-04-24T22:45:19.993 INFO:tasks.workunit.client.1.smithi192.stdout:0/238: write d3/d27/d28/f1d [2026116,70880] 0 2024-04-24T22:45:19.995 INFO:tasks.workunit.client.0.smithi063.stdout:5/204: rename d5/d11/d23/d2d/c4c to d5/d24/d36/c4d 0 2024-04-24T22:45:19.997 INFO:tasks.workunit.client.1.smithi192.stdout:4/99: rmdir d2/d9 39 2024-04-24T22:45:19.998 INFO:tasks.workunit.client.1.smithi192.stdout:5/159: creat d0/f31 x:0 0 0 2024-04-24T22:45:19.999 INFO:tasks.workunit.client.1.smithi192.stdout:7/159: truncate d2/db/d2a/f2e 1162200 0 2024-04-24T22:45:19.999 INFO:tasks.workunit.client.1.smithi192.stdout:5/160: dread - d0/f30 zero size 2024-04-24T22:45:20.000 INFO:tasks.workunit.client.1.smithi192.stdout:5/161: write d0/f30 [204106,26654] 0 2024-04-24T22:45:20.001 INFO:tasks.workunit.client.1.smithi192.stdout:8/137: creat db/dc/d22/f23 x:0 0 0 2024-04-24T22:45:20.003 INFO:tasks.workunit.client.0.smithi063.stdout:0/198: creat d6/dd/d20/d21/f48 x:0 0 0 2024-04-24T22:45:20.009 INFO:tasks.workunit.client.0.smithi063.stdout:9/183: symlink d6/d8/l41 0 2024-04-24T22:45:20.010 INFO:tasks.workunit.client.1.smithi192.stdout:2/242: symlink d4/d9/d36/l48 0 2024-04-24T22:45:20.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/184: write d6/d8/d1f/d20/d22/f3b [179438,73626] 0 2024-04-24T22:45:20.011 INFO:tasks.workunit.client.0.smithi063.stdout:7/174: write d0/d4/f19 [169487,44487] 0 2024-04-24T22:45:20.013 INFO:tasks.workunit.client.0.smithi063.stdout:7/175: chown d0/d2f/c30 52278250 1 2024-04-24T22:45:20.013 INFO:tasks.workunit.client.0.smithi063.stdout:9/185: read - d6/d8/d1f/d20/d22/d31/f33 zero size 2024-04-24T22:45:20.018 INFO:tasks.workunit.client.1.smithi192.stdout:6/132: mknod d0/d8/c2d 0 2024-04-24T22:45:20.019 INFO:tasks.workunit.client.1.smithi192.stdout:6/133: truncate d0/f19 685408 0 2024-04-24T22:45:20.019 INFO:tasks.workunit.client.1.smithi192.stdout:9/142: mknod d3/d2c/c32 0 2024-04-24T22:45:20.019 INFO:tasks.workunit.client.0.smithi063.stdout:1/191: write d3/f23 [116464,96371] 0 2024-04-24T22:45:20.020 INFO:tasks.workunit.client.1.smithi192.stdout:0/239: creat d3/d27/d28/d2e/f53 x:0 0 0 2024-04-24T22:45:20.023 INFO:tasks.workunit.client.0.smithi063.stdout:6/205: rename db/d1d/d20/f23 to db/d1d/d2d/f42 0 2024-04-24T22:45:20.026 INFO:tasks.workunit.client.0.smithi063.stdout:5/205: creat d5/d11/d23/f4e x:0 0 0 2024-04-24T22:45:20.027 INFO:tasks.workunit.client.0.smithi063.stdout:2/183: sync 2024-04-24T22:45:20.028 INFO:tasks.workunit.client.1.smithi192.stdout:8/138: creat db/dc/f24 x:0 0 0 2024-04-24T22:45:20.030 INFO:tasks.workunit.client.1.smithi192.stdout:3/159: getdents d2/d4/d12/d1a/d22 0 2024-04-24T22:45:20.034 INFO:tasks.workunit.client.0.smithi063.stdout:1/192: dread d3/da/dc/d11/f35 [0,4194304] 0 2024-04-24T22:45:20.035 INFO:tasks.workunit.client.0.smithi063.stdout:9/186: mkdir d6/d2f/d42 0 2024-04-24T22:45:20.036 INFO:tasks.workunit.client.1.smithi192.stdout:6/134: mkdir d0/d8/d18/d2e 0 2024-04-24T22:45:20.039 INFO:tasks.workunit.client.1.smithi192.stdout:9/143: mkdir d3/dd/de/d17/d33 0 2024-04-24T22:45:20.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/193: dwrite d3/da/dc/d1f/f30 [0,4194304] 0 2024-04-24T22:45:20.041 INFO:tasks.workunit.client.1.smithi192.stdout:0/240: mknod d3/d27/d3c/c54 0 2024-04-24T22:45:20.041 INFO:tasks.workunit.client.0.smithi063.stdout:1/194: read d3/da/dc/d1f/f3e [66481,36050] 0 2024-04-24T22:45:20.042 INFO:tasks.workunit.client.1.smithi192.stdout:0/241: write d3/f14 [4838373,24774] 0 2024-04-24T22:45:20.043 INFO:tasks.workunit.client.1.smithi192.stdout:0/242: write d3/d27/d28/f24 [934530,443] 0 2024-04-24T22:45:20.044 INFO:tasks.workunit.client.0.smithi063.stdout:3/217: link da/lf da/d39/l46 0 2024-04-24T22:45:20.045 INFO:tasks.workunit.client.1.smithi192.stdout:4/100: chown d2/d9/l1c 76797731 1 2024-04-24T22:45:20.046 INFO:tasks.workunit.client.1.smithi192.stdout:4/101: write d2/d9/d25/f23 [63386,56022] 0 2024-04-24T22:45:20.048 INFO:tasks.workunit.client.1.smithi192.stdout:7/160: rename d2/c18 to d2/db/c30 0 2024-04-24T22:45:20.050 INFO:tasks.workunit.client.1.smithi192.stdout:4/102: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:20.051 INFO:tasks.workunit.client.1.smithi192.stdout:4/103: stat d2/d9/d25 0 2024-04-24T22:45:20.052 INFO:tasks.workunit.client.1.smithi192.stdout:4/104: fsync d2/f17 0 2024-04-24T22:45:20.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:19 smithi063 ceph-mon[26912]: mgrmap e25: smithi192.ctmwwc(active, since 9s), standbys: smithi063.upcnvd 2024-04-24T22:45:20.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:19 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:45:20.053 INFO:tasks.workunit.client.0.smithi063.stdout:2/184: mkdir d0/d2/d35 0 2024-04-24T22:45:20.056 INFO:tasks.workunit.client.0.smithi063.stdout:2/185: dread d0/d2/d27/f29 [0,4194304] 0 2024-04-24T22:45:20.056 INFO:tasks.workunit.client.0.smithi063.stdout:2/186: chown d0/d2/d35 0 1 2024-04-24T22:45:20.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/243: chown d4/d9/d10/f11 12739 1 2024-04-24T22:45:20.060 INFO:tasks.workunit.client.0.smithi063.stdout:4/190: write d2/d8/f11 [3399972,64061] 0 2024-04-24T22:45:20.062 INFO:tasks.workunit.client.1.smithi192.stdout:1/171: truncate d9/dd/f19 3395260 0 2024-04-24T22:45:20.063 INFO:tasks.workunit.client.0.smithi063.stdout:9/187: symlink d6/d8/db/d1c/l43 0 2024-04-24T22:45:20.064 INFO:tasks.workunit.client.0.smithi063.stdout:9/188: fdatasync d6/f7 0 2024-04-24T22:45:20.065 INFO:tasks.workunit.client.0.smithi063.stdout:9/189: write d6/d8/db/d2b/f3d [1001787,122692] 0 2024-04-24T22:45:20.074 INFO:tasks.workunit.client.0.smithi063.stdout:1/195: fdatasync d3/da/dc/d11/f41 0 2024-04-24T22:45:20.075 INFO:tasks.workunit.client.1.smithi192.stdout:5/162: creat d0/d1b/d1c/d2b/f32 x:0 0 0 2024-04-24T22:45:20.077 INFO:tasks.workunit.client.1.smithi192.stdout:3/160: dwrite d2/d4/d7/f17 [0,4194304] 0 2024-04-24T22:45:20.078 INFO:tasks.workunit.client.1.smithi192.stdout:5/163: chown d0/d1b/d1c/d2b/f32 120 1 2024-04-24T22:45:20.079 INFO:tasks.workunit.client.0.smithi063.stdout:8/216: dwrite d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:20.079 INFO:tasks.workunit.client.0.smithi063.stdout:6/206: creat db/d1d/d20/d3e/f43 x:0 0 0 2024-04-24T22:45:20.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/217: readlink d1/d2/lf 0 2024-04-24T22:45:20.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/218: chown d1/d2 51 1 2024-04-24T22:45:20.085 INFO:tasks.workunit.client.0.smithi063.stdout:2/187: creat d0/f36 x:0 0 0 2024-04-24T22:45:20.086 INFO:tasks.workunit.client.0.smithi063.stdout:0/199: link d6/dd/l1f d6/dd/d20/d24/d42/l49 0 2024-04-24T22:45:20.092 INFO:tasks.workunit.client.0.smithi063.stdout:0/200: dwrite d6/dd/d20/f3b [0,4194304] 0 2024-04-24T22:45:20.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/190: rename d6/f1b to d6/d8/db/d2b/f44 0 2024-04-24T22:45:20.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/176: link d0/d4/de/d17/d28/c2b d0/c46 0 2024-04-24T22:45:20.095 INFO:tasks.workunit.client.0.smithi063.stdout:1/196: creat d3/da/dc/d1f/d2b/f43 x:0 0 0 2024-04-24T22:45:20.095 INFO:tasks.workunit.client.1.smithi192.stdout:7/161: symlink d2/db/d25/l31 0 2024-04-24T22:45:20.096 INFO:tasks.workunit.client.0.smithi063.stdout:3/218: mkdir da/d18/d47 0 2024-04-24T22:45:20.099 INFO:tasks.workunit.client.0.smithi063.stdout:8/219: chown d1/d11/d49/c4a 3608487 1 2024-04-24T22:45:20.100 INFO:tasks.workunit.client.1.smithi192.stdout:3/161: mkdir d2/d27/d28/d38 0 2024-04-24T22:45:20.102 INFO:tasks.workunit.client.1.smithi192.stdout:3/162: fdatasync d2/d4/f14 0 2024-04-24T22:45:20.103 INFO:tasks.workunit.client.0.smithi063.stdout:2/188: unlink d0/d2/d3/d7/l18 0 2024-04-24T22:45:20.104 INFO:tasks.workunit.client.1.smithi192.stdout:7/162: dwrite d2/f2c [0,4194304] 0 2024-04-24T22:45:20.105 INFO:tasks.workunit.client.1.smithi192.stdout:4/105: mknod d2/c29 0 2024-04-24T22:45:20.105 INFO:tasks.workunit.client.1.smithi192.stdout:5/164: creat d0/d1b/d1c/f33 x:0 0 0 2024-04-24T22:45:20.107 INFO:tasks.workunit.client.1.smithi192.stdout:2/244: mknod d4/d9/c49 0 2024-04-24T22:45:20.109 INFO:tasks.workunit.client.1.smithi192.stdout:6/135: rmdir d0/d28 0 2024-04-24T22:45:20.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/201: symlink d6/da/d2b/l4a 0 2024-04-24T22:45:20.112 INFO:tasks.workunit.client.0.smithi063.stdout:9/191: rmdir d6/d8/d24 39 2024-04-24T22:45:20.112 INFO:tasks.workunit.client.1.smithi192.stdout:9/144: link d3/fb d3/dd/de/d17/f34 0 2024-04-24T22:45:20.113 INFO:tasks.workunit.client.0.smithi063.stdout:9/192: fdatasync d6/d8/d1f/d20/d22/f3b 0 2024-04-24T22:45:20.114 INFO:tasks.workunit.client.0.smithi063.stdout:7/177: creat d0/d26/f47 x:0 0 0 2024-04-24T22:45:20.115 INFO:tasks.workunit.client.0.smithi063.stdout:9/193: truncate d6/d8/d1f/d20/f21 5035291 0 2024-04-24T22:45:20.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/178: chown d0/d4/c18 993 1 2024-04-24T22:45:20.116 INFO:tasks.workunit.client.0.smithi063.stdout:7/179: chown d0/f7 0 1 2024-04-24T22:45:20.117 INFO:tasks.workunit.client.0.smithi063.stdout:1/197: mknod d3/da/dc/d39/c44 0 2024-04-24T22:45:20.117 INFO:tasks.workunit.client.0.smithi063.stdout:7/180: stat d0/fa 0 2024-04-24T22:45:20.118 INFO:tasks.workunit.client.1.smithi192.stdout:8/139: getdents db/dc/d22 0 2024-04-24T22:45:20.118 INFO:tasks.workunit.client.0.smithi063.stdout:7/181: write d0/fa [3311944,88645] 0 2024-04-24T22:45:20.119 INFO:tasks.workunit.client.1.smithi192.stdout:8/140: truncate db/f1d 512021 0 2024-04-24T22:45:20.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/141: write db/dc/f24 [10964,104295] 0 2024-04-24T22:45:20.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/142: readlink db/l16 0 2024-04-24T22:45:20.121 INFO:tasks.workunit.client.0.smithi063.stdout:9/194: dread d6/d8/db/d2b/f34 [0,4194304] 0 2024-04-24T22:45:20.125 INFO:tasks.workunit.client.1.smithi192.stdout:3/163: mkdir d2/d4/d12/d39 0 2024-04-24T22:45:20.127 INFO:tasks.workunit.client.1.smithi192.stdout:3/164: write d2/f13 [929629,40797] 0 2024-04-24T22:45:20.128 INFO:tasks.workunit.client.1.smithi192.stdout:8/143: dwrite db/dc/f24 [0,4194304] 0 2024-04-24T22:45:20.131 INFO:tasks.workunit.client.1.smithi192.stdout:4/106: symlink d2/d9/l2a 0 2024-04-24T22:45:20.132 INFO:tasks.workunit.client.0.smithi063.stdout:4/191: truncate d2/d8/f1d 1734008 0 2024-04-24T22:45:20.133 INFO:tasks.workunit.client.1.smithi192.stdout:3/165: dread d2/d4/f14 [0,4194304] 0 2024-04-24T22:45:20.133 INFO:tasks.workunit.client.0.smithi063.stdout:4/192: fdatasync d2/d8/f1a 0 2024-04-24T22:45:20.133 INFO:tasks.workunit.client.0.smithi063.stdout:4/193: dread - d2/d19/f1f zero size 2024-04-24T22:45:20.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/194: fdatasync d2/d8/f10 0 2024-04-24T22:45:20.135 INFO:tasks.workunit.client.0.smithi063.stdout:3/219: mkdir da/d48 0 2024-04-24T22:45:20.136 INFO:tasks.workunit.client.0.smithi063.stdout:3/220: read da/d39/f41 [936831,71815] 0 2024-04-24T22:45:20.137 INFO:tasks.workunit.client.0.smithi063.stdout:1/198: rename d3/da/dc/d11/c40 to d3/da/dc/d39/c45 0 2024-04-24T22:45:20.139 INFO:tasks.workunit.client.0.smithi063.stdout:7/182: fsync d0/f7 0 2024-04-24T22:45:20.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/145: creat d3/dd/de/d17/f35 x:0 0 0 2024-04-24T22:45:20.145 INFO:tasks.workunit.client.0.smithi063.stdout:0/202: symlink d6/dd/d20/d24/d42/l4b 0 2024-04-24T22:45:20.145 INFO:tasks.workunit.client.0.smithi063.stdout:0/203: readlink d6/dd/d20/l2c 0 2024-04-24T22:45:20.149 INFO:tasks.workunit.client.1.smithi192.stdout:5/165: write d0/d13/d19/f1a [1567065,47007] 0 2024-04-24T22:45:20.154 INFO:tasks.workunit.client.0.smithi063.stdout:4/195: fdatasync d2/f26 0 2024-04-24T22:45:20.154 INFO:tasks.workunit.client.1.smithi192.stdout:8/144: dread db/dc/f18 [0,4194304] 0 2024-04-24T22:45:20.156 INFO:tasks.workunit.client.0.smithi063.stdout:3/221: dread da/d18/f27 [0,4194304] 0 2024-04-24T22:45:20.157 INFO:tasks.workunit.client.1.smithi192.stdout:5/166: dwrite d0/d13/d19/f1d [0,4194304] 0 2024-04-24T22:45:20.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/245: link d4/d29/f34 d4/d9/d37/f4a 0 2024-04-24T22:45:20.160 INFO:tasks.workunit.client.0.smithi063.stdout:1/199: creat d3/da/dc/d11/f46 x:0 0 0 2024-04-24T22:45:20.161 INFO:tasks.workunit.client.0.smithi063.stdout:1/200: fdatasync d3/f23 0 2024-04-24T22:45:20.163 INFO:tasks.workunit.client.1.smithi192.stdout:9/146: mkdir d3/d24/d36 0 2024-04-24T22:45:20.164 INFO:tasks.workunit.client.0.smithi063.stdout:7/183: creat d0/f48 x:0 0 0 2024-04-24T22:45:20.165 INFO:tasks.workunit.client.1.smithi192.stdout:5/167: dwrite d0/f30 [0,4194304] 0 2024-04-24T22:45:20.165 INFO:tasks.workunit.client.1.smithi192.stdout:1/172: getdents d9/d15 0 2024-04-24T22:45:20.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/195: symlink d6/d8/l45 0 2024-04-24T22:45:20.170 INFO:tasks.workunit.client.0.smithi063.stdout:9/196: dread d6/d8/db/d2b/f3d [0,4194304] 0 2024-04-24T22:45:20.170 INFO:tasks.workunit.client.0.smithi063.stdout:0/204: write d6/dd/d20/d21/d25/d31/f35 [129538,120157] 0 2024-04-24T22:45:20.171 INFO:tasks.workunit.client.0.smithi063.stdout:4/196: unlink d2/c21 0 2024-04-24T22:45:20.172 INFO:tasks.workunit.client.0.smithi063.stdout:0/205: chown d6/dd/d20/d24/d42/l4b 24511 1 2024-04-24T22:45:20.174 INFO:tasks.workunit.client.0.smithi063.stdout:0/206: readlink d6/dd/d20/d24/l46 0 2024-04-24T22:45:20.175 INFO:tasks.workunit.client.1.smithi192.stdout:8/145: mknod db/dc/c25 0 2024-04-24T22:45:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:19 smithi192 ceph-mon[33991]: mgrmap e25: smithi192.ctmwwc(active, since 9s), standbys: smithi063.upcnvd 2024-04-24T22:45:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:19 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:45:20.179 INFO:tasks.workunit.client.0.smithi063.stdout:3/222: mknod da/d40/c49 0 2024-04-24T22:45:20.180 INFO:tasks.workunit.client.0.smithi063.stdout:9/197: dwrite d6/d8/d1f/d20/d22/f25 [0,4194304] 0 2024-04-24T22:45:20.181 INFO:tasks.workunit.client.1.smithi192.stdout:3/166: dwrite d2/d4/d12/d1a/d1d/f25 [0,4194304] 0 2024-04-24T22:45:20.182 INFO:tasks.workunit.client.1.smithi192.stdout:6/136: link d0/l11 d0/d8/d18/l2f 0 2024-04-24T22:45:20.183 INFO:tasks.workunit.client.1.smithi192.stdout:8/146: dread db/dc/f24 [0,4194304] 0 2024-04-24T22:45:20.185 INFO:tasks.workunit.client.1.smithi192.stdout:9/147: symlink d3/dd/de/d17/l37 0 2024-04-24T22:45:20.186 INFO:tasks.workunit.client.1.smithi192.stdout:7/163: getdents d2/db/d21 0 2024-04-24T22:45:20.187 INFO:tasks.workunit.client.1.smithi192.stdout:7/164: write f0 [1273816,88159] 0 2024-04-24T22:45:20.188 INFO:tasks.workunit.client.0.smithi063.stdout:1/201: truncate d3/da/dc/d1f/f3e 495401 0 2024-04-24T22:45:20.190 INFO:tasks.workunit.client.0.smithi063.stdout:1/202: dread d3/f17 [0,4194304] 0 2024-04-24T22:45:20.192 INFO:tasks.workunit.client.1.smithi192.stdout:7/165: dread d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:20.194 INFO:tasks.workunit.client.0.smithi063.stdout:8/220: getdents d1/d11 0 2024-04-24T22:45:20.195 INFO:tasks.workunit.client.0.smithi063.stdout:8/221: dread - d1/d11/f47 zero size 2024-04-24T22:45:20.196 INFO:tasks.workunit.client.1.smithi192.stdout:2/246: symlink d4/d9/l4b 0 2024-04-24T22:45:20.198 INFO:tasks.workunit.client.1.smithi192.stdout:3/167: unlink d2/d4/f34 0 2024-04-24T22:45:20.199 INFO:tasks.workunit.client.0.smithi063.stdout:4/197: mkdir d2/d8/d36 0 2024-04-24T22:45:20.201 INFO:tasks.workunit.client.1.smithi192.stdout:6/137: rename d0/d8/fc to d0/d8/d29/f30 0 2024-04-24T22:45:20.203 INFO:tasks.workunit.client.0.smithi063.stdout:5/206: sync 2024-04-24T22:45:20.204 INFO:tasks.workunit.client.1.smithi192.stdout:8/147: mknod db/dc/d22/c26 0 2024-04-24T22:45:20.206 INFO:tasks.workunit.client.1.smithi192.stdout:1/173: getdents d9/d15/d2d/d31/d2f/d35 0 2024-04-24T22:45:20.207 INFO:tasks.workunit.client.1.smithi192.stdout:1/174: stat d9/d15/d2d/c34 0 2024-04-24T22:45:20.211 INFO:tasks.workunit.client.1.smithi192.stdout:7/166: rmdir d2 39 2024-04-24T22:45:20.213 INFO:tasks.workunit.client.1.smithi192.stdout:4/107: write d2/db/d10/f11 [3374240,116850] 0 2024-04-24T22:45:20.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/175: dwrite d9/d15/f1f [0,4194304] 0 2024-04-24T22:45:20.215 INFO:tasks.workunit.client.1.smithi192.stdout:4/108: fdatasync d2/db/d10/f11 0 2024-04-24T22:45:20.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/168: fsync d2/d4/d12/f1c 0 2024-04-24T22:45:20.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/169: chown d2/d27/d28/f36 11229258 1 2024-04-24T22:45:20.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/203: mknod d3/da/dc/d1f/d2b/c47 0 2024-04-24T22:45:20.224 INFO:tasks.workunit.client.0.smithi063.stdout:1/204: fsync d3/da/dc/d11/f46 0 2024-04-24T22:45:20.224 INFO:tasks.workunit.client.1.smithi192.stdout:8/148: symlink db/dc/d22/l27 0 2024-04-24T22:45:20.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/198: dread d2/d8/f1d [0,4194304] 0 2024-04-24T22:45:20.229 INFO:tasks.workunit.client.0.smithi063.stdout:0/207: mknod d6/dd/d20/d21/d25/c4c 0 2024-04-24T22:45:20.229 INFO:tasks.workunit.client.0.smithi063.stdout:8/222: dread d1/d22/f26 [0,4194304] 0 2024-04-24T22:45:20.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/208: chown d6/dd/d20 63 1 2024-04-24T22:45:20.232 INFO:tasks.workunit.client.1.smithi192.stdout:1/176: rename d9/d15/d2d/c34 to d9/d15/c3d 0 2024-04-24T22:45:20.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/223: creat da/d48/f4a x:0 0 0 2024-04-24T22:45:20.234 INFO:tasks.workunit.client.0.smithi063.stdout:8/223: dwrite d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:20.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/167: dwrite d2/f1a [0,4194304] 0 2024-04-24T22:45:20.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/198: creat d6/d8/d24/f46 x:0 0 0 2024-04-24T22:45:20.237 INFO:tasks.workunit.client.1.smithi192.stdout:4/109: chown d2/l6 187415858 1 2024-04-24T22:45:20.238 INFO:tasks.workunit.client.0.smithi063.stdout:9/199: write d6/d8/db/d2b/d36/f40 [877083,15989] 0 2024-04-24T22:45:20.241 INFO:tasks.workunit.client.0.smithi063.stdout:9/200: dread - d6/d8/d1f/d20/f3f zero size 2024-04-24T22:45:20.242 INFO:tasks.workunit.client.0.smithi063.stdout:5/207: link d5/d11/f47 d5/d24/d49/f4f 0 2024-04-24T22:45:20.242 INFO:tasks.workunit.client.1.smithi192.stdout:8/149: creat db/dc/d22/f28 x:0 0 0 2024-04-24T22:45:20.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/224: dwrite da/d18/d3a/f3f [4194304,4194304] 0 2024-04-24T22:45:20.244 INFO:tasks.workunit.client.1.smithi192.stdout:9/148: rmdir d3/d24/d36 0 2024-04-24T22:45:20.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/149: chown d3/dd/de/f2f 6869 1 2024-04-24T22:45:20.245 INFO:tasks.workunit.client.1.smithi192.stdout:4/110: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:20.246 INFO:tasks.workunit.client.0.smithi063.stdout:4/199: creat d2/d19/f37 x:0 0 0 2024-04-24T22:45:20.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/207: sync 2024-04-24T22:45:20.248 INFO:tasks.workunit.client.0.smithi063.stdout:2/189: sync 2024-04-24T22:45:20.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/247: creat d4/d9/f4c x:0 0 0 2024-04-24T22:45:20.249 INFO:tasks.workunit.client.0.smithi063.stdout:8/224: rmdir d1/d2 39 2024-04-24T22:45:20.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/248: dread - d4/f2a zero size 2024-04-24T22:45:20.250 INFO:tasks.workunit.client.1.smithi192.stdout:3/170: mkdir d2/d4/d12/d39/d3a 0 2024-04-24T22:45:20.250 INFO:tasks.workunit.client.0.smithi063.stdout:7/184: link d0/d4/c1d d0/d4/de/c49 0 2024-04-24T22:45:20.251 INFO:tasks.workunit.client.1.smithi192.stdout:6/138: creat d0/d8/f31 x:0 0 0 2024-04-24T22:45:20.252 INFO:tasks.workunit.client.0.smithi063.stdout:1/205: symlink d3/da/dc/d1f/d21/d42/l48 0 2024-04-24T22:45:20.252 INFO:tasks.workunit.client.1.smithi192.stdout:8/150: mkdir db/dc/d29 0 2024-04-24T22:45:20.253 INFO:tasks.workunit.client.1.smithi192.stdout:8/151: write db/dc/f1c [1288636,62018] 0 2024-04-24T22:45:20.254 INFO:tasks.workunit.client.1.smithi192.stdout:9/150: creat d3/dd/f38 x:0 0 0 2024-04-24T22:45:20.255 INFO:tasks.workunit.client.0.smithi063.stdout:1/206: read d3/da/dc/f1c [3634474,16063] 0 2024-04-24T22:45:20.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/208: chown f0 7 1 2024-04-24T22:45:20.257 INFO:tasks.workunit.client.0.smithi063.stdout:9/201: creat d6/d8/d1f/d20/d22/f47 x:0 0 0 2024-04-24T22:45:20.257 INFO:tasks.workunit.client.0.smithi063.stdout:2/190: dwrite d0/d2/f1d [0,4194304] 0 2024-04-24T22:45:20.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/207: fsync d3/da/f3d 0 2024-04-24T22:45:20.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/191: stat d0/d2/d27/f29 0 2024-04-24T22:45:20.259 INFO:tasks.workunit.client.1.smithi192.stdout:6/139: dread - d0/d8/d22/f27 zero size 2024-04-24T22:45:20.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/225: creat da/d39/f4b x:0 0 0 2024-04-24T22:45:20.260 INFO:tasks.workunit.client.1.smithi192.stdout:6/140: read d0/d8/f17 [1215691,45100] 0 2024-04-24T22:45:20.262 INFO:tasks.workunit.client.0.smithi063.stdout:6/208: mknod db/d1d/d20/d3f/c44 0 2024-04-24T22:45:20.265 INFO:tasks.workunit.client.1.smithi192.stdout:4/111: creat d2/d9/d1e/d28/f2b x:0 0 0 2024-04-24T22:45:20.271 INFO:tasks.workunit.client.0.smithi063.stdout:5/209: fsync d5/ff 0 2024-04-24T22:45:20.272 INFO:tasks.workunit.client.1.smithi192.stdout:8/152: symlink db/dc/d29/l2a 0 2024-04-24T22:45:20.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/192: creat d0/d2/d27/f37 x:0 0 0 2024-04-24T22:45:20.275 INFO:tasks.workunit.client.0.smithi063.stdout:9/202: truncate d6/d8/db/d2b/f3d 975659 0 2024-04-24T22:45:20.276 INFO:tasks.workunit.client.0.smithi063.stdout:5/210: dread d5/d11/d18/f41 [0,4194304] 0 2024-04-24T22:45:20.276 INFO:tasks.workunit.client.0.smithi063.stdout:5/211: stat d5/d11/d23/d2d 0 2024-04-24T22:45:20.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/151: fdatasync d3/d2c/f2d 0 2024-04-24T22:45:20.278 INFO:tasks.workunit.client.0.smithi063.stdout:5/212: truncate d5/d24/d44/f46 208320 0 2024-04-24T22:45:20.278 INFO:tasks.workunit.client.1.smithi192.stdout:4/112: readlink d2/db/l13 0 2024-04-24T22:45:20.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/153: dwrite db/dc/f1b [0,4194304] 0 2024-04-24T22:45:20.280 INFO:tasks.workunit.client.0.smithi063.stdout:4/200: symlink d2/d8/d36/l38 0 2024-04-24T22:45:20.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/141: link d0/d8/d22/f27 d0/d8/d22/f32 0 2024-04-24T22:45:20.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/208: dread d3/da/dc/d11/f41 [0,4194304] 0 2024-04-24T22:45:20.294 INFO:tasks.workunit.client.0.smithi063.stdout:0/209: truncate d6/da/dc/f1e 1623846 0 2024-04-24T22:45:20.297 INFO:tasks.workunit.client.1.smithi192.stdout:2/249: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:20.297 INFO:tasks.workunit.client.1.smithi192.stdout:8/154: chown ca 108 1 2024-04-24T22:45:20.298 INFO:tasks.workunit.client.0.smithi063.stdout:0/210: truncate d6/dd/d20/d21/f36 432197 0 2024-04-24T22:45:20.298 INFO:tasks.workunit.client.1.smithi192.stdout:8/155: dread - db/dc/d22/f23 zero size 2024-04-24T22:45:20.298 INFO:tasks.workunit.client.0.smithi063.stdout:1/209: dwrite d3/da/dc/d1f/d2b/f43 [0,4194304] 0 2024-04-24T22:45:20.299 INFO:tasks.workunit.client.0.smithi063.stdout:1/210: chown d3/f3c 14583050 1 2024-04-24T22:45:20.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/177: creat d9/d15/d2d/d31/d2f/f3e x:0 0 0 2024-04-24T22:45:20.301 INFO:tasks.workunit.client.1.smithi192.stdout:3/171: write d2/d4/d7/fd [890593,122503] 0 2024-04-24T22:45:20.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/213: creat d5/d24/d2c/f50 x:0 0 0 2024-04-24T22:45:20.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/201: rename d2/l18 to d2/d8/l39 0 2024-04-24T22:45:20.307 INFO:tasks.workunit.client.0.smithi063.stdout:6/209: unlink f7 0 2024-04-24T22:45:20.311 INFO:tasks.workunit.client.1.smithi192.stdout:8/156: rename db/dc/f1e to db/dc/d29/f2b 0 2024-04-24T22:45:20.313 INFO:tasks.workunit.client.1.smithi192.stdout:8/157: dread db/f1d [0,4194304] 0 2024-04-24T22:45:20.313 INFO:tasks.workunit.client.0.smithi063.stdout:7/185: link d0/d4/de/d17/f22 d0/d4/de/d17/d28/f4a 0 2024-04-24T22:45:20.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/178: chown d9/d15/f2e 198167 1 2024-04-24T22:45:20.321 INFO:tasks.workunit.client.0.smithi063.stdout:0/211: creat d6/dd/f4d x:0 0 0 2024-04-24T22:45:20.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/250: unlink d4/d9/d10/f23 0 2024-04-24T22:45:20.324 INFO:tasks.workunit.client.0.smithi063.stdout:2/193: write d0/d2/d3/d7/f13 [3070463,1117] 0 2024-04-24T22:45:20.325 INFO:tasks.workunit.client.0.smithi063.stdout:1/211: mkdir d3/da/dc/d1f/d21/d49 0 2024-04-24T22:45:20.329 INFO:tasks.workunit.client.0.smithi063.stdout:3/226: creat da/f4c x:0 0 0 2024-04-24T22:45:20.331 INFO:tasks.workunit.client.1.smithi192.stdout:8/158: creat db/dc/d29/f2c x:0 0 0 2024-04-24T22:45:20.331 INFO:tasks.workunit.client.0.smithi063.stdout:5/214: creat d5/d24/f51 x:0 0 0 2024-04-24T22:45:20.332 INFO:tasks.workunit.client.1.smithi192.stdout:4/113: dwrite d2/f1f [0,4194304] 0 2024-04-24T22:45:20.333 INFO:tasks.workunit.client.1.smithi192.stdout:1/179: creat d9/d15/d2d/d31/d2f/f3f x:0 0 0 2024-04-24T22:45:20.334 INFO:tasks.workunit.client.1.smithi192.stdout:1/180: write d9/d15/d2d/d31/f39 [1307623,112066] 0 2024-04-24T22:45:20.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/181: write d9/fa [474811,60910] 0 2024-04-24T22:45:20.336 INFO:tasks.workunit.client.1.smithi192.stdout:6/142: creat d0/d8/f33 x:0 0 0 2024-04-24T22:45:20.337 INFO:tasks.workunit.client.0.smithi063.stdout:6/210: fdatasync f8 0 2024-04-24T22:45:20.339 INFO:tasks.workunit.client.1.smithi192.stdout:2/251: symlink d4/d29/l4d 0 2024-04-24T22:45:20.340 INFO:tasks.workunit.client.0.smithi063.stdout:7/186: symlink d0/d26/d2e/l4b 0 2024-04-24T22:45:20.344 INFO:tasks.workunit.client.0.smithi063.stdout:2/194: rename d0/d2/d3/d1c/l20 to d0/d2/d3/l38 0 2024-04-24T22:45:20.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/114: mknod d2/d9/d1e/d28/c2c 0 2024-04-24T22:45:20.346 INFO:tasks.workunit.client.0.smithi063.stdout:2/195: write d0/d2/d3/d7/f13 [4654966,77104] 0 2024-04-24T22:45:20.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/212: mknod d3/da/dc/d1f/d21/d42/c4a 0 2024-04-24T22:45:20.348 INFO:tasks.workunit.client.0.smithi063.stdout:1/213: write d3/f19 [3607601,15586] 0 2024-04-24T22:45:20.349 INFO:tasks.workunit.client.0.smithi063.stdout:1/214: chown d3/da/dc/d1f/d2b/f43 31 1 2024-04-24T22:45:20.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/172: creat d2/f3b x:0 0 0 2024-04-24T22:45:20.354 INFO:tasks.workunit.client.0.smithi063.stdout:1/215: dwrite d3/d2d/f2e [0,4194304] 0 2024-04-24T22:45:20.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/173: fsync d2/d27/d28/f30 0 2024-04-24T22:45:20.355 INFO:tasks.workunit.client.1.smithi192.stdout:4/115: rename d2/db/d10/f27 to d2/d9/f2d 0 2024-04-24T22:45:20.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/211: creat db/d1d/d20/d3e/f45 x:0 0 0 2024-04-24T22:45:20.363 INFO:tasks.workunit.client.0.smithi063.stdout:7/187: symlink d0/l4c 0 2024-04-24T22:45:20.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/188: truncate d0/d4/fb 4209220 0 2024-04-24T22:45:20.365 INFO:tasks.workunit.client.0.smithi063.stdout:2/196: mkdir d0/d2/d39 0 2024-04-24T22:45:20.365 INFO:tasks.workunit.client.0.smithi063.stdout:2/197: readlink d0/d2/l6 0 2024-04-24T22:45:20.366 INFO:tasks.workunit.client.0.smithi063.stdout:2/198: fsync d0/d2/d3/d7/f13 0 2024-04-24T22:45:20.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/203: getdents d6/d8/d1f/d20/d2a 0 2024-04-24T22:45:20.368 INFO:tasks.workunit.client.0.smithi063.stdout:3/227: mkdir da/d18/d47/d4d 0 2024-04-24T22:45:20.368 INFO:tasks.workunit.client.0.smithi063.stdout:3/228: chown l9 28492598 1 2024-04-24T22:45:20.370 INFO:tasks.workunit.client.0.smithi063.stdout:1/216: symlink d3/da/l4b 0 2024-04-24T22:45:20.370 INFO:tasks.workunit.client.0.smithi063.stdout:1/217: readlink d3/l8 0 2024-04-24T22:45:20.372 INFO:tasks.workunit.client.1.smithi192.stdout:6/143: dwrite d0/d8/fe [0,4194304] 0 2024-04-24T22:45:20.373 INFO:tasks.workunit.client.1.smithi192.stdout:4/116: dread d2/d9/f2d [0,4194304] 0 2024-04-24T22:45:20.374 INFO:tasks.workunit.client.0.smithi063.stdout:9/204: dwrite d6/d8/d1f/d20/d22/f3b [0,4194304] 0 2024-04-24T22:45:20.374 INFO:tasks.workunit.client.1.smithi192.stdout:8/159: getdents db/dc/d22 0 2024-04-24T22:45:20.375 INFO:tasks.workunit.client.1.smithi192.stdout:3/174: unlink d2/d4/d12/d1a/d22/fa 0 2024-04-24T22:45:20.375 INFO:tasks.workunit.client.1.smithi192.stdout:4/117: chown d2/db 43275 1 2024-04-24T22:45:20.375 INFO:tasks.workunit.client.0.smithi063.stdout:9/205: chown d6/d8/d1f/f28 386642 1 2024-04-24T22:45:20.375 INFO:tasks.workunit.client.0.smithi063.stdout:0/212: creat d6/dd/d20/d21/d25/f4e x:0 0 0 2024-04-24T22:45:20.376 INFO:tasks.workunit.client.0.smithi063.stdout:0/213: write d6/dd/f3f [244415,19198] 0 2024-04-24T22:45:20.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/118: write d2/d9/d1e/d28/f2b [165405,41763] 0 2024-04-24T22:45:20.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/160: fdatasync db/f15 0 2024-04-24T22:45:20.383 INFO:tasks.workunit.client.0.smithi063.stdout:2/199: mkdir d0/d2/d27/d3a 0 2024-04-24T22:45:20.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/161: chown db/dc/f13 2030 1 2024-04-24T22:45:20.384 INFO:tasks.workunit.client.1.smithi192.stdout:6/144: dwrite d0/d8/f2a [0,4194304] 0 2024-04-24T22:45:20.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/200: read d0/d2/d3/d1c/f2a [1961493,98284] 0 2024-04-24T22:45:20.390 INFO:tasks.workunit.client.0.smithi063.stdout:4/202: getdents d2/d8 0 2024-04-24T22:45:20.390 INFO:tasks.workunit.client.0.smithi063.stdout:5/215: truncate d5/d24/f3d 2415240 0 2024-04-24T22:45:20.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/229: fdatasync da/f2c 0 2024-04-24T22:45:20.393 INFO:tasks.workunit.client.1.smithi192.stdout:3/175: rename d2/d4/d12/d1a/d1d/c23 to d2/d27/d28/c3c 0 2024-04-24T22:45:20.393 INFO:tasks.workunit.client.1.smithi192.stdout:4/119: mkdir d2/d9/d25/d2e 0 2024-04-24T22:45:20.394 INFO:tasks.workunit.client.0.smithi063.stdout:3/230: write da/f28 [1093578,17067] 0 2024-04-24T22:45:20.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/120: write d2/d9/d25/f23 [1036168,100565] 0 2024-04-24T22:45:20.396 INFO:tasks.workunit.client.1.smithi192.stdout:8/162: fdatasync f6 0 2024-04-24T22:45:20.396 INFO:tasks.workunit.client.0.smithi063.stdout:4/203: dread f1 [0,4194304] 0 2024-04-24T22:45:20.397 INFO:tasks.workunit.client.0.smithi063.stdout:4/204: chown d2/d8/l9 12519973 1 2024-04-24T22:45:20.397 INFO:tasks.workunit.client.0.smithi063.stdout:8/225: link d1/d2/d17/l1a d1/d11/d49/l4c 0 2024-04-24T22:45:20.399 INFO:tasks.workunit.client.1.smithi192.stdout:6/145: read d0/d8/f2b [3229388,4375] 0 2024-04-24T22:45:20.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/218: creat d3/da/dc/d11/f4c x:0 0 0 2024-04-24T22:45:20.401 INFO:tasks.workunit.client.0.smithi063.stdout:1/219: truncate d3/fd 4529039 0 2024-04-24T22:45:20.402 INFO:tasks.workunit.client.1.smithi192.stdout:4/121: mkdir d2/d9/d1e/d28/d2f 0 2024-04-24T22:45:20.403 INFO:tasks.workunit.client.0.smithi063.stdout:0/214: symlink d6/dd/d20/l4f 0 2024-04-24T22:45:20.406 INFO:tasks.workunit.client.0.smithi063.stdout:0/215: stat d6/da/c28 0 2024-04-24T22:45:20.406 INFO:tasks.workunit.client.0.smithi063.stdout:8/226: dwrite d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:20.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/227: stat d1/d11/f29 0 2024-04-24T22:45:20.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/206: dwrite d6/d8/d24/f1e [0,4194304] 0 2024-04-24T22:45:20.410 INFO:tasks.workunit.client.0.smithi063.stdout:9/207: stat d6/d8/d1f/f28 0 2024-04-24T22:45:20.410 INFO:tasks.workunit.client.0.smithi063.stdout:5/216: creat d5/d11/d33/f52 x:0 0 0 2024-04-24T22:45:20.412 INFO:tasks.workunit.client.0.smithi063.stdout:9/208: write d6/d8/d24/f46 [854448,121528] 0 2024-04-24T22:45:20.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/209: write d6/d8/d1f/d20/f3f [156147,52616] 0 2024-04-24T22:45:20.415 INFO:tasks.workunit.client.1.smithi192.stdout:8/163: write f9 [885905,88236] 0 2024-04-24T22:45:20.417 INFO:tasks.workunit.client.0.smithi063.stdout:3/231: symlink da/d40/l4e 0 2024-04-24T22:45:20.418 INFO:tasks.workunit.client.1.smithi192.stdout:6/146: creat d0/d8/f34 x:0 0 0 2024-04-24T22:45:20.420 INFO:tasks.workunit.client.0.smithi063.stdout:4/205: chown d2/d19/c2f 32800397 1 2024-04-24T22:45:20.421 INFO:tasks.workunit.client.0.smithi063.stdout:4/206: fsync d2/f6 0 2024-04-24T22:45:20.422 INFO:tasks.workunit.client.0.smithi063.stdout:9/210: dwrite d6/d8/f17 [0,4194304] 0 2024-04-24T22:45:20.423 INFO:tasks.workunit.client.1.smithi192.stdout:8/164: dread db/f1d [0,4194304] 0 2024-04-24T22:45:20.425 INFO:tasks.workunit.client.1.smithi192.stdout:8/165: truncate db/dc/d29/f2c 953202 0 2024-04-24T22:45:20.426 INFO:tasks.workunit.client.0.smithi063.stdout:2/201: symlink d0/d2/d27/d3a/l3b 0 2024-04-24T22:45:20.426 INFO:tasks.workunit.client.0.smithi063.stdout:3/232: dwrite da/d39/f45 [0,4194304] 0 2024-04-24T22:45:20.427 INFO:tasks.workunit.client.1.smithi192.stdout:6/147: mknod d0/d8/d18/c35 0 2024-04-24T22:45:20.428 INFO:tasks.workunit.client.0.smithi063.stdout:3/233: write da/f4c [1015347,34985] 0 2024-04-24T22:45:20.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/202: truncate d0/d2/f1e 1339688 0 2024-04-24T22:45:20.429 INFO:tasks.workunit.client.1.smithi192.stdout:6/148: write d0/f19 [1099795,9106] 0 2024-04-24T22:45:20.430 INFO:tasks.workunit.client.0.smithi063.stdout:1/220: unlink d3/da/dc/d1f/d21/d42/l48 0 2024-04-24T22:45:20.430 INFO:tasks.workunit.client.1.smithi192.stdout:6/149: write d0/d8/f31 [889005,116384] 0 2024-04-24T22:45:20.431 INFO:tasks.workunit.client.1.smithi192.stdout:4/122: link d2/db/l12 d2/d9/d1e/l30 0 2024-04-24T22:45:20.431 INFO:tasks.workunit.client.1.smithi192.stdout:8/166: dread f9 [0,4194304] 0 2024-04-24T22:45:20.432 INFO:tasks.workunit.client.0.smithi063.stdout:1/221: chown d3/da/dc/d1f/f22 13708570 1 2024-04-24T22:45:20.432 INFO:tasks.workunit.client.1.smithi192.stdout:4/123: chown d2/d9/d1e 1 1 2024-04-24T22:45:20.432 INFO:tasks.workunit.client.1.smithi192.stdout:8/167: chown db/dc/f17 14 1 2024-04-24T22:45:20.435 INFO:tasks.workunit.client.1.smithi192.stdout:3/176: link d2/d27/d28/c3c d2/d27/c3d 0 2024-04-24T22:45:20.435 INFO:tasks.workunit.client.1.smithi192.stdout:3/177: chown d2 2747 1 2024-04-24T22:45:20.438 INFO:tasks.workunit.client.0.smithi063.stdout:2/203: dread d0/d2/f1d [0,4194304] 0 2024-04-24T22:45:20.441 INFO:tasks.workunit.client.1.smithi192.stdout:8/168: dwrite f6 [4194304,4194304] 0 2024-04-24T22:45:20.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/222: dwrite d3/f23 [0,4194304] 0 2024-04-24T22:45:20.443 INFO:tasks.workunit.client.0.smithi063.stdout:6/212: getdents db/d1c 0 2024-04-24T22:45:20.445 INFO:tasks.workunit.client.0.smithi063.stdout:0/216: dwrite d6/da/dc/f17 [0,4194304] 0 2024-04-24T22:45:20.447 INFO:tasks.workunit.client.0.smithi063.stdout:6/213: dread db/f1a [4194304,4194304] 0 2024-04-24T22:45:20.448 INFO:tasks.workunit.client.0.smithi063.stdout:6/214: fsync db/ff 0 2024-04-24T22:45:20.450 INFO:tasks.workunit.client.1.smithi192.stdout:8/169: dwrite db/dc/f1b [4194304,4194304] 0 2024-04-24T22:45:20.452 INFO:tasks.workunit.client.0.smithi063.stdout:6/215: dread db/d1d/d20/f2b [0,4194304] 0 2024-04-24T22:45:20.452 INFO:tasks.workunit.client.1.smithi192.stdout:6/150: mknod d0/d8/d18/d2e/c36 0 2024-04-24T22:45:20.453 INFO:tasks.workunit.client.0.smithi063.stdout:7/189: getdents d0/d4/de 0 2024-04-24T22:45:20.454 INFO:tasks.workunit.client.0.smithi063.stdout:6/216: fdatasync db/d1d/d20/f38 0 2024-04-24T22:45:20.455 INFO:tasks.workunit.client.0.smithi063.stdout:7/190: truncate d0/f3d 600351 0 2024-04-24T22:45:20.455 INFO:tasks.workunit.client.0.smithi063.stdout:7/191: dread - d0/d26/f45 zero size 2024-04-24T22:45:20.457 INFO:tasks.workunit.client.0.smithi063.stdout:4/207: mkdir d2/d8/d36/d3a 0 2024-04-24T22:45:20.459 INFO:tasks.workunit.client.1.smithi192.stdout:4/124: mkdir d2/d9/d25/d2e/d31 0 2024-04-24T22:45:20.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/125: write d2/f17 [171841,82801] 0 2024-04-24T22:45:20.462 INFO:tasks.workunit.client.1.smithi192.stdout:4/126: chown d2/f17 8456993 1 2024-04-24T22:45:20.463 INFO:tasks.workunit.client.1.smithi192.stdout:4/127: truncate d2/d9/f2d 876470 0 2024-04-24T22:45:20.465 INFO:tasks.workunit.client.0.smithi063.stdout:9/211: mkdir d6/d8/db/d2b/d48 0 2024-04-24T22:45:20.466 INFO:tasks.workunit.client.0.smithi063.stdout:9/212: read - d6/d8/d1f/d20/d22/f47 zero size 2024-04-24T22:45:20.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/234: rename da/d18/c34 to da/d48/c4f 0 2024-04-24T22:45:20.470 INFO:tasks.workunit.client.1.smithi192.stdout:3/178: dwrite d2/d4/d7/f2c [0,4194304] 0 2024-04-24T22:45:20.470 INFO:tasks.workunit.client.1.smithi192.stdout:4/128: dread d2/d9/d25/f23 [0,4194304] 0 2024-04-24T22:45:20.471 INFO:tasks.workunit.client.1.smithi192.stdout:6/151: symlink d0/d8/d18/d2e/l37 0 2024-04-24T22:45:20.471 INFO:tasks.workunit.client.1.smithi192.stdout:3/179: write d2/f13 [1432130,121520] 0 2024-04-24T22:45:20.472 INFO:tasks.workunit.client.1.smithi192.stdout:6/152: readlink d0/d8/l13 0 2024-04-24T22:45:20.473 INFO:tasks.workunit.client.0.smithi063.stdout:9/213: dwrite d6/d8/db/d2b/d36/f40 [0,4194304] 0 2024-04-24T22:45:20.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/153: truncate d0/d8/d22/f27 41119 0 2024-04-24T22:45:20.477 INFO:tasks.workunit.client.0.smithi063.stdout:8/228: fdatasync d1/d22/f26 0 2024-04-24T22:45:20.482 INFO:tasks.workunit.client.1.smithi192.stdout:4/129: creat d2/d9/d25/f32 x:0 0 0 2024-04-24T22:45:20.485 INFO:tasks.workunit.client.1.smithi192.stdout:3/180: fsync d2/d4/d7/f9 0 2024-04-24T22:45:20.489 INFO:tasks.workunit.client.1.smithi192.stdout:6/154: symlink d0/d8/d18/l38 0 2024-04-24T22:45:20.490 INFO:tasks.workunit.client.1.smithi192.stdout:6/155: write d0/d8/f31 [86591,86698] 0 2024-04-24T22:45:20.490 INFO:tasks.workunit.client.0.smithi063.stdout:6/217: symlink db/d1d/d2d/l46 0 2024-04-24T22:45:20.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/192: symlink d0/d4/de/d17/d28/l4d 0 2024-04-24T22:45:20.494 INFO:tasks.workunit.client.1.smithi192.stdout:3/181: unlink d2/d4/d12/d1a/d22/f2a 0 2024-04-24T22:45:20.495 INFO:tasks.workunit.client.0.smithi063.stdout:7/193: dread d0/d4/de/f10 [0,4194304] 0 2024-04-24T22:45:20.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/130: dwrite d2/f3 [0,4194304] 0 2024-04-24T22:45:20.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/131: readlink d2/d9/l1c 0 2024-04-24T22:45:20.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/194: readlink d0/d4/de/d17/l1f 0 2024-04-24T22:45:20.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/223: rename d3/da/dc/d11/f4c to d3/da/dc/d1f/f4d 0 2024-04-24T22:45:20.498 INFO:tasks.workunit.client.1.smithi192.stdout:4/132: dread d2/d9/f2d [0,4194304] 0 2024-04-24T22:45:20.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/235: symlink da/d18/d47/l50 0 2024-04-24T22:45:20.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/217: write d5/d24/d49/d3b/f48 [469143,39128] 0 2024-04-24T22:45:20.501 INFO:tasks.workunit.client.1.smithi192.stdout:3/182: creat d2/d4/f3e x:0 0 0 2024-04-24T22:45:20.502 INFO:tasks.workunit.client.1.smithi192.stdout:8/170: dwrite f3 [4194304,4194304] 0 2024-04-24T22:45:20.503 INFO:tasks.workunit.client.0.smithi063.stdout:7/195: dwrite d0/d4/de/f1a [4194304,4194304] 0 2024-04-24T22:45:20.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/156: symlink d0/d8/l39 0 2024-04-24T22:45:20.586 INFO:tasks.workunit.client.1.smithi192.stdout:4/133: symlink d2/db/d10/l33 0 2024-04-24T22:45:20.586 INFO:tasks.workunit.client.0.smithi063.stdout:2/204: mknod d0/d2/d39/c3c 0 2024-04-24T22:45:20.587 INFO:tasks.workunit.client.1.smithi192.stdout:8/171: mknod db/dc/c2d 0 2024-04-24T22:45:20.587 INFO:tasks.workunit.client.1.smithi192.stdout:8/172: chown f9 6738 1 2024-04-24T22:45:20.588 INFO:tasks.workunit.client.1.smithi192.stdout:4/134: stat d2/d9/d1e/l30 0 2024-04-24T22:45:20.589 INFO:tasks.workunit.client.1.smithi192.stdout:8/173: dread db/dc/f18 [0,4194304] 0 2024-04-24T22:45:20.590 INFO:tasks.workunit.client.1.smithi192.stdout:8/174: write f5 [2625900,46563] 0 2024-04-24T22:45:20.591 INFO:tasks.workunit.client.0.smithi063.stdout:4/208: mknod d2/d19/d33/c3b 0 2024-04-24T22:45:20.592 INFO:tasks.workunit.client.0.smithi063.stdout:2/205: dwrite d0/d2/d3/d7/f19 [0,4194304] 0 2024-04-24T22:45:20.594 INFO:tasks.workunit.client.0.smithi063.stdout:1/224: symlink d3/da/dc/d1f/l4e 0 2024-04-24T22:45:20.596 INFO:tasks.workunit.client.0.smithi063.stdout:1/225: stat d3/da/dc/d1f/f22 0 2024-04-24T22:45:20.597 INFO:tasks.workunit.client.0.smithi063.stdout:3/236: symlink da/d18/d47/l51 0 2024-04-24T22:45:20.598 INFO:tasks.workunit.client.1.smithi192.stdout:8/175: symlink db/dc/d22/l2e 0 2024-04-24T22:45:20.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/218: mknod d5/d11/d18/c53 0 2024-04-24T22:45:20.604 INFO:tasks.workunit.client.1.smithi192.stdout:8/176: symlink db/dc/l2f 0 2024-04-24T22:45:20.606 INFO:tasks.workunit.client.0.smithi063.stdout:7/196: mknod d0/d4/c4e 0 2024-04-24T22:45:20.608 INFO:tasks.workunit.client.0.smithi063.stdout:1/226: dread d3/f7 [0,4194304] 0 2024-04-24T22:45:20.608 INFO:tasks.workunit.client.1.smithi192.stdout:4/135: rmdir d2/db/d20 0 2024-04-24T22:45:20.609 INFO:tasks.workunit.client.0.smithi063.stdout:8/229: mknod d1/c4d 0 2024-04-24T22:45:20.610 INFO:tasks.workunit.client.1.smithi192.stdout:8/177: mknod db/dc/d22/c30 0 2024-04-24T22:45:20.611 INFO:tasks.workunit.client.1.smithi192.stdout:8/178: read db/dc/f13 [2703212,73128] 0 2024-04-24T22:45:20.614 INFO:tasks.workunit.client.0.smithi063.stdout:0/217: link d6/dd/d20/l4f d6/dd/d20/d21/d25/d31/l50 0 2024-04-24T22:45:20.616 INFO:tasks.workunit.client.0.smithi063.stdout:8/230: dwrite d1/d2/d17/f3d [0,4194304] 0 2024-04-24T22:45:20.618 INFO:tasks.workunit.client.0.smithi063.stdout:4/209: mkdir d2/d8/d3c 0 2024-04-24T22:45:20.619 INFO:tasks.workunit.client.1.smithi192.stdout:8/179: mkdir db/dc/d22/d31 0 2024-04-24T22:45:20.620 INFO:tasks.workunit.client.0.smithi063.stdout:6/218: truncate db/d1d/d20/f2b 1745924 0 2024-04-24T22:45:20.620 INFO:tasks.workunit.client.0.smithi063.stdout:6/219: chown l9 102 1 2024-04-24T22:45:20.621 INFO:tasks.workunit.client.0.smithi063.stdout:6/220: fsync db/f10 0 2024-04-24T22:45:20.622 INFO:tasks.workunit.client.0.smithi063.stdout:2/206: mknod d0/d2/d27/d3a/c3d 0 2024-04-24T22:45:20.623 INFO:tasks.workunit.client.0.smithi063.stdout:2/207: truncate d0/f33 608310 0 2024-04-24T22:45:20.623 INFO:tasks.workunit.client.0.smithi063.stdout:2/208: readlink d0/l2c 0 2024-04-24T22:45:20.623 INFO:tasks.workunit.client.1.smithi192.stdout:4/136: creat d2/d9/d1e/d28/d2f/f34 x:0 0 0 2024-04-24T22:45:20.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/209: write d0/f36 [44629,93018] 0 2024-04-24T22:45:20.624 INFO:tasks.workunit.client.1.smithi192.stdout:6/157: truncate d0/d8/d29/f30 5697465 0 2024-04-24T22:45:20.626 INFO:tasks.workunit.client.0.smithi063.stdout:3/237: dread - da/f1d zero size 2024-04-24T22:45:20.626 INFO:tasks.workunit.client.0.smithi063.stdout:5/219: rename d5 to d5/d54 22 2024-04-24T22:45:20.628 INFO:tasks.workunit.client.0.smithi063.stdout:7/197: creat d0/f4f x:0 0 0 2024-04-24T22:45:20.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/137: fdatasync d2/d9/d25/f23 0 2024-04-24T22:45:20.631 INFO:tasks.workunit.client.0.smithi063.stdout:7/198: fdatasync d0/d4/de/f25 0 2024-04-24T22:45:20.631 INFO:tasks.workunit.client.0.smithi063.stdout:9/214: truncate d6/d8/db/d2b/f34 1455714 0 2024-04-24T22:45:20.632 INFO:tasks.workunit.client.1.smithi192.stdout:8/180: symlink db/l32 0 2024-04-24T22:45:20.633 INFO:tasks.workunit.client.0.smithi063.stdout:2/210: dwrite d0/d2/d3/d7/f13 [0,4194304] 0 2024-04-24T22:45:20.633 INFO:tasks.workunit.client.1.smithi192.stdout:8/181: read f8 [1189139,103904] 0 2024-04-24T22:45:20.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/199: chown d0/d2f/c30 621850928 1 2024-04-24T22:45:20.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/210: symlink d2/d8/l3d 0 2024-04-24T22:45:20.636 INFO:tasks.workunit.client.1.smithi192.stdout:4/138: rmdir d2/d9 39 2024-04-24T22:45:20.638 INFO:tasks.workunit.client.1.smithi192.stdout:6/158: mknod d0/d8/c3a 0 2024-04-24T22:45:20.639 INFO:tasks.workunit.client.0.smithi063.stdout:6/221: mkdir db/d1d/d20/d3e/d47 0 2024-04-24T22:45:20.641 INFO:tasks.workunit.client.0.smithi063.stdout:7/200: dwrite d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:20.646 INFO:tasks.workunit.client.0.smithi063.stdout:1/227: rename d3/da/dc/f1c to d3/da/dc/d1f/d21/d49/f4f 0 2024-04-24T22:45:20.647 INFO:tasks.workunit.client.1.smithi192.stdout:6/159: creat d0/d8/d29/f3b x:0 0 0 2024-04-24T22:45:20.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/139: dwrite d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:20.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/228: dread d3/da/f37 [0,4194304] 0 2024-04-24T22:45:20.657 INFO:tasks.workunit.client.1.smithi192.stdout:6/160: symlink d0/d8/d29/l3c 0 2024-04-24T22:45:20.658 INFO:tasks.workunit.client.0.smithi063.stdout:9/215: stat d6/d8/db/d2b/f3d 0 2024-04-24T22:45:20.659 INFO:tasks.workunit.client.0.smithi063.stdout:9/216: chown d6/d8/l45 234544978 1 2024-04-24T22:45:20.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/140: write d2/d9/f2d [884996,39606] 0 2024-04-24T22:45:20.663 INFO:tasks.workunit.client.0.smithi063.stdout:8/231: write d1/f5 [3063584,26123] 0 2024-04-24T22:45:20.663 INFO:tasks.workunit.client.1.smithi192.stdout:4/141: dread d2/f1f [0,4194304] 0 2024-04-24T22:45:20.666 INFO:tasks.workunit.client.1.smithi192.stdout:6/161: mkdir d0/d8/d3d 0 2024-04-24T22:45:20.667 INFO:tasks.workunit.client.0.smithi063.stdout:0/218: dwrite f5 [0,4194304] 0 2024-04-24T22:45:20.668 INFO:tasks.workunit.client.0.smithi063.stdout:8/232: read d1/d11/d32/d38/f3f [3446100,69453] 0 2024-04-24T22:45:20.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/233: stat d1/d11/d49/c4a 0 2024-04-24T22:45:20.670 INFO:tasks.workunit.client.0.smithi063.stdout:4/211: creat d2/d8/f3e x:0 0 0 2024-04-24T22:45:20.672 INFO:tasks.workunit.client.1.smithi192.stdout:4/142: dwrite d2/d9/d1e/d28/d2f/f34 [0,4194304] 0 2024-04-24T22:45:20.674 INFO:tasks.workunit.client.1.smithi192.stdout:4/143: read d2/f1f [3972550,47459] 0 2024-04-24T22:45:20.676 INFO:tasks.workunit.client.1.smithi192.stdout:6/162: dwrite d0/d8/f2a [0,4194304] 0 2024-04-24T22:45:20.676 INFO:tasks.workunit.client.0.smithi063.stdout:7/201: dread d0/fa [4194304,4194304] 0 2024-04-24T22:45:20.683 INFO:tasks.workunit.client.1.smithi192.stdout:4/144: symlink d2/db/d10/l35 0 2024-04-24T22:45:20.688 INFO:tasks.workunit.client.0.smithi063.stdout:1/229: truncate d3/f17 1461964 0 2024-04-24T22:45:20.689 INFO:tasks.workunit.client.0.smithi063.stdout:9/217: creat d6/d8/db/d2b/f49 x:0 0 0 2024-04-24T22:45:20.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/163: mkdir d0/d8/d3d/d3e 0 2024-04-24T22:45:20.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/145: mkdir d2/d9/d25/d2e/d31/d36 0 2024-04-24T22:45:20.693 INFO:tasks.workunit.client.0.smithi063.stdout:6/222: write f8 [140833,121245] 0 2024-04-24T22:45:20.695 INFO:tasks.workunit.client.0.smithi063.stdout:6/223: truncate db/d1d/d20/d3e/f43 638890 0 2024-04-24T22:45:20.695 INFO:tasks.workunit.client.0.smithi063.stdout:0/219: creat d6/da/dc/d26/d33/f51 x:0 0 0 2024-04-24T22:45:20.698 INFO:tasks.workunit.client.0.smithi063.stdout:4/212: unlink d2/l1b 0 2024-04-24T22:45:20.700 INFO:tasks.workunit.client.1.smithi192.stdout:4/146: dwrite d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:20.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/211: link d0/d2/d27/d3a/l3b d0/d2/d2d/l3e 0 2024-04-24T22:45:20.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/147: creat d2/d9/d1e/f37 x:0 0 0 2024-04-24T22:45:20.703 INFO:tasks.workunit.client.0.smithi063.stdout:8/234: symlink d1/l4e 0 2024-04-24T22:45:20.704 INFO:tasks.workunit.client.1.smithi192.stdout:4/148: write d2/d9/d25/f32 [2701985,21012] 0 2024-04-24T22:45:20.707 INFO:tasks.workunit.client.0.smithi063.stdout:5/220: truncate d5/f14 2735980 0 2024-04-24T22:45:20.707 INFO:tasks.workunit.client.0.smithi063.stdout:9/218: symlink d6/l4a 0 2024-04-24T22:45:20.709 INFO:tasks.workunit.client.0.smithi063.stdout:9/219: chown d6/d8/l29 20409 1 2024-04-24T22:45:20.710 INFO:tasks.workunit.client.0.smithi063.stdout:9/220: fsync d6/f7 0 2024-04-24T22:45:20.711 INFO:tasks.workunit.client.0.smithi063.stdout:2/212: mkdir d0/d2/d3/d1c/d3f 0 2024-04-24T22:45:20.713 INFO:tasks.workunit.client.0.smithi063.stdout:3/238: rename da/d48/c4f to da/c52 0 2024-04-24T22:45:20.714 INFO:tasks.workunit.client.1.smithi192.stdout:6/164: write d0/d8/f14 [4647602,109410] 0 2024-04-24T22:45:20.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/149: dwrite d2/f1f [0,4194304] 0 2024-04-24T22:45:20.715 INFO:tasks.workunit.client.0.smithi063.stdout:1/230: link d3/da/dc/l10 d3/da/d34/l50 0 2024-04-24T22:45:20.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/165: dread - d0/d8/f33 zero size 2024-04-24T22:45:20.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/243: sync 2024-04-24T22:45:20.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/150: chown d2/db/l1b 0 1 2024-04-24T22:45:20.723 INFO:tasks.workunit.client.1.smithi192.stdout:6/166: dwrite d0/d8/f34 [0,4194304] 0 2024-04-24T22:45:20.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/221: dwrite d6/d8/db/d2b/d36/f40 [0,4194304] 0 2024-04-24T22:45:20.724 INFO:tasks.workunit.client.1.smithi192.stdout:4/151: fdatasync d2/d9/d25/f23 0 2024-04-24T22:45:20.724 INFO:tasks.workunit.client.0.smithi063.stdout:2/213: dwrite d0/d2/d3/f25 [0,4194304] 0 2024-04-24T22:45:20.726 INFO:tasks.workunit.client.0.smithi063.stdout:4/213: dread d2/fb [0,4194304] 0 2024-04-24T22:45:20.727 INFO:tasks.workunit.client.0.smithi063.stdout:3/239: dread - da/f15 zero size 2024-04-24T22:45:20.729 INFO:tasks.workunit.client.0.smithi063.stdout:0/220: truncate d6/dd/f3f 13927 0 2024-04-24T22:45:20.731 INFO:tasks.workunit.client.0.smithi063.stdout:8/235: creat d1/d21/f4f x:0 0 0 2024-04-24T22:45:20.733 INFO:tasks.workunit.client.0.smithi063.stdout:2/214: write d0/d2/d27/f34 [953749,113388] 0 2024-04-24T22:45:20.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/152: mknod d2/d9/d25/d2e/c38 0 2024-04-24T22:45:20.734 INFO:tasks.workunit.client.0.smithi063.stdout:0/221: write d6/f7 [8777207,103702] 0 2024-04-24T22:45:20.735 INFO:tasks.workunit.client.1.smithi192.stdout:0/244: creat d3/f55 x:0 0 0 2024-04-24T22:45:20.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/236: chown d1/d21/f4f 4 1 2024-04-24T22:45:20.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/231: dwrite f1 [0,4194304] 0 2024-04-24T22:45:20.737 INFO:tasks.workunit.client.1.smithi192.stdout:0/245: creat d3/d27/d28/f56 x:0 0 0 2024-04-24T22:45:20.738 INFO:tasks.workunit.client.0.smithi063.stdout:1/232: stat d3/da/dc/d1f/f22 0 2024-04-24T22:45:20.738 INFO:tasks.workunit.client.1.smithi192.stdout:0/246: write d3/d27/d28/d2e/d4a/f4d [3808300,64053] 0 2024-04-24T22:45:20.739 INFO:tasks.workunit.client.0.smithi063.stdout:4/214: fsync d2/f1c 0 2024-04-24T22:45:20.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/222: mkdir d6/d8/d1f/d20/d2a/d4b 0 2024-04-24T22:45:20.739 INFO:tasks.workunit.client.1.smithi192.stdout:0/247: chown d3/d27/d28/f19 507594 1 2024-04-24T22:45:20.744 INFO:tasks.workunit.client.1.smithi192.stdout:0/248: mknod d3/d27/d28/d2e/d3f/c57 0 2024-04-24T22:45:20.745 INFO:tasks.workunit.client.1.smithi192.stdout:0/249: fsync d3/d27/d3c/f4e 0 2024-04-24T22:45:20.745 INFO:tasks.workunit.client.0.smithi063.stdout:2/215: rename d0/f36 to d0/f40 0 2024-04-24T22:45:20.747 INFO:tasks.workunit.client.0.smithi063.stdout:8/237: mkdir d1/d11/d32/d38/d50 0 2024-04-24T22:45:20.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/223: chown d6/d8/l45 2046 1 2024-04-24T22:45:20.748 INFO:tasks.workunit.client.0.smithi063.stdout:4/215: mknod d2/d19/d33/c3f 0 2024-04-24T22:45:20.749 INFO:tasks.workunit.client.0.smithi063.stdout:1/233: symlink d3/da/dc/d1f/d21/d42/l51 0 2024-04-24T22:45:20.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/216: dread - d2/f35 zero size 2024-04-24T22:45:20.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/222: dwrite d6/da/f29 [0,4194304] 0 2024-04-24T22:45:20.752 INFO:tasks.workunit.client.0.smithi063.stdout:2/216: symlink d0/d2/d35/l41 0 2024-04-24T22:45:20.752 INFO:tasks.workunit.client.0.smithi063.stdout:8/238: dread - d1/d11/d32/d38/f42 zero size 2024-04-24T22:45:20.753 INFO:tasks.workunit.client.0.smithi063.stdout:1/234: chown d3/da/dc/c24 14227 1 2024-04-24T22:45:20.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/217: mkdir d2/d8/d40 0 2024-04-24T22:45:20.755 INFO:tasks.workunit.client.0.smithi063.stdout:2/217: rmdir d0/d2 39 2024-04-24T22:45:20.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/223: truncate d6/dd/d20/d21/f48 279400 0 2024-04-24T22:45:20.758 INFO:tasks.workunit.client.1.smithi192.stdout:0/250: dread d3/d27/d28/f1d [0,4194304] 0 2024-04-24T22:45:20.759 INFO:tasks.workunit.client.0.smithi063.stdout:9/224: dwrite d6/d8/d24/f1e [4194304,4194304] 0 2024-04-24T22:45:20.762 INFO:tasks.workunit.client.0.smithi063.stdout:1/235: rmdir d3/da/dc/d1f/d21/d49 39 2024-04-24T22:45:20.763 INFO:tasks.workunit.client.1.smithi192.stdout:0/251: rmdir d3/d27/d3c 39 2024-04-24T22:45:20.771 INFO:tasks.workunit.client.1.smithi192.stdout:6/167: write d0/d8/f2b [4394339,86513] 0 2024-04-24T22:45:20.772 INFO:tasks.workunit.client.0.smithi063.stdout:1/236: creat d3/d2d/f52 x:0 0 0 2024-04-24T22:45:20.772 INFO:tasks.workunit.client.0.smithi063.stdout:0/224: symlink d6/l52 0 2024-04-24T22:45:20.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/237: readlink d3/l8 0 2024-04-24T22:45:20.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/218: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:20.773 INFO:tasks.workunit.client.0.smithi063.stdout:2/218: creat d0/d2/d3/d1c/d3f/f42 x:0 0 0 2024-04-24T22:45:20.774 INFO:tasks.workunit.client.0.smithi063.stdout:8/239: dwrite d1/d11/d32/d38/f42 [0,4194304] 0 2024-04-24T22:45:20.776 INFO:tasks.workunit.client.1.smithi192.stdout:6/168: creat d0/d8/d18/f3f x:0 0 0 2024-04-24T22:45:20.776 INFO:tasks.workunit.client.0.smithi063.stdout:5/221: dwrite d5/d24/f3d [0,4194304] 0 2024-04-24T22:45:20.776 INFO:tasks.workunit.client.0.smithi063.stdout:2/219: chown d0/d2/d39 2545 1 2024-04-24T22:45:20.777 INFO:tasks.workunit.client.1.smithi192.stdout:6/169: stat d0/d8/f2a 0 2024-04-24T22:45:20.777 INFO:tasks.workunit.client.0.smithi063.stdout:4/219: stat d2/d19/d33/c3f 0 2024-04-24T22:45:20.780 INFO:tasks.workunit.client.0.smithi063.stdout:3/240: write da/f2c [1270974,75305] 0 2024-04-24T22:45:20.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/238: readlink d3/da/dc/d1f/d21/d42/l51 0 2024-04-24T22:45:20.783 INFO:tasks.workunit.client.1.smithi192.stdout:0/252: link d3/d27/d3c/d3b/c44 d3/d27/d28/d2e/d3f/c58 0 2024-04-24T22:45:20.785 INFO:tasks.workunit.client.1.smithi192.stdout:6/170: mkdir d0/d8/d3d/d3e/d40 0 2024-04-24T22:45:20.787 INFO:tasks.workunit.client.1.smithi192.stdout:6/171: dread d0/d8/d22/f27 [0,4194304] 0 2024-04-24T22:45:20.787 INFO:tasks.workunit.client.1.smithi192.stdout:6/172: stat d0/c5 0 2024-04-24T22:45:20.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/240: dread d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:20.788 INFO:tasks.workunit.client.1.smithi192.stdout:6/173: chown d0/d8/d22/c23 7162 1 2024-04-24T22:45:20.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/220: dwrite d0/d2/d27/f29 [0,4194304] 0 2024-04-24T22:45:20.789 INFO:tasks.workunit.client.1.smithi192.stdout:6/174: chown d0/d8/c2c 22565203 1 2024-04-24T22:45:20.790 INFO:tasks.workunit.client.0.smithi063.stdout:9/225: link d6/d8/l2e d6/d8/d1f/d20/d22/l4c 0 2024-04-24T22:45:20.790 INFO:tasks.workunit.client.0.smithi063.stdout:2/221: write d0/d2/d3/d7/f13 [2030887,82500] 0 2024-04-24T22:45:20.794 INFO:tasks.workunit.client.0.smithi063.stdout:8/241: dread d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:20.795 INFO:tasks.workunit.client.0.smithi063.stdout:3/241: rename da/d39/f45 to da/d40/f53 0 2024-04-24T22:45:20.797 INFO:tasks.workunit.client.0.smithi063.stdout:3/242: truncate da/d40/f53 4740329 0 2024-04-24T22:45:20.797 INFO:tasks.workunit.client.0.smithi063.stdout:5/222: unlink d5/d24/d49/d3b/f42 0 2024-04-24T22:45:20.797 INFO:tasks.workunit.client.0.smithi063.stdout:3/243: truncate da/d48/f4a 1029339 0 2024-04-24T22:45:20.799 INFO:tasks.workunit.client.1.smithi192.stdout:6/175: creat d0/d8/d18/d2e/f41 x:0 0 0 2024-04-24T22:45:20.801 INFO:tasks.workunit.client.1.smithi192.stdout:0/253: creat d3/d25/f59 x:0 0 0 2024-04-24T22:45:20.802 INFO:tasks.workunit.client.1.smithi192.stdout:0/254: fsync d3/d27/d28/d31/f3a 0 2024-04-24T22:45:20.803 INFO:tasks.workunit.client.1.smithi192.stdout:6/176: mknod d0/d8/d3d/d3e/d40/c42 0 2024-04-24T22:45:20.804 INFO:tasks.workunit.client.0.smithi063.stdout:5/223: dwrite d5/d24/d44/f46 [0,4194304] 0 2024-04-24T22:45:20.805 INFO:tasks.workunit.client.0.smithi063.stdout:5/224: fdatasync d5/d24/d49/d3b/f45 0 2024-04-24T22:45:20.805 INFO:tasks.workunit.client.0.smithi063.stdout:2/222: creat d0/d2/d35/f43 x:0 0 0 2024-04-24T22:45:20.805 INFO:tasks.workunit.client.0.smithi063.stdout:5/225: dread - d5/d11/f47 zero size 2024-04-24T22:45:20.808 INFO:tasks.workunit.client.1.smithi192.stdout:0/255: dwrite d3/d25/f3d [0,4194304] 0 2024-04-24T22:45:20.809 INFO:tasks.workunit.client.1.smithi192.stdout:0/256: write d3/f10 [3135,53434] 0 2024-04-24T22:45:20.810 INFO:tasks.workunit.client.1.smithi192.stdout:0/257: dread - d3/d25/f59 zero size 2024-04-24T22:45:20.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/242: rename c0 to d1/d11/d49/c51 0 2024-04-24T22:45:20.813 INFO:tasks.workunit.client.1.smithi192.stdout:6/177: link d0/d8/c2c d0/d8/d18/d2e/c43 0 2024-04-24T22:45:20.814 INFO:tasks.workunit.client.0.smithi063.stdout:3/244: fsync da/d18/f27 0 2024-04-24T22:45:20.816 INFO:tasks.workunit.client.0.smithi063.stdout:3/245: write da/d18/d3a/f3f [4942684,70802] 0 2024-04-24T22:45:20.820 INFO:tasks.workunit.client.0.smithi063.stdout:4/220: write d2/d8/f1d [80496,65057] 0 2024-04-24T22:45:20.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/178: mkdir d0/d8/d3d/d3e/d40/d44 0 2024-04-24T22:45:20.824 INFO:tasks.workunit.client.0.smithi063.stdout:3/246: read f7 [923935,52710] 0 2024-04-24T22:45:20.825 INFO:tasks.workunit.client.1.smithi192.stdout:0/258: creat d3/f5a x:0 0 0 2024-04-24T22:45:20.828 INFO:tasks.workunit.client.0.smithi063.stdout:2/223: symlink d0/d2/d39/l44 0 2024-04-24T22:45:20.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/226: creat d5/d24/d2c/f55 x:0 0 0 2024-04-24T22:45:20.831 INFO:tasks.workunit.client.0.smithi063.stdout:3/247: dwrite da/f28 [4194304,4194304] 0 2024-04-24T22:45:20.837 INFO:tasks.workunit.client.0.smithi063.stdout:2/224: symlink d0/d2/d39/l45 0 2024-04-24T22:45:20.838 INFO:tasks.workunit.client.0.smithi063.stdout:2/225: fdatasync d0/d2/d3/d7/f13 0 2024-04-24T22:45:20.839 INFO:tasks.workunit.client.0.smithi063.stdout:2/226: write d0/d2/d3/d1c/d3f/f42 [767802,67989] 0 2024-04-24T22:45:20.840 INFO:tasks.workunit.client.0.smithi063.stdout:5/227: symlink d5/d1d/l56 0 2024-04-24T22:45:20.840 INFO:tasks.workunit.client.0.smithi063.stdout:5/228: chown d5/d11/f15 59 1 2024-04-24T22:45:20.841 INFO:tasks.workunit.client.0.smithi063.stdout:5/229: chown d5/d11/d23 47 1 2024-04-24T22:45:20.841 INFO:tasks.workunit.client.0.smithi063.stdout:2/227: truncate d0/d2/d27/f37 1023539 0 2024-04-24T22:45:20.842 INFO:tasks.workunit.client.0.smithi063.stdout:5/230: fdatasync d5/d24/d2c/f55 0 2024-04-24T22:45:20.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/221: mknod d2/d8/d3c/c41 0 2024-04-24T22:45:20.845 INFO:tasks.workunit.client.0.smithi063.stdout:9/226: write d6/d8/db/d2b/f34 [1785524,11516] 0 2024-04-24T22:45:20.845 INFO:tasks.workunit.client.0.smithi063.stdout:2/228: mknod d0/d2/d3/d7/c46 0 2024-04-24T22:45:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:9/227: write d6/d8/d1f/d20/d22/f47 [79214,50950] 0 2024-04-24T22:45:20.847 INFO:tasks.workunit.client.0.smithi063.stdout:5/231: unlink d5/d11/f47 0 2024-04-24T22:45:20.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/243: getdents d1/d11/d32 0 2024-04-24T22:45:20.850 INFO:tasks.workunit.client.0.smithi063.stdout:3/248: creat da/f54 x:0 0 0 2024-04-24T22:45:20.851 INFO:tasks.workunit.client.0.smithi063.stdout:2/229: creat d0/d2/f47 x:0 0 0 2024-04-24T22:45:20.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/259: truncate d3/d27/d28/f26 3448321 0 2024-04-24T22:45:20.854 INFO:tasks.workunit.client.0.smithi063.stdout:4/222: symlink d2/l42 0 2024-04-24T22:45:20.854 INFO:tasks.workunit.client.0.smithi063.stdout:5/232: read d5/d11/d18/f41 [3216298,14551] 0 2024-04-24T22:45:20.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/244: symlink d1/d11/d3c/l52 0 2024-04-24T22:45:20.856 INFO:tasks.workunit.client.0.smithi063.stdout:4/223: dread d2/d8/f31 [0,4194304] 0 2024-04-24T22:45:20.857 INFO:tasks.workunit.client.0.smithi063.stdout:4/224: write d2/d19/f25 [1253836,95532] 0 2024-04-24T22:45:20.859 INFO:tasks.workunit.client.0.smithi063.stdout:3/249: rename da/d18/c1c to da/d18/d47/d4d/c55 0 2024-04-24T22:45:20.859 INFO:tasks.workunit.client.0.smithi063.stdout:0/225: write d6/dd/f3f [588592,41723] 0 2024-04-24T22:45:20.859 INFO:tasks.workunit.client.1.smithi192.stdout:0/260: dwrite d3/d27/d28/f24 [0,4194304] 0 2024-04-24T22:45:20.859 INFO:tasks.workunit.client.1.smithi192.stdout:0/261: dread - d3/f5a zero size 2024-04-24T22:45:20.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/262: write d3/f1c [202872,86110] 0 2024-04-24T22:45:20.861 INFO:tasks.workunit.client.0.smithi063.stdout:2/230: mkdir d0/d2/d3/d48 0 2024-04-24T22:45:20.863 INFO:tasks.workunit.client.1.smithi192.stdout:0/263: mknod d3/d27/d3c/c5b 0 2024-04-24T22:45:20.863 INFO:tasks.workunit.client.0.smithi063.stdout:2/231: write d0/d2/d27/f34 [795323,61583] 0 2024-04-24T22:45:20.864 INFO:tasks.workunit.client.0.smithi063.stdout:5/233: symlink d5/d24/d49/d3b/l57 0 2024-04-24T22:45:20.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/250: dwrite da/f3b [0,4194304] 0 2024-04-24T22:45:20.867 INFO:tasks.workunit.client.1.smithi192.stdout:7/168: sync 2024-04-24T22:45:20.867 INFO:tasks.workunit.client.1.smithi192.stdout:5/168: sync 2024-04-24T22:45:20.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/226: mkdir d6/dd/d20/d21/d53 0 2024-04-24T22:45:20.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/264: dwrite d3/d27/d28/d2e/f4c [0,4194304] 0 2024-04-24T22:45:20.871 INFO:tasks.workunit.client.1.smithi192.stdout:7/169: truncate d2/db/d2a/f2b 686840 0 2024-04-24T22:45:20.877 INFO:tasks.workunit.client.0.smithi063.stdout:0/227: truncate d6/dd/d20/d21/f36 446041 0 2024-04-24T22:45:20.878 INFO:tasks.workunit.client.0.smithi063.stdout:2/232: dwrite d0/d2/d3/d7/f19 [0,4194304] 0 2024-04-24T22:45:20.878 INFO:tasks.workunit.client.0.smithi063.stdout:4/225: dwrite f0 [0,4194304] 0 2024-04-24T22:45:20.879 INFO:tasks.workunit.client.1.smithi192.stdout:5/169: unlink d0/f29 0 2024-04-24T22:45:20.879 INFO:tasks.workunit.client.1.smithi192.stdout:5/170: readlink d0/d13/d19/l9 0 2024-04-24T22:45:20.882 INFO:tasks.workunit.client.1.smithi192.stdout:0/265: rename d3/d25 to d3/d27/d28/d2e/d4a/d5c 0 2024-04-24T22:45:20.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/245: symlink d1/l53 0 2024-04-24T22:45:20.891 INFO:tasks.workunit.client.0.smithi063.stdout:9/228: dwrite d6/d8/db/d2b/f44 [0,4194304] 0 2024-04-24T22:45:20.892 INFO:tasks.workunit.client.0.smithi063.stdout:6/224: sync 2024-04-24T22:45:20.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/202: sync 2024-04-24T22:45:20.893 INFO:tasks.workunit.client.0.smithi063.stdout:1/239: sync 2024-04-24T22:45:20.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/266: mkdir d3/d27/d3c/d5d 0 2024-04-24T22:45:20.894 INFO:tasks.workunit.client.0.smithi063.stdout:6/225: readlink db/d1d/d20/d29/l34 0 2024-04-24T22:45:20.894 INFO:tasks.workunit.client.1.smithi192.stdout:0/267: chown d3/f14 77779306 1 2024-04-24T22:45:20.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/240: write d3/da/f3d [944516,70138] 0 2024-04-24T22:45:20.897 INFO:tasks.workunit.client.0.smithi063.stdout:3/251: unlink da/d39/f4b 0 2024-04-24T22:45:20.899 INFO:tasks.workunit.client.0.smithi063.stdout:6/226: read db/f10 [1201566,97002] 0 2024-04-24T22:45:20.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/228: creat d6/dd/f54 x:0 0 0 2024-04-24T22:45:20.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/268: readlink d3/d27/l36 0 2024-04-24T22:45:20.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/269: chown d3/d27 15359 1 2024-04-24T22:45:20.902 INFO:tasks.workunit.client.0.smithi063.stdout:7/203: dwrite d0/d4/de/d17/f42 [0,4194304] 0 2024-04-24T22:45:20.905 INFO:tasks.workunit.client.0.smithi063.stdout:2/233: creat d0/dd/f49 x:0 0 0 2024-04-24T22:45:20.909 INFO:tasks.workunit.client.1.smithi192.stdout:7/170: getdents d2 0 2024-04-24T22:45:20.912 INFO:tasks.workunit.client.0.smithi063.stdout:3/252: creat da/d48/f56 x:0 0 0 2024-04-24T22:45:20.914 INFO:tasks.workunit.client.0.smithi063.stdout:4/226: dread d2/fd [0,4194304] 0 2024-04-24T22:45:20.917 INFO:tasks.workunit.client.1.smithi192.stdout:5/171: getdents d0/d1b/d1c/d2b/d2f 0 2024-04-24T22:45:20.918 INFO:tasks.workunit.client.0.smithi063.stdout:5/234: write d5/f10 [1221784,14271] 0 2024-04-24T22:45:20.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/270: creat d3/f5e x:0 0 0 2024-04-24T22:45:20.920 INFO:tasks.workunit.client.0.smithi063.stdout:6/227: mkdir db/d1d/d2d/d48 0 2024-04-24T22:45:20.922 INFO:tasks.workunit.client.1.smithi192.stdout:7/171: truncate f1 500790 0 2024-04-24T22:45:20.928 INFO:tasks.workunit.client.0.smithi063.stdout:8/246: dread d1/d2/f1c [0,4194304] 0 2024-04-24T22:45:20.929 INFO:tasks.workunit.client.1.smithi192.stdout:5/172: mkdir d0/d1b/d1c/d2b/d2f/d34 0 2024-04-24T22:45:20.929 INFO:tasks.workunit.client.0.smithi063.stdout:8/247: chown d1/d2/d17/l18 466755 1 2024-04-24T22:45:20.930 INFO:tasks.workunit.client.1.smithi192.stdout:5/173: fdatasync d0/f23 0 2024-04-24T22:45:20.930 INFO:tasks.workunit.client.0.smithi063.stdout:8/248: stat d1/d2/fe 0 2024-04-24T22:45:20.931 INFO:tasks.workunit.client.1.smithi192.stdout:5/174: write d0/d1b/d1c/f33 [105113,4184] 0 2024-04-24T22:45:20.935 INFO:tasks.workunit.client.1.smithi192.stdout:7/172: unlink d2/db/c1c 0 2024-04-24T22:45:20.938 INFO:tasks.workunit.client.1.smithi192.stdout:0/271: mknod d3/c5f 0 2024-04-24T22:45:20.941 INFO:tasks.workunit.client.0.smithi063.stdout:4/227: symlink d2/d19/d33/l43 0 2024-04-24T22:45:20.944 INFO:tasks.workunit.client.1.smithi192.stdout:0/272: mknod d3/d27/d28/d2e/d3f/c60 0 2024-04-24T22:45:20.945 INFO:tasks.workunit.client.1.smithi192.stdout:0/273: write d3/d27/d28/d2e/f4c [31748,36482] 0 2024-04-24T22:45:20.946 INFO:tasks.workunit.client.1.smithi192.stdout:0/274: read d3/d27/d28/f17 [684477,15997] 0 2024-04-24T22:45:20.946 INFO:tasks.workunit.client.0.smithi063.stdout:6/228: chown db/cc 4127850 1 2024-04-24T22:45:20.947 INFO:tasks.workunit.client.1.smithi192.stdout:0/275: chown d3/d27/d28/c1e 0 1 2024-04-24T22:45:20.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/229: dwrite d6/dd/f12 [4194304,4194304] 0 2024-04-24T22:45:20.949 INFO:tasks.workunit.client.1.smithi192.stdout:7/173: rmdir d2/db 39 2024-04-24T22:45:20.950 INFO:tasks.workunit.client.0.smithi063.stdout:8/249: mkdir d1/d11/d49/d54 0 2024-04-24T22:45:20.951 INFO:tasks.workunit.client.1.smithi192.stdout:5/175: write d0/d1b/d1c/f27 [7767653,126310] 0 2024-04-24T22:45:20.952 INFO:tasks.workunit.client.1.smithi192.stdout:5/176: truncate d0/fc 4848125 0 2024-04-24T22:45:20.953 INFO:tasks.workunit.client.0.smithi063.stdout:9/229: rename d6/d8/d24 to d6/d8/d4d 0 2024-04-24T22:45:20.954 INFO:tasks.workunit.client.0.smithi063.stdout:9/230: readlink d6/d2f/l39 0 2024-04-24T22:45:20.955 INFO:tasks.workunit.client.1.smithi192.stdout:0/276: creat d3/d27/d28/d2e/d4a/d5c/f61 x:0 0 0 2024-04-24T22:45:20.955 INFO:tasks.workunit.client.0.smithi063.stdout:3/253: getdents da/d18/d3a 0 2024-04-24T22:45:20.957 INFO:tasks.workunit.client.0.smithi063.stdout:4/228: creat d2/d8/d36/f44 x:0 0 0 2024-04-24T22:45:20.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/229: write d2/d8/f11 [2781228,71510] 0 2024-04-24T22:45:20.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/230: readlink d2/d8/l39 0 2024-04-24T22:45:20.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/231: read d2/fb [195566,54193] 0 2024-04-24T22:45:20.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/241: creat d3/da/dc/d1f/d21/f53 x:0 0 0 2024-04-24T22:45:20.962 INFO:tasks.workunit.client.1.smithi192.stdout:0/277: rename d3/d27/d28/c2d to d3/d40/c62 0 2024-04-24T22:45:20.963 INFO:tasks.workunit.client.0.smithi063.stdout:6/229: creat db/d1d/f49 x:0 0 0 2024-04-24T22:45:20.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/230: fsync f8 0 2024-04-24T22:45:20.966 INFO:tasks.workunit.client.0.smithi063.stdout:1/242: dwrite d3/f3c [0,4194304] 0 2024-04-24T22:45:20.967 INFO:tasks.workunit.client.1.smithi192.stdout:5/177: link d0/f30 d0/d1b/d1c/d2c/f35 0 2024-04-24T22:45:20.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/278: dwrite d3/f5e [0,4194304] 0 2024-04-24T22:45:20.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/174: dread d2/db/f10 [0,4194304] 0 2024-04-24T22:45:20.970 INFO:tasks.workunit.client.1.smithi192.stdout:0/279: chown d3/d27/d28/f17 1573 1 2024-04-24T22:45:20.973 INFO:tasks.workunit.client.1.smithi192.stdout:5/178: mknod d0/d13/d19/c36 0 2024-04-24T22:45:20.973 INFO:tasks.workunit.client.1.smithi192.stdout:5/179: dread - d0/d1b/d1c/d2b/f32 zero size 2024-04-24T22:45:20.975 INFO:tasks.workunit.client.0.smithi063.stdout:6/231: dwrite db/d1d/d20/f38 [0,4194304] 0 2024-04-24T22:45:20.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/231: mknod d6/d8/db/d2b/d36/c4e 0 2024-04-24T22:45:20.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/235: creat d5/d24/d49/d3b/f58 x:0 0 0 2024-04-24T22:45:20.979 INFO:tasks.workunit.client.0.smithi063.stdout:8/250: dwrite d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:20.979 INFO:tasks.workunit.client.1.smithi192.stdout:5/180: rename d0/f30 to d0/d1b/f37 0 2024-04-24T22:45:20.980 INFO:tasks.workunit.client.0.smithi063.stdout:4/232: dread d2/f26 [0,4194304] 0 2024-04-24T22:45:20.984 INFO:tasks.workunit.client.1.smithi192.stdout:0/280: dwrite d3/d40/f50 [0,4194304] 0 2024-04-24T22:45:20.986 INFO:tasks.workunit.client.1.smithi192.stdout:5/181: creat d0/d13/d19/f38 x:0 0 0 2024-04-24T22:45:20.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/182: sync 2024-04-24T22:45:20.987 INFO:tasks.workunit.client.1.smithi192.stdout:9/152: sync 2024-04-24T22:45:20.987 INFO:tasks.workunit.client.1.smithi192.stdout:2/252: sync 2024-04-24T22:45:20.987 INFO:tasks.workunit.client.0.smithi063.stdout:2/234: getdents d0/d2/d35 0 2024-04-24T22:45:20.988 INFO:tasks.workunit.client.0.smithi063.stdout:2/235: write d0/dd/f49 [312841,44019] 0 2024-04-24T22:45:20.988 INFO:tasks.workunit.client.1.smithi192.stdout:2/253: dread - d4/d9/d37/f3e zero size 2024-04-24T22:45:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:2/236: dread d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:20.990 INFO:tasks.workunit.client.1.smithi192.stdout:0/281: write d3/d27/d28/d2e/d4a/d5c/f51 [898483,72255] 0 2024-04-24T22:45:20.992 INFO:tasks.workunit.client.0.smithi063.stdout:2/237: fdatasync d0/dd/f49 0 2024-04-24T22:45:20.993 INFO:tasks.workunit.client.1.smithi192.stdout:0/282: write d3/d27/d28/f1d [180394,104997] 0 2024-04-24T22:45:20.995 INFO:tasks.workunit.client.1.smithi192.stdout:1/183: dread d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:20.996 INFO:tasks.workunit.client.1.smithi192.stdout:0/283: dread - d3/d27/d28/d31/f52 zero size 2024-04-24T22:45:20.996 INFO:tasks.workunit.client.0.smithi063.stdout:9/232: fdatasync d6/d8/db/f10 0 2024-04-24T22:45:20.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/254: dread d4/d9/d37/f3b [0,4194304] 0 2024-04-24T22:45:20.997 INFO:tasks.workunit.client.1.smithi192.stdout:9/153: mkdir d3/dd/de/d39 0 2024-04-24T22:45:20.998 INFO:tasks.workunit.client.1.smithi192.stdout:9/154: dread - d3/dd/de/f2f zero size 2024-04-24T22:45:20.999 INFO:tasks.workunit.client.1.smithi192.stdout:9/155: chown d3/dd/de/c20 10882 1 2024-04-24T22:45:21.000 INFO:tasks.workunit.client.0.smithi063.stdout:5/236: rmdir d5/d11/d33 39 2024-04-24T22:45:21.001 INFO:tasks.workunit.client.0.smithi063.stdout:5/237: stat d5/d11/d23/f4e 0 2024-04-24T22:45:21.002 INFO:tasks.workunit.client.0.smithi063.stdout:8/251: creat d1/d11/d3c/f55 x:0 0 0 2024-04-24T22:45:21.003 INFO:tasks.workunit.client.0.smithi063.stdout:5/238: read d5/d24/d44/f46 [1082243,5130] 0 2024-04-24T22:45:21.007 INFO:tasks.workunit.client.1.smithi192.stdout:7/175: dwrite f1 [0,4194304] 0 2024-04-24T22:45:21.008 INFO:tasks.workunit.client.1.smithi192.stdout:7/176: fsync f0 0 2024-04-24T22:45:21.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/177: write f0 [2296651,47566] 0 2024-04-24T22:45:21.010 INFO:tasks.workunit.client.0.smithi063.stdout:0/230: rename d6/da/dc/d26/f43 to d6/dd/d20/d21/d25/f55 0 2024-04-24T22:45:21.013 INFO:tasks.workunit.client.0.smithi063.stdout:1/243: stat d3/da/dc/d1f/d21/d49/f4f 0 2024-04-24T22:45:21.014 INFO:tasks.workunit.client.0.smithi063.stdout:1/244: stat d3/da/dc/d1f/d21 0 2024-04-24T22:45:21.016 INFO:tasks.workunit.client.1.smithi192.stdout:1/184: write d9/d15/f1f [2759716,82359] 0 2024-04-24T22:45:21.016 INFO:tasks.workunit.client.1.smithi192.stdout:2/255: write d4/f19 [429459,37188] 0 2024-04-24T22:45:21.018 INFO:tasks.workunit.client.0.smithi063.stdout:6/232: mknod db/d1d/d20/c4a 0 2024-04-24T22:45:21.018 INFO:tasks.workunit.client.1.smithi192.stdout:2/256: read d4/d9/f18 [243600,24251] 0 2024-04-24T22:45:21.019 INFO:tasks.workunit.client.1.smithi192.stdout:0/284: creat d3/d27/d28/d2e/f63 x:0 0 0 2024-04-24T22:45:21.020 INFO:tasks.workunit.client.0.smithi063.stdout:6/233: read db/ff [3114278,111069] 0 2024-04-24T22:45:21.020 INFO:tasks.workunit.client.0.smithi063.stdout:6/234: chown db/c15 348 1 2024-04-24T22:45:21.021 INFO:tasks.workunit.client.1.smithi192.stdout:8/182: sync 2024-04-24T22:45:21.021 INFO:tasks.workunit.client.1.smithi192.stdout:3/183: sync 2024-04-24T22:45:21.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/153: sync 2024-04-24T22:45:21.022 INFO:tasks.workunit.client.1.smithi192.stdout:6/179: sync 2024-04-24T22:45:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:6/235: fdatasync db/f37 0 2024-04-24T22:45:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:6/236: dread - db/f37 zero size 2024-04-24T22:45:21.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/182: truncate d0/d1b/d1c/f27 7496874 0 2024-04-24T22:45:21.024 INFO:tasks.workunit.client.0.smithi063.stdout:9/233: chown d6/ce 11007 1 2024-04-24T22:45:21.025 INFO:tasks.workunit.client.0.smithi063.stdout:9/234: chown d6/d8/l26 276888386 1 2024-04-24T22:45:21.026 INFO:tasks.workunit.client.1.smithi192.stdout:1/185: dwrite d9/d15/d2d/f37 [0,4194304] 0 2024-04-24T22:45:21.028 INFO:tasks.workunit.client.0.smithi063.stdout:8/252: creat d1/d2/d3/f56 x:0 0 0 2024-04-24T22:45:21.028 INFO:tasks.workunit.client.0.smithi063.stdout:6/237: dread db/d1d/d20/d29/f2e [0,4194304] 0 2024-04-24T22:45:21.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/178: mknod d2/db/c32 0 2024-04-24T22:45:21.031 INFO:tasks.workunit.client.0.smithi063.stdout:4/233: truncate d2/d19/f20 1488393 0 2024-04-24T22:45:21.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/179: dread f1 [0,4194304] 0 2024-04-24T22:45:21.035 INFO:tasks.workunit.client.1.smithi192.stdout:2/257: rename d4/d9/d36/f3f to d4/d9/d37/f4e 0 2024-04-24T22:45:21.039 INFO:tasks.workunit.client.0.smithi063.stdout:2/238: symlink d0/l4a 0 2024-04-24T22:45:21.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/154: rmdir d2/db/d10 39 2024-04-24T22:45:21.042 INFO:tasks.workunit.client.0.smithi063.stdout:2/239: dread d0/d2/f1b [0,4194304] 0 2024-04-24T22:45:21.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/180: mknod d0/d8/d18/d2e/c45 0 2024-04-24T22:45:21.045 INFO:tasks.workunit.client.1.smithi192.stdout:4/155: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:21.045 INFO:tasks.workunit.client.0.smithi063.stdout:9/235: mkdir d6/d8/d4d/d4f 0 2024-04-24T22:45:21.046 INFO:tasks.workunit.client.0.smithi063.stdout:9/236: stat d6/d8/db/d1c/l43 0 2024-04-24T22:45:21.047 INFO:tasks.workunit.client.0.smithi063.stdout:9/237: chown d6/d2f/f35 3 1 2024-04-24T22:45:21.048 INFO:tasks.workunit.client.1.smithi192.stdout:5/183: creat d0/d13/f39 x:0 0 0 2024-04-24T22:45:21.049 INFO:tasks.workunit.client.1.smithi192.stdout:6/181: dread d0/d8/f31 [0,4194304] 0 2024-04-24T22:45:21.049 INFO:tasks.workunit.client.0.smithi063.stdout:8/253: creat d1/d22/d33/f57 x:0 0 0 2024-04-24T22:45:21.049 INFO:tasks.workunit.client.1.smithi192.stdout:6/182: chown d0 110991118 1 2024-04-24T22:45:21.051 INFO:tasks.workunit.client.1.smithi192.stdout:3/184: creat d2/d4/d12/d1a/d1d/f3f x:0 0 0 2024-04-24T22:45:21.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/239: mknod d5/d11/d33/c59 0 2024-04-24T22:45:21.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/254: dwrite d1/d11/f29 [0,4194304] 0 2024-04-24T22:45:21.057 INFO:tasks.workunit.client.0.smithi063.stdout:0/231: symlink d6/l56 0 2024-04-24T22:45:21.058 INFO:tasks.workunit.client.1.smithi192.stdout:5/184: dwrite d0/d13/f39 [0,4194304] 0 2024-04-24T22:45:21.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/285: dwrite d3/d27/d28/f19 [0,4194304] 0 2024-04-24T22:45:21.060 INFO:tasks.workunit.client.0.smithi063.stdout:1/245: dwrite d3/da/f36 [0,4194304] 0 2024-04-24T22:45:21.061 INFO:tasks.workunit.client.1.smithi192.stdout:7/180: mknod d2/db/c33 0 2024-04-24T22:45:21.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/255: dwrite d1/d2/d17/f3d [0,4194304] 0 2024-04-24T22:45:21.063 INFO:tasks.workunit.client.1.smithi192.stdout:3/185: dwrite d2/d4/f3e [0,4194304] 0 2024-04-24T22:45:21.066 INFO:tasks.workunit.client.1.smithi192.stdout:5/185: truncate d0/d1b/d1c/d2b/f32 172907 0 2024-04-24T22:45:21.070 INFO:tasks.workunit.client.0.smithi063.stdout:2/240: rmdir d0/dd 39 2024-04-24T22:45:21.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/186: fdatasync d0/d13/d19/f38 0 2024-04-24T22:45:21.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/187: stat d0/d1b 0 2024-04-24T22:45:21.075 INFO:tasks.workunit.client.0.smithi063.stdout:9/238: rmdir d6/d8/d1f/d20/d2a 39 2024-04-24T22:45:21.075 INFO:tasks.workunit.client.1.smithi192.stdout:7/181: dwrite d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:21.077 INFO:tasks.workunit.client.1.smithi192.stdout:8/183: mkdir db/dc/d22/d31/d33 0 2024-04-24T22:45:21.078 INFO:tasks.workunit.client.1.smithi192.stdout:8/184: chown f6 1243 1 2024-04-24T22:45:21.078 INFO:tasks.workunit.client.0.smithi063.stdout:2/241: dwrite d0/d2/d3/f25 [0,4194304] 0 2024-04-24T22:45:21.080 INFO:tasks.workunit.client.0.smithi063.stdout:5/240: creat d5/d24/d36/f5a x:0 0 0 2024-04-24T22:45:21.083 INFO:tasks.workunit.client.1.smithi192.stdout:6/183: chown d0/d8/d29/f30 154709 1 2024-04-24T22:45:21.086 INFO:tasks.workunit.client.1.smithi192.stdout:1/186: mknod d9/c40 0 2024-04-24T22:45:21.088 INFO:tasks.workunit.client.0.smithi063.stdout:0/232: mknod d6/dd/d20/d21/c57 0 2024-04-24T22:45:21.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/241: dwrite d5/d24/d49/d3b/f58 [0,4194304] 0 2024-04-24T22:45:21.089 INFO:tasks.workunit.client.1.smithi192.stdout:7/182: dread f0 [0,4194304] 0 2024-04-24T22:45:21.090 INFO:tasks.workunit.client.0.smithi063.stdout:5/242: chown d5/d11 6 1 2024-04-24T22:45:21.090 INFO:tasks.workunit.client.0.smithi063.stdout:0/233: write d6/dd/d20/d21/f36 [967820,44328] 0 2024-04-24T22:45:21.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/246: symlink d3/da/dc/d1f/d21/d42/l54 0 2024-04-24T22:45:21.094 INFO:tasks.workunit.client.0.smithi063.stdout:5/243: truncate d5/d24/d49/d3b/f45 178129 0 2024-04-24T22:45:21.095 INFO:tasks.workunit.client.0.smithi063.stdout:5/244: fsync d5/d11/d18/f19 0 2024-04-24T22:45:21.095 INFO:tasks.workunit.client.0.smithi063.stdout:2/242: unlink d0/d2/d3/l17 0 2024-04-24T22:45:21.096 INFO:tasks.workunit.client.0.smithi063.stdout:1/247: dread d3/f23 [0,4194304] 0 2024-04-24T22:45:21.097 INFO:tasks.workunit.client.1.smithi192.stdout:0/286: mkdir d3/d27/d3c/d64 0 2024-04-24T22:45:21.097 INFO:tasks.workunit.client.0.smithi063.stdout:5/245: read d5/fd [1548804,110042] 0 2024-04-24T22:45:21.097 INFO:tasks.workunit.client.0.smithi063.stdout:1/248: write d3/f3c [5179332,83413] 0 2024-04-24T22:45:21.098 INFO:tasks.workunit.client.0.smithi063.stdout:1/249: dread - d3/da/dc/d1f/f3f zero size 2024-04-24T22:45:21.099 INFO:tasks.workunit.client.0.smithi063.stdout:4/234: fsync d2/d19/f20 0 2024-04-24T22:45:21.100 INFO:tasks.workunit.client.1.smithi192.stdout:9/156: getdents d3/d24 0 2024-04-24T22:45:21.101 INFO:tasks.workunit.client.0.smithi063.stdout:4/235: readlink d2/d8/d36/l38 0 2024-04-24T22:45:21.101 INFO:tasks.workunit.client.1.smithi192.stdout:3/186: symlink d2/d4/d12/d1a/d1d/l40 0 2024-04-24T22:45:21.103 INFO:tasks.workunit.client.1.smithi192.stdout:0/287: dread d3/d40/f50 [0,4194304] 0 2024-04-24T22:45:21.104 INFO:tasks.workunit.client.0.smithi063.stdout:5/246: dwrite d5/d11/f2f [4194304,4194304] 0 2024-04-24T22:45:21.106 INFO:tasks.workunit.client.0.smithi063.stdout:0/234: rename d6/l9 to d6/dd/l58 0 2024-04-24T22:45:21.107 INFO:tasks.workunit.client.0.smithi063.stdout:0/235: read d6/dd/d20/d21/f48 [175870,78648] 0 2024-04-24T22:45:21.110 INFO:tasks.workunit.client.0.smithi063.stdout:2/243: dread d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:21.110 INFO:tasks.workunit.client.1.smithi192.stdout:6/184: mknod d0/d8/d3d/c46 0 2024-04-24T22:45:21.112 INFO:tasks.workunit.client.0.smithi063.stdout:1/250: symlink d3/da/l55 0 2024-04-24T22:45:21.112 INFO:tasks.workunit.client.1.smithi192.stdout:7/183: mknod d2/db/c34 0 2024-04-24T22:45:21.115 INFO:tasks.workunit.client.0.smithi063.stdout:8/256: creat d1/f58 x:0 0 0 2024-04-24T22:45:21.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/157: fsync f2 0 2024-04-24T22:45:21.117 INFO:tasks.workunit.client.0.smithi063.stdout:8/257: chown d1/d22/f31 895 1 2024-04-24T22:45:21.119 INFO:tasks.workunit.client.1.smithi192.stdout:0/288: creat d3/d27/d3c/d3b/f65 x:0 0 0 2024-04-24T22:45:21.119 INFO:tasks.workunit.client.1.smithi192.stdout:0/289: readlink d3/l2f 0 2024-04-24T22:45:21.120 INFO:tasks.workunit.client.0.smithi063.stdout:4/236: dread d2/f5 [0,4194304] 0 2024-04-24T22:45:21.120 INFO:tasks.workunit.client.0.smithi063.stdout:1/251: dwrite d3/da/f3d [0,4194304] 0 2024-04-24T22:45:21.121 INFO:tasks.workunit.client.0.smithi063.stdout:0/236: creat d6/dd/d20/d24/f59 x:0 0 0 2024-04-24T22:45:21.121 INFO:tasks.workunit.client.1.smithi192.stdout:9/158: dread d3/dd/de/f1a [0,4194304] 0 2024-04-24T22:45:21.122 INFO:tasks.workunit.client.0.smithi063.stdout:4/237: write d2/d19/f25 [2003940,25887] 0 2024-04-24T22:45:21.122 INFO:tasks.workunit.client.1.smithi192.stdout:6/185: creat d0/d8/d3d/d3e/d40/f47 x:0 0 0 2024-04-24T22:45:21.126 INFO:tasks.workunit.client.0.smithi063.stdout:2/244: rename d0/d2/d35/f43 to d0/d2/f4b 0 2024-04-24T22:45:21.126 INFO:tasks.workunit.client.0.smithi063.stdout:0/237: dread - d6/da/f3d zero size 2024-04-24T22:45:21.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/184: write d2/fa [3351149,101091] 0 2024-04-24T22:45:21.127 INFO:tasks.workunit.client.1.smithi192.stdout:2/258: getdents d4/d9 0 2024-04-24T22:45:21.127 INFO:tasks.workunit.client.0.smithi063.stdout:2/245: chown d0/d2/l6 378 1 2024-04-24T22:45:21.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/259: readlink d4/d9/l2e 0 2024-04-24T22:45:21.129 INFO:tasks.workunit.client.1.smithi192.stdout:2/260: chown d4 1 1 2024-04-24T22:45:21.129 INFO:tasks.workunit.client.0.smithi063.stdout:9/239: getdents d6/d8/db/d2b/d36 0 2024-04-24T22:45:21.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/159: dwrite d3/f21 [0,4194304] 0 2024-04-24T22:45:21.130 INFO:tasks.workunit.client.0.smithi063.stdout:8/258: dwrite d1/d22/f2e [0,4194304] 0 2024-04-24T22:45:21.132 INFO:tasks.workunit.client.1.smithi192.stdout:2/261: dread d4/d29/f34 [0,4194304] 0 2024-04-24T22:45:21.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/187: creat d9/dd/f41 x:0 0 0 2024-04-24T22:45:21.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/262: dread d4/d29/f34 [0,4194304] 0 2024-04-24T22:45:21.136 INFO:tasks.workunit.client.0.smithi063.stdout:8/259: dread d1/d11/f29 [0,4194304] 0 2024-04-24T22:45:21.137 INFO:tasks.workunit.client.1.smithi192.stdout:2/263: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:21.138 INFO:tasks.workunit.client.0.smithi063.stdout:4/238: unlink f0 0 2024-04-24T22:45:21.139 INFO:tasks.workunit.client.1.smithi192.stdout:2/264: write d4/f41 [330831,100940] 0 2024-04-24T22:45:21.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/247: creat d5/f5b x:0 0 0 2024-04-24T22:45:21.145 INFO:tasks.workunit.client.1.smithi192.stdout:7/185: symlink d2/db/d21/l35 0 2024-04-24T22:45:21.146 INFO:tasks.workunit.client.0.smithi063.stdout:5/248: chown d5/d11/d23/f4e 992 1 2024-04-24T22:45:21.146 INFO:tasks.workunit.client.0.smithi063.stdout:2/246: symlink d0/d2/d39/l4c 0 2024-04-24T22:45:21.148 INFO:tasks.workunit.client.1.smithi192.stdout:3/187: mkdir d2/d4/d7/d41 0 2024-04-24T22:45:21.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/188: chown d2/d27/d28/l31 136764 1 2024-04-24T22:45:21.150 INFO:tasks.workunit.client.1.smithi192.stdout:3/189: chown d2/d4 5 1 2024-04-24T22:45:21.150 INFO:tasks.workunit.client.0.smithi063.stdout:4/239: dwrite d2/d19/f2b [0,4194304] 0 2024-04-24T22:45:21.151 INFO:tasks.workunit.client.1.smithi192.stdout:9/160: creat d3/dd/f3a x:0 0 0 2024-04-24T22:45:21.151 INFO:tasks.workunit.client.0.smithi063.stdout:2/247: write d0/d2/d3/d7/f19 [3313791,12067] 0 2024-04-24T22:45:21.152 INFO:tasks.workunit.client.1.smithi192.stdout:3/190: write d2/d4/d12/d1a/d1d/f3f [797107,20055] 0 2024-04-24T22:45:21.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/290: symlink d3/l66 0 2024-04-24T22:45:21.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/291: chown d3/l6 4867 1 2024-04-24T22:45:21.155 INFO:tasks.workunit.client.0.smithi063.stdout:2/248: dread d0/d2/d27/f34 [0,4194304] 0 2024-04-24T22:45:21.157 INFO:tasks.workunit.client.1.smithi192.stdout:2/265: fdatasync d4/d9/d37/f3b 0 2024-04-24T22:45:21.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/266: chown d4/d9/f2d 61 1 2024-04-24T22:45:21.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/186: mkdir d2/db/d36 0 2024-04-24T22:45:21.161 INFO:tasks.workunit.client.1.smithi192.stdout:7/187: stat d2/db/l1b 0 2024-04-24T22:45:21.162 INFO:tasks.workunit.client.0.smithi063.stdout:9/240: mknod d6/d8/d4d/d4f/c50 0 2024-04-24T22:45:21.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/292: dwrite d3/d27/d28/f24 [0,4194304] 0 2024-04-24T22:45:21.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/249: creat d5/d24/d44/f5c x:0 0 0 2024-04-24T22:45:21.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/293: stat d3/d27/d3c 0 2024-04-24T22:45:21.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/161: getdents d3/dd/de/d17/d33 0 2024-04-24T22:45:21.166 INFO:tasks.workunit.client.1.smithi192.stdout:2/267: unlink d4/d9/f2d 0 2024-04-24T22:45:21.167 INFO:tasks.workunit.client.0.smithi063.stdout:5/250: dread d5/d11/d18/f41 [0,4194304] 0 2024-04-24T22:45:21.168 INFO:tasks.workunit.client.0.smithi063.stdout:5/251: chown d5/f1c 1378 1 2024-04-24T22:45:21.169 INFO:tasks.workunit.client.0.smithi063.stdout:8/260: link d1/d22/d33/f57 d1/d2/d3/f59 0 2024-04-24T22:45:21.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/188: creat d9/f42 x:0 0 0 2024-04-24T22:45:21.171 INFO:tasks.workunit.client.0.smithi063.stdout:2/249: fsync d0/dd/f49 0 2024-04-24T22:45:21.172 INFO:tasks.workunit.client.0.smithi063.stdout:1/252: link d3/da/dc/d1f/d2b/c47 d3/d2d/c56 0 2024-04-24T22:45:21.173 INFO:tasks.workunit.client.1.smithi192.stdout:9/162: creat d3/dd/f3b x:0 0 0 2024-04-24T22:45:21.176 INFO:tasks.workunit.client.1.smithi192.stdout:1/189: dwrite d9/dd/f29 [0,4194304] 0 2024-04-24T22:45:21.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/253: truncate d3/da/dc/d1f/d2b/f32 37629 0 2024-04-24T22:45:21.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/191: creat d2/d4/d12/f42 x:0 0 0 2024-04-24T22:45:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:20 smithi192 ceph-mon[33991]: pgmap v8: 65 pgs: 65 active+clean; 542 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 46 MiB/s wr, 138 op/s 2024-04-24T22:45:21.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/163: symlink d3/dd/de/l3c 0 2024-04-24T22:45:21.182 INFO:tasks.workunit.client.1.smithi192.stdout:8/185: rmdir db/dc 39 2024-04-24T22:45:21.185 INFO:tasks.workunit.client.0.smithi063.stdout:2/250: dwrite d0/d2/d3/d1c/d3f/f42 [0,4194304] 0 2024-04-24T22:45:21.185 INFO:tasks.workunit.client.0.smithi063.stdout:5/252: link d5/d11/d23/f27 d5/d24/d36/f5d 0 2024-04-24T22:45:21.188 INFO:tasks.workunit.client.0.smithi063.stdout:5/253: stat d5/d24/d49/d3b/f45 0 2024-04-24T22:45:21.189 INFO:tasks.workunit.client.1.smithi192.stdout:8/186: read db/dc/d29/f2c [703614,639] 0 2024-04-24T22:45:21.190 INFO:tasks.workunit.client.0.smithi063.stdout:1/254: dread d3/f23 [0,4194304] 0 2024-04-24T22:45:21.190 INFO:tasks.workunit.client.0.smithi063.stdout:1/255: stat d3/da/dc/d1f/d21/c2a 0 2024-04-24T22:45:21.192 INFO:tasks.workunit.client.1.smithi192.stdout:1/190: mknod d9/d15/d2d/d31/d2f/d35/c43 0 2024-04-24T22:45:21.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/251: read d0/d2/d3/d7/fc [1499793,2930] 0 2024-04-24T22:45:21.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/252: readlink d0/l4 0 2024-04-24T22:45:21.198 INFO:tasks.workunit.client.0.smithi063.stdout:7/204: sync 2024-04-24T22:45:21.198 INFO:tasks.workunit.client.0.smithi063.stdout:3/254: sync 2024-04-24T22:45:21.200 INFO:tasks.workunit.client.0.smithi063.stdout:1/256: dwrite d3/da/dc/d1f/d2b/f43 [4194304,4194304] 0 2024-04-24T22:45:21.201 INFO:tasks.workunit.client.1.smithi192.stdout:1/191: dwrite d9/f3c [0,4194304] 0 2024-04-24T22:45:21.202 INFO:tasks.workunit.client.0.smithi063.stdout:7/205: write d0/d4/de/d17/f43 [166644,69904] 0 2024-04-24T22:45:21.205 INFO:tasks.workunit.client.1.smithi192.stdout:9/164: dread d3/f4 [0,4194304] 0 2024-04-24T22:45:21.206 INFO:tasks.workunit.client.0.smithi063.stdout:8/261: rename d1/c20 to d1/d11/d32/d38/c5a 0 2024-04-24T22:45:21.206 INFO:tasks.workunit.client.1.smithi192.stdout:9/165: dread - d3/dd/f30 zero size 2024-04-24T22:45:21.212 INFO:tasks.workunit.client.1.smithi192.stdout:8/187: creat db/f34 x:0 0 0 2024-04-24T22:45:21.214 INFO:tasks.workunit.client.0.smithi063.stdout:2/253: creat d0/d2/d2d/f4d x:0 0 0 2024-04-24T22:45:21.215 INFO:tasks.workunit.client.1.smithi192.stdout:9/166: dwrite d3/dd/f38 [0,4194304] 0 2024-04-24T22:45:21.216 INFO:tasks.workunit.client.0.smithi063.stdout:3/255: mknod da/d39/c57 0 2024-04-24T22:45:21.218 INFO:tasks.workunit.client.1.smithi192.stdout:4/156: truncate d2/d9/d25/f32 498728 0 2024-04-24T22:45:21.218 INFO:tasks.workunit.client.0.smithi063.stdout:5/254: dread d5/d11/f15 [0,4194304] 0 2024-04-24T22:45:21.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/157: dread - d2/d9/d1e/f24 zero size 2024-04-24T22:45:21.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/257: symlink d3/d2d/l57 0 2024-04-24T22:45:21.223 INFO:tasks.workunit.client.1.smithi192.stdout:8/188: creat db/dc/d22/f35 x:0 0 0 2024-04-24T22:45:21.226 INFO:tasks.workunit.client.1.smithi192.stdout:1/192: rename d9/d15/d2d/d31/d2f/d35/c43 to d9/c44 0 2024-04-24T22:45:21.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/193: readlink d9/dd/le 0 2024-04-24T22:45:21.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/158: dwrite d2/f1f [0,4194304] 0 2024-04-24T22:45:21.228 INFO:tasks.workunit.client.1.smithi192.stdout:4/159: read - d2/d9/d1e/f24 zero size 2024-04-24T22:45:21.230 INFO:tasks.workunit.client.1.smithi192.stdout:8/189: symlink db/dc/d29/l36 0 2024-04-24T22:45:21.232 INFO:tasks.workunit.client.0.smithi063.stdout:2/254: symlink d0/d2/d2d/l4e 0 2024-04-24T22:45:21.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/262: dwrite d1/d11/d32/d38/f3f [0,4194304] 0 2024-04-24T22:45:21.234 INFO:tasks.workunit.client.1.smithi192.stdout:4/160: mknod d2/d9/d25/c39 0 2024-04-24T22:45:21.235 INFO:tasks.workunit.client.1.smithi192.stdout:4/161: write d2/d9/d1e/d28/f2b [316994,75847] 0 2024-04-24T22:45:21.236 INFO:tasks.workunit.client.1.smithi192.stdout:6/186: rmdir d0/d8 39 2024-04-24T22:45:21.238 INFO:tasks.workunit.client.1.smithi192.stdout:7/188: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:21.239 INFO:tasks.workunit.client.1.smithi192.stdout:8/190: symlink db/dc/l37 0 2024-04-24T22:45:21.242 INFO:tasks.workunit.client.1.smithi192.stdout:8/191: dread db/f15 [0,4194304] 0 2024-04-24T22:45:21.245 INFO:tasks.workunit.client.1.smithi192.stdout:6/187: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:21.247 INFO:tasks.workunit.client.1.smithi192.stdout:6/188: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:21.247 INFO:tasks.workunit.client.0.smithi063.stdout:6/238: sync 2024-04-24T22:45:21.248 INFO:tasks.workunit.client.1.smithi192.stdout:6/189: chown d0/d8/d18/l2f 169 1 2024-04-24T22:45:21.251 INFO:tasks.workunit.client.1.smithi192.stdout:9/167: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:21.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/190: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:21.253 INFO:tasks.workunit.client.1.smithi192.stdout:6/191: write d0/d8/f33 [80377,15342] 0 2024-04-24T22:45:21.253 INFO:tasks.workunit.client.1.smithi192.stdout:8/192: mknod db/dc/d29/c38 0 2024-04-24T22:45:21.254 INFO:tasks.workunit.client.0.smithi063.stdout:6/239: dwrite db/f37 [0,4194304] 0 2024-04-24T22:45:21.258 INFO:tasks.workunit.client.0.smithi063.stdout:5/255: fdatasync d5/f14 0 2024-04-24T22:45:21.258 INFO:tasks.workunit.client.0.smithi063.stdout:0/238: write d6/da/f10 [885003,119147] 0 2024-04-24T22:45:21.259 INFO:tasks.workunit.client.1.smithi192.stdout:4/162: mknod d2/d9/d25/d2e/d31/d36/c3a 0 2024-04-24T22:45:21.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/206: link d0/d4/de/d17/d28/c2b d0/d4/de/c50 0 2024-04-24T22:45:21.262 INFO:tasks.workunit.client.1.smithi192.stdout:9/168: mknod d3/dd/de/c3d 0 2024-04-24T22:45:21.264 INFO:tasks.workunit.client.0.smithi063.stdout:2/255: creat d0/d2/d3/d48/f4f x:0 0 0 2024-04-24T22:45:21.264 INFO:tasks.workunit.client.0.smithi063.stdout:2/256: fdatasync d0/d2/d3/d7/f19 0 2024-04-24T22:45:21.267 INFO:tasks.workunit.client.0.smithi063.stdout:2/257: truncate d0/d2/d3/f25 4490023 0 2024-04-24T22:45:21.268 INFO:tasks.workunit.client.1.smithi192.stdout:4/163: mknod d2/d9/d25/c3b 0 2024-04-24T22:45:21.272 INFO:tasks.workunit.client.0.smithi063.stdout:3/256: creat da/d18/f58 x:0 0 0 2024-04-24T22:45:21.273 INFO:tasks.workunit.client.0.smithi063.stdout:3/257: write da/d39/f3c [773966,48140] 0 2024-04-24T22:45:21.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/192: chown d2/d4/d7/f2e 497185 1 2024-04-24T22:45:21.275 INFO:tasks.workunit.client.1.smithi192.stdout:3/193: write d2/d4/d12/f1c [1063787,17743] 0 2024-04-24T22:45:21.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/169: rename f2 to d3/dd/de/d17/d33/f3e 0 2024-04-24T22:45:21.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/170: stat d3/dd/de/l1b 0 2024-04-24T22:45:21.278 INFO:tasks.workunit.client.1.smithi192.stdout:9/171: fsync d3/dd/de/f12 0 2024-04-24T22:45:21.282 INFO:tasks.workunit.client.1.smithi192.stdout:4/164: creat d2/d9/d1e/d28/d2f/f3c x:0 0 0 2024-04-24T22:45:21.283 INFO:tasks.workunit.client.1.smithi192.stdout:4/165: truncate d2/f17 615661 0 2024-04-24T22:45:21.283 INFO:tasks.workunit.client.1.smithi192.stdout:4/166: chown d2/db/l1b 4748838 1 2024-04-24T22:45:21.286 INFO:tasks.workunit.client.0.smithi063.stdout:0/239: unlink d6/dd/d20/d21/d25/d31/c39 0 2024-04-24T22:45:21.286 INFO:tasks.workunit.client.1.smithi192.stdout:3/194: mknod d2/d4/d12/d39/c43 0 2024-04-24T22:45:21.287 INFO:tasks.workunit.client.1.smithi192.stdout:3/195: fdatasync d2/d27/d28/f30 0 2024-04-24T22:45:21.288 INFO:tasks.workunit.client.0.smithi063.stdout:7/207: mknod d0/d26/d2e/c51 0 2024-04-24T22:45:21.290 INFO:tasks.workunit.client.0.smithi063.stdout:9/241: write d6/f14 [69285,17475] 0 2024-04-24T22:45:21.291 INFO:tasks.workunit.client.0.smithi063.stdout:8/263: creat d1/d22/f5b x:0 0 0 2024-04-24T22:45:21.295 INFO:tasks.workunit.client.1.smithi192.stdout:3/196: creat d2/d4/d12/d39/f44 x:0 0 0 2024-04-24T22:45:21.296 INFO:tasks.workunit.client.1.smithi192.stdout:3/197: fsync d2/d4/f14 0 2024-04-24T22:45:21.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/258: mknod d0/d2/d3/c50 0 2024-04-24T22:45:21.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/198: dread d2/d4/f3e [0,4194304] 0 2024-04-24T22:45:21.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/199: chown d2/d4/f14 91 1 2024-04-24T22:45:21.302 INFO:tasks.workunit.client.0.smithi063.stdout:6/240: truncate db/ff 3234276 0 2024-04-24T22:45:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:20 smithi063 ceph-mon[26912]: pgmap v8: 65 pgs: 65 active+clean; 542 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 46 MiB/s wr, 138 op/s 2024-04-24T22:45:21.304 INFO:tasks.workunit.client.1.smithi192.stdout:2/268: truncate d4/d9/d37/f3b 695938 0 2024-04-24T22:45:21.307 INFO:tasks.workunit.client.1.smithi192.stdout:4/167: creat d2/db/f3d x:0 0 0 2024-04-24T22:45:21.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/240: symlink d6/dd/d20/d24/l5a 0 2024-04-24T22:45:21.310 INFO:tasks.workunit.client.0.smithi063.stdout:7/208: unlink d0/l16 0 2024-04-24T22:45:21.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/258: rmdir d3/d2d 39 2024-04-24T22:45:21.318 INFO:tasks.workunit.client.1.smithi192.stdout:3/200: symlink d2/d4/d12/d39/d3a/l45 0 2024-04-24T22:45:21.319 INFO:tasks.workunit.client.1.smithi192.stdout:3/201: write d2/f33 [44954,84082] 0 2024-04-24T22:45:21.320 INFO:tasks.workunit.client.0.smithi063.stdout:2/259: mkdir d0/d2/d51 0 2024-04-24T22:45:21.321 INFO:tasks.workunit.client.1.smithi192.stdout:2/269: readlink d4/d9/d10/l32 0 2024-04-24T22:45:21.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/168: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:21.322 INFO:tasks.workunit.client.0.smithi063.stdout:3/258: symlink da/d18/l59 0 2024-04-24T22:45:21.325 INFO:tasks.workunit.client.1.smithi192.stdout:3/202: fdatasync d2/d4/d7/f9 0 2024-04-24T22:45:21.329 INFO:tasks.workunit.client.1.smithi192.stdout:7/189: write d2/db/f20 [4464026,85551] 0 2024-04-24T22:45:21.330 INFO:tasks.workunit.client.1.smithi192.stdout:2/270: write d4/d9/d10/f1e [1291582,77500] 0 2024-04-24T22:45:21.335 INFO:tasks.workunit.client.1.smithi192.stdout:6/192: write d0/d8/f17 [4243172,106124] 0 2024-04-24T22:45:21.337 INFO:tasks.workunit.client.1.smithi192.stdout:6/193: dread - d0/d8/d3d/d3e/d40/f47 zero size 2024-04-24T22:45:21.337 INFO:tasks.workunit.client.0.smithi063.stdout:8/264: getdents d1/d11/d49/d54 0 2024-04-24T22:45:21.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/194: rmdir d9/d15/d2d/d31/d2f/d35 0 2024-04-24T22:45:21.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/172: rmdir d3/dd 39 2024-04-24T22:45:21.345 INFO:tasks.workunit.client.1.smithi192.stdout:7/190: dread d2/db/f20 [0,4194304] 0 2024-04-24T22:45:21.346 INFO:tasks.workunit.client.0.smithi063.stdout:2/260: unlink d0/f33 0 2024-04-24T22:45:21.347 INFO:tasks.workunit.client.1.smithi192.stdout:4/169: rmdir d2/d9/d1e/d28 39 2024-04-24T22:45:21.350 INFO:tasks.workunit.client.0.smithi063.stdout:3/259: creat da/d18/d47/d4d/f5a x:0 0 0 2024-04-24T22:45:21.353 INFO:tasks.workunit.client.0.smithi063.stdout:5/256: getdents d5/d11/d23/d2d 0 2024-04-24T22:45:21.355 INFO:tasks.workunit.client.0.smithi063.stdout:4/240: write d2/d8/f2c [3793250,61752] 0 2024-04-24T22:45:21.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/257: dread d5/d11/d18/f3a [0,4194304] 0 2024-04-24T22:45:21.359 INFO:tasks.workunit.client.0.smithi063.stdout:5/258: dread - d5/d24/d2c/f55 zero size 2024-04-24T22:45:21.360 INFO:tasks.workunit.client.0.smithi063.stdout:5/259: dread - d5/d24/f32 zero size 2024-04-24T22:45:21.367 INFO:tasks.workunit.client.1.smithi192.stdout:2/271: symlink d4/d9/d10/l4f 0 2024-04-24T22:45:21.369 INFO:tasks.workunit.client.0.smithi063.stdout:2/261: creat d0/d2/d35/f52 x:0 0 0 2024-04-24T22:45:21.371 INFO:tasks.workunit.client.0.smithi063.stdout:3/260: mknod da/d18/d47/d4d/c5b 0 2024-04-24T22:45:21.377 INFO:tasks.workunit.client.0.smithi063.stdout:4/241: symlink d2/d19/l45 0 2024-04-24T22:45:21.378 INFO:tasks.workunit.client.0.smithi063.stdout:4/242: write d2/d19/f1f [38049,29327] 0 2024-04-24T22:45:21.379 INFO:tasks.workunit.client.0.smithi063.stdout:1/259: write d3/da/f1b [2677020,112652] 0 2024-04-24T22:45:21.380 INFO:tasks.workunit.client.0.smithi063.stdout:1/260: read d3/da/f3d [2514415,46270] 0 2024-04-24T22:45:21.381 INFO:tasks.workunit.client.1.smithi192.stdout:3/203: dwrite d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:21.382 INFO:tasks.workunit.client.0.smithi063.stdout:7/209: symlink d0/d4/de/l52 0 2024-04-24T22:45:21.383 INFO:tasks.workunit.client.1.smithi192.stdout:3/204: chown d2/d4/d12/d39/d3a 18 1 2024-04-24T22:45:21.383 INFO:tasks.workunit.client.0.smithi063.stdout:7/210: chown d0/d4/fb 69778 1 2024-04-24T22:45:21.384 INFO:tasks.workunit.client.0.smithi063.stdout:9/242: getdents d6/d8/db/d1c 0 2024-04-24T22:45:21.387 INFO:tasks.workunit.client.1.smithi192.stdout:7/191: write f0 [2367398,64898] 0 2024-04-24T22:45:21.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/192: write d2/db/d2a/f2b [892714,24203] 0 2024-04-24T22:45:21.389 INFO:tasks.workunit.client.1.smithi192.stdout:7/193: fdatasync d2/fa 0 2024-04-24T22:45:21.390 INFO:tasks.workunit.client.0.smithi063.stdout:5/260: mknod d5/d24/d36/c5e 0 2024-04-24T22:45:21.392 INFO:tasks.workunit.client.0.smithi063.stdout:6/241: dwrite db/f1a [0,4194304] 0 2024-04-24T22:45:21.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/195: truncate d9/fa 4148726 0 2024-04-24T22:45:21.395 INFO:tasks.workunit.client.1.smithi192.stdout:9/173: creat d3/f3f x:0 0 0 2024-04-24T22:45:21.395 INFO:tasks.workunit.client.1.smithi192.stdout:1/196: chown d9/d15/d2d/d31/d2f 15513 1 2024-04-24T22:45:21.396 INFO:tasks.workunit.client.0.smithi063.stdout:2/262: rmdir d0/d2/d3/d7 39 2024-04-24T22:45:21.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/205: fdatasync d2/d4/d7/f2c 0 2024-04-24T22:45:21.398 INFO:tasks.workunit.client.0.smithi063.stdout:9/243: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:21.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/241: dwrite d6/da/f14 [0,4194304] 0 2024-04-24T22:45:21.401 INFO:tasks.workunit.client.0.smithi063.stdout:9/244: chown d6/d2f/l39 0 1 2024-04-24T22:45:21.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/211: dwrite d0/f3d [0,4194304] 0 2024-04-24T22:45:21.401 INFO:tasks.workunit.client.1.smithi192.stdout:4/170: link d2/db/d10/l35 d2/db/d10/l3e 0 2024-04-24T22:45:21.403 INFO:tasks.workunit.client.0.smithi063.stdout:0/242: fsync d6/da/d2b/f40 0 2024-04-24T22:45:21.403 INFO:tasks.workunit.client.0.smithi063.stdout:8/265: truncate d1/d22/f31 461066 0 2024-04-24T22:45:21.404 INFO:tasks.workunit.client.0.smithi063.stdout:0/243: chown d6/dd/f3f 510 1 2024-04-24T22:45:21.405 INFO:tasks.workunit.client.0.smithi063.stdout:8/266: fdatasync d1/d11/d32/d38/f42 0 2024-04-24T22:45:21.407 INFO:tasks.workunit.client.0.smithi063.stdout:0/244: chown d6/da/lf 408162 1 2024-04-24T22:45:21.410 INFO:tasks.workunit.client.0.smithi063.stdout:7/212: dwrite d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:21.451 INFO:tasks.workunit.client.0.smithi063.stdout:1/261: rmdir d3 39 2024-04-24T22:45:21.451 INFO:tasks.workunit.client.1.smithi192.stdout:7/194: mkdir d2/db/d21/d37 0 2024-04-24T22:45:21.453 INFO:tasks.workunit.client.1.smithi192.stdout:2/272: mknod d4/d9/d10/c50 0 2024-04-24T22:45:21.454 INFO:tasks.workunit.client.1.smithi192.stdout:2/273: fdatasync d4/d9/d37/f40 0 2024-04-24T22:45:21.455 INFO:tasks.workunit.client.1.smithi192.stdout:6/194: link d0/d8/d18/d2e/c43 d0/d8/d3d/d3e/c48 0 2024-04-24T22:45:21.460 INFO:tasks.workunit.client.1.smithi192.stdout:1/197: read d9/f14 [891501,96576] 0 2024-04-24T22:45:21.461 INFO:tasks.workunit.client.1.smithi192.stdout:3/206: rmdir d2/d4/d12/d1a/d1d 39 2024-04-24T22:45:21.462 INFO:tasks.workunit.client.1.smithi192.stdout:3/207: write d2/f33 [817059,115534] 0 2024-04-24T22:45:21.464 INFO:tasks.workunit.client.1.smithi192.stdout:4/171: creat d2/d9/f3f x:0 0 0 2024-04-24T22:45:21.465 INFO:tasks.workunit.client.0.smithi063.stdout:6/242: mkdir db/d1d/d20/d3e/d4b 0 2024-04-24T22:45:21.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/261: link da/d18/d3a/f3f da/d48/f5c 0 2024-04-24T22:45:21.471 INFO:tasks.workunit.client.1.smithi192.stdout:2/274: creat d4/d29/f51 x:0 0 0 2024-04-24T22:45:21.472 INFO:tasks.workunit.client.1.smithi192.stdout:6/195: truncate d0/d8/fe 2028664 0 2024-04-24T22:45:21.474 INFO:tasks.workunit.client.0.smithi063.stdout:8/267: mkdir d1/d2/d3/d5c 0 2024-04-24T22:45:21.474 INFO:tasks.workunit.client.1.smithi192.stdout:7/195: dread d2/db/d25/f27 [4194304,4194304] 0 2024-04-24T22:45:21.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/245: write d6/dd/d20/d21/d25/f55 [483629,83756] 0 2024-04-24T22:45:21.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/243: readlink d2/d8/l9 0 2024-04-24T22:45:21.480 INFO:tasks.workunit.client.1.smithi192.stdout:4/172: mknod d2/d9/d25/c40 0 2024-04-24T22:45:21.483 INFO:tasks.workunit.client.1.smithi192.stdout:6/196: readlink d0/l15 0 2024-04-24T22:45:21.484 INFO:tasks.workunit.client.1.smithi192.stdout:1/198: mknod d9/dd/c45 0 2024-04-24T22:45:21.485 INFO:tasks.workunit.client.1.smithi192.stdout:6/197: dread - d0/d8/d29/f3b zero size 2024-04-24T22:45:21.485 INFO:tasks.workunit.client.0.smithi063.stdout:6/243: creat db/d1d/f4c x:0 0 0 2024-04-24T22:45:21.486 INFO:tasks.workunit.client.1.smithi192.stdout:4/173: dwrite d2/f1f [0,4194304] 0 2024-04-24T22:45:21.487 INFO:tasks.workunit.client.1.smithi192.stdout:3/208: symlink d2/d4/d12/l46 0 2024-04-24T22:45:21.488 INFO:tasks.workunit.client.1.smithi192.stdout:4/174: write d2/d9/d1e/f37 [1005097,4225] 0 2024-04-24T22:45:21.490 INFO:tasks.workunit.client.1.smithi192.stdout:3/209: dread d2/d4/f14 [0,4194304] 0 2024-04-24T22:45:21.490 INFO:tasks.workunit.client.1.smithi192.stdout:1/199: dread d9/d15/f1f [0,4194304] 0 2024-04-24T22:45:21.492 INFO:tasks.workunit.client.1.smithi192.stdout:7/196: creat d2/db/d36/f38 x:0 0 0 2024-04-24T22:45:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/246: rmdir d6/dd/d20/d24 39 2024-04-24T22:45:21.496 INFO:tasks.workunit.client.1.smithi192.stdout:6/198: creat d0/d8/d3d/f49 x:0 0 0 2024-04-24T22:45:21.498 INFO:tasks.workunit.client.1.smithi192.stdout:2/275: creat d4/d9/f52 x:0 0 0 2024-04-24T22:45:21.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/261: rename d5/fd to d5/f5f 0 2024-04-24T22:45:21.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/262: fdatasync d5/f10 0 2024-04-24T22:45:21.501 INFO:tasks.workunit.client.0.smithi063.stdout:9/245: truncate d6/d8/d1f/d20/d22/f25 3227921 0 2024-04-24T22:45:21.502 INFO:tasks.workunit.client.1.smithi192.stdout:4/175: mknod d2/d9/d25/d2e/d31/c41 0 2024-04-24T22:45:21.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/246: dread d6/d2f/f35 [0,4194304] 0 2024-04-24T22:45:21.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/247: write d6/f30 [1732131,112575] 0 2024-04-24T22:45:21.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/244: creat db/d1d/d2d/f4d x:0 0 0 2024-04-24T22:45:21.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/245: write db/d1d/d2d/f4d [522333,72306] 0 2024-04-24T22:45:21.507 INFO:tasks.workunit.client.1.smithi192.stdout:1/200: unlink d9/d15/d2d/d31/f39 0 2024-04-24T22:45:21.510 INFO:tasks.workunit.client.0.smithi063.stdout:4/244: dwrite f1 [0,4194304] 0 2024-04-24T22:45:21.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/268: link d1/d11/f47 d1/d21/f5d 0 2024-04-24T22:45:21.511 INFO:tasks.workunit.client.0.smithi063.stdout:8/269: fdatasync d1/f58 0 2024-04-24T22:45:21.511 INFO:tasks.workunit.client.1.smithi192.stdout:6/199: unlink d0/d8/d18/d2e/f41 0 2024-04-24T22:45:21.517 INFO:tasks.workunit.client.1.smithi192.stdout:4/176: rmdir d2/d9/d25/d2e 39 2024-04-24T22:45:21.519 INFO:tasks.workunit.client.1.smithi192.stdout:6/200: dwrite d0/d8/f2a [0,4194304] 0 2024-04-24T22:45:21.521 INFO:tasks.workunit.client.1.smithi192.stdout:2/276: dwrite d4/d9/f2c [0,4194304] 0 2024-04-24T22:45:21.523 INFO:tasks.workunit.client.0.smithi063.stdout:3/262: truncate da/f31 451596 0 2024-04-24T22:45:21.525 INFO:tasks.workunit.client.1.smithi192.stdout:0/294: sync 2024-04-24T22:45:21.525 INFO:tasks.workunit.client.1.smithi192.stdout:5/188: sync 2024-04-24T22:45:21.525 INFO:tasks.workunit.client.1.smithi192.stdout:8/193: sync 2024-04-24T22:45:21.526 INFO:tasks.workunit.client.1.smithi192.stdout:5/189: fsync d0/d1b/f25 0 2024-04-24T22:45:21.527 INFO:tasks.workunit.client.1.smithi192.stdout:7/197: getdents d2/db/d21/d37 0 2024-04-24T22:45:21.527 INFO:tasks.workunit.client.0.smithi063.stdout:5/263: dread d5/f6 [0,4194304] 0 2024-04-24T22:45:21.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/246: creat db/d1d/f4e x:0 0 0 2024-04-24T22:45:21.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/263: link d0/d2/f1b d0/d2/d3/d7/f53 0 2024-04-24T22:45:21.534 INFO:tasks.workunit.client.1.smithi192.stdout:7/198: dwrite d2/fa [4194304,4194304] 0 2024-04-24T22:45:21.536 INFO:tasks.workunit.client.0.smithi063.stdout:4/245: fsync d2/d8/f10 0 2024-04-24T22:45:21.537 INFO:tasks.workunit.client.1.smithi192.stdout:3/210: creat d2/d4/d7/f47 x:0 0 0 2024-04-24T22:45:21.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/201: symlink d9/d15/l46 0 2024-04-24T22:45:21.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/202: fsync f3 0 2024-04-24T22:45:21.545 INFO:tasks.workunit.client.0.smithi063.stdout:3/263: creat da/d48/f5d x:0 0 0 2024-04-24T22:45:21.546 INFO:tasks.workunit.client.1.smithi192.stdout:8/194: creat db/dc/d22/f39 x:0 0 0 2024-04-24T22:45:21.546 INFO:tasks.workunit.client.1.smithi192.stdout:8/195: stat f8 0 2024-04-24T22:45:21.550 INFO:tasks.workunit.client.1.smithi192.stdout:7/199: creat d2/db/d21/f39 x:0 0 0 2024-04-24T22:45:21.554 INFO:tasks.workunit.client.1.smithi192.stdout:6/201: fsync d0/d8/d29/f30 0 2024-04-24T22:45:21.555 INFO:tasks.workunit.client.1.smithi192.stdout:6/202: chown d0/d8/d18/d2e/l37 948221761 1 2024-04-24T22:45:21.556 INFO:tasks.workunit.client.0.smithi063.stdout:9/248: rename d6/d8/d1f/d20/d22/f25 to d6/d2f/f51 0 2024-04-24T22:45:21.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/247: creat db/d1d/d20/d3e/f4f x:0 0 0 2024-04-24T22:45:21.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/248: chown db 1540441 1 2024-04-24T22:45:21.559 INFO:tasks.workunit.client.1.smithi192.stdout:0/295: symlink d3/l67 0 2024-04-24T22:45:21.560 INFO:tasks.workunit.client.0.smithi063.stdout:0/247: truncate d6/dd/d20/d21/f36 60247 0 2024-04-24T22:45:21.560 INFO:tasks.workunit.client.1.smithi192.stdout:0/296: chown d3/d27/d3c/c54 121184266 1 2024-04-24T22:45:21.560 INFO:tasks.workunit.client.0.smithi063.stdout:0/248: chown d6/dd/d20/d21/d25/d31 124386712 1 2024-04-24T22:45:21.562 INFO:tasks.workunit.client.0.smithi063.stdout:4/246: write d2/fd [6554698,104021] 0 2024-04-24T22:45:21.566 INFO:tasks.workunit.client.0.smithi063.stdout:1/262: getdents d3/da/dc/d1f/d21/d42 0 2024-04-24T22:45:21.566 INFO:tasks.workunit.client.1.smithi192.stdout:3/211: dread d2/d4/d7/f17 [0,4194304] 0 2024-04-24T22:45:21.567 INFO:tasks.workunit.client.0.smithi063.stdout:0/249: dwrite d6/da/dc/d26/d33/f51 [0,4194304] 0 2024-04-24T22:45:21.567 INFO:tasks.workunit.client.1.smithi192.stdout:3/212: readlink d2/d4/d12/d1a/d22/l1f 0 2024-04-24T22:45:21.568 INFO:tasks.workunit.client.0.smithi063.stdout:3/264: mknod da/d18/d47/c5e 0 2024-04-24T22:45:21.569 INFO:tasks.workunit.client.0.smithi063.stdout:3/265: write da/f15 [766605,8593] 0 2024-04-24T22:45:21.570 INFO:tasks.workunit.client.0.smithi063.stdout:5/264: mknod d5/d11/c60 0 2024-04-24T22:45:21.570 INFO:tasks.workunit.client.0.smithi063.stdout:5/265: readlink d5/le 0 2024-04-24T22:45:21.571 INFO:tasks.workunit.client.1.smithi192.stdout:4/177: dwrite d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:21.571 INFO:tasks.workunit.client.0.smithi063.stdout:3/266: chown da/d18/d47/c5e 1273433 1 2024-04-24T22:45:21.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/267: read - da/d18/f58 zero size 2024-04-24T22:45:21.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/250: dwrite d6/da/f14 [0,4194304] 0 2024-04-24T22:45:21.579 INFO:tasks.workunit.client.0.smithi063.stdout:8/270: write d1/d22/f31 [851999,98988] 0 2024-04-24T22:45:21.580 INFO:tasks.workunit.client.1.smithi192.stdout:3/213: mknod d2/d2d/c48 0 2024-04-24T22:45:21.580 INFO:tasks.workunit.client.0.smithi063.stdout:8/271: write d1/d2/d3/f14 [913281,70846] 0 2024-04-24T22:45:21.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/272: dread - d1/d21/f3a zero size 2024-04-24T22:45:21.582 INFO:tasks.workunit.client.1.smithi192.stdout:3/214: write d2/d4/d7/f47 [37326,120592] 0 2024-04-24T22:45:21.583 INFO:tasks.workunit.client.0.smithi063.stdout:2/264: truncate d0/d2/d3/d7/f19 623116 0 2024-04-24T22:45:21.583 INFO:tasks.workunit.client.0.smithi063.stdout:2/265: chown d0/d2/d51 5989 1 2024-04-24T22:45:21.584 INFO:tasks.workunit.client.1.smithi192.stdout:2/277: creat d4/d9/d10/f53 x:0 0 0 2024-04-24T22:45:21.585 INFO:tasks.workunit.client.1.smithi192.stdout:3/215: truncate d2/d4/d12/d1a/d22/f32 332055 0 2024-04-24T22:45:21.588 INFO:tasks.workunit.client.1.smithi192.stdout:7/200: write d2/db/d2a/f2e [1858331,7077] 0 2024-04-24T22:45:21.593 INFO:tasks.workunit.client.0.smithi063.stdout:1/263: creat d3/da/dc/d1f/d21/d42/f58 x:0 0 0 2024-04-24T22:45:21.594 INFO:tasks.workunit.client.1.smithi192.stdout:3/216: dread d2/f13 [0,4194304] 0 2024-04-24T22:45:21.594 INFO:tasks.workunit.client.1.smithi192.stdout:3/217: fsync d2/f33 0 2024-04-24T22:45:21.601 INFO:tasks.workunit.client.0.smithi063.stdout:5/266: symlink d5/d11/d33/l61 0 2024-04-24T22:45:21.603 INFO:tasks.workunit.client.1.smithi192.stdout:4/178: mknod d2/d9/d1e/d28/d2f/c42 0 2024-04-24T22:45:21.603 INFO:tasks.workunit.client.1.smithi192.stdout:4/179: dread - d2/d9/d1e/f24 zero size 2024-04-24T22:45:21.603 INFO:tasks.workunit.client.0.smithi063.stdout:3/268: rmdir da/d48 39 2024-04-24T22:45:21.605 INFO:tasks.workunit.client.0.smithi063.stdout:0/251: stat d6/da/dc/f1e 0 2024-04-24T22:45:21.605 INFO:tasks.workunit.client.0.smithi063.stdout:9/249: truncate d6/fd 5134953 0 2024-04-24T22:45:21.606 INFO:tasks.workunit.client.0.smithi063.stdout:9/250: readlink d6/d8/db/d1c/l38 0 2024-04-24T22:45:21.607 INFO:tasks.workunit.client.1.smithi192.stdout:6/203: getdents d0/d8 0 2024-04-24T22:45:21.609 INFO:tasks.workunit.client.1.smithi192.stdout:3/218: write d2/d4/d12/d1a/d1d/f25 [2667620,108395] 0 2024-04-24T22:45:21.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/273: mknod d1/d21/c5e 0 2024-04-24T22:45:21.611 INFO:tasks.workunit.client.1.smithi192.stdout:2/278: rename d4/d9/d10/c14 to d4/d29/c54 0 2024-04-24T22:45:21.613 INFO:tasks.workunit.client.0.smithi063.stdout:2/266: rmdir d0/d2/d3/d48 39 2024-04-24T22:45:21.615 INFO:tasks.workunit.client.0.smithi063.stdout:4/247: symlink d2/d8/d36/l46 0 2024-04-24T22:45:21.616 INFO:tasks.workunit.client.1.smithi192.stdout:0/297: getdents d3/d27/d28/d2e/d4a 0 2024-04-24T22:45:21.617 INFO:tasks.workunit.client.1.smithi192.stdout:4/180: fsync d2/f3 0 2024-04-24T22:45:21.622 INFO:tasks.workunit.client.0.smithi063.stdout:0/252: dread d6/f7 [4194304,4194304] 0 2024-04-24T22:45:21.624 INFO:tasks.workunit.client.0.smithi063.stdout:3/269: mknod da/d18/d47/c5f 0 2024-04-24T22:45:21.624 INFO:tasks.workunit.client.0.smithi063.stdout:3/270: write da/f3d [361923,27484] 0 2024-04-24T22:45:21.627 INFO:tasks.workunit.client.0.smithi063.stdout:6/249: creat db/d1d/d20/f50 x:0 0 0 2024-04-24T22:45:21.628 INFO:tasks.workunit.client.0.smithi063.stdout:8/274: rmdir d1/d2/d17 39 2024-04-24T22:45:21.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/275: truncate d1/d2/d3/f56 994525 0 2024-04-24T22:45:21.630 INFO:tasks.workunit.client.0.smithi063.stdout:2/267: rename d0/d2/d3 to d0/d2/d54 0 2024-04-24T22:45:21.630 INFO:tasks.workunit.client.0.smithi063.stdout:2/268: readlink d0/d2/d39/l4c 0 2024-04-24T22:45:21.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/264: symlink d3/d2d/l59 0 2024-04-24T22:45:21.633 INFO:tasks.workunit.client.0.smithi063.stdout:0/253: truncate d6/da/dc/f1e 1213747 0 2024-04-24T22:45:21.638 INFO:tasks.workunit.client.0.smithi063.stdout:6/250: rename db/d1d/d2d/f33 to db/d1d/d2d/d48/f51 0 2024-04-24T22:45:21.638 INFO:tasks.workunit.client.0.smithi063.stdout:5/267: write f0 [2481971,108630] 0 2024-04-24T22:45:21.639 INFO:tasks.workunit.client.0.smithi063.stdout:1/265: mkdir d3/da/dc/d39/d5a 0 2024-04-24T22:45:21.640 INFO:tasks.workunit.client.0.smithi063.stdout:0/254: rmdir d6/da/d2b 39 2024-04-24T22:45:21.641 INFO:tasks.workunit.client.0.smithi063.stdout:5/268: fsync d5/d24/f3d 0 2024-04-24T22:45:21.645 INFO:tasks.workunit.client.0.smithi063.stdout:5/269: fdatasync d5/d24/f51 0 2024-04-24T22:45:21.645 INFO:tasks.workunit.client.0.smithi063.stdout:4/248: dwrite d2/f26 [0,4194304] 0 2024-04-24T22:45:21.651 INFO:tasks.workunit.client.0.smithi063.stdout:3/271: dwrite da/d48/f4a [0,4194304] 0 2024-04-24T22:45:21.651 INFO:tasks.workunit.client.0.smithi063.stdout:8/276: link d1/d2/d3/lb d1/d11/d32/d38/l5f 0 2024-04-24T22:45:21.652 INFO:tasks.workunit.client.0.smithi063.stdout:8/277: write d1/d22/f2e [4558886,84507] 0 2024-04-24T22:45:21.654 INFO:tasks.workunit.client.0.smithi063.stdout:5/270: fsync d5/f14 0 2024-04-24T22:45:21.658 INFO:tasks.workunit.client.0.smithi063.stdout:3/272: chown da/f31 3 1 2024-04-24T22:45:21.660 INFO:tasks.workunit.client.0.smithi063.stdout:4/249: truncate d2/d8/f16 1077524 0 2024-04-24T22:45:21.662 INFO:tasks.workunit.client.1.smithi192.stdout:9/174: sync 2024-04-24T22:45:21.662 INFO:tasks.workunit.client.0.smithi063.stdout:1/266: dwrite d3/da/dc/d1f/f4d [0,4194304] 0 2024-04-24T22:45:21.663 INFO:tasks.workunit.client.0.smithi063.stdout:6/251: getdents db 0 2024-04-24T22:45:21.666 INFO:tasks.workunit.client.0.smithi063.stdout:2/269: dwrite d0/dd/f22 [0,4194304] 0 2024-04-24T22:45:21.667 INFO:tasks.workunit.client.0.smithi063.stdout:8/278: creat d1/d11/d49/d54/f60 x:0 0 0 2024-04-24T22:45:21.670 INFO:tasks.workunit.client.0.smithi063.stdout:1/267: truncate d3/da/f3d 5084258 0 2024-04-24T22:45:21.670 INFO:tasks.workunit.client.0.smithi063.stdout:4/250: read d2/d8/f2c [1373767,51809] 0 2024-04-24T22:45:21.670 INFO:tasks.workunit.client.0.smithi063.stdout:0/255: write d6/dd/d20/d21/f48 [72963,110731] 0 2024-04-24T22:45:21.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/204: mknod d0/d8/d18/d2e/c4a 0 2024-04-24T22:45:21.672 INFO:tasks.workunit.client.1.smithi192.stdout:2/279: dwrite d4/d9/d10/f53 [0,4194304] 0 2024-04-24T22:45:21.672 INFO:tasks.workunit.client.1.smithi192.stdout:0/298: truncate d3/d27/d3c/d3b/f65 411092 0 2024-04-24T22:45:21.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/270: read - d0/d2/d35/f52 zero size 2024-04-24T22:45:21.677 INFO:tasks.workunit.client.0.smithi063.stdout:8/279: dwrite d1/d2/d3/f56 [0,4194304] 0 2024-04-24T22:45:21.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/251: truncate d2/d8/fe 3102184 0 2024-04-24T22:45:21.678 INFO:tasks.workunit.client.0.smithi063.stdout:4/252: fsync d2/d8/fe 0 2024-04-24T22:45:21.679 INFO:tasks.workunit.client.0.smithi063.stdout:8/280: fsync d1/d11/d49/d54/f60 0 2024-04-24T22:45:21.684 INFO:tasks.workunit.client.0.smithi063.stdout:5/271: truncate f0 2448574 0 2024-04-24T22:45:21.685 INFO:tasks.workunit.client.0.smithi063.stdout:5/272: truncate d5/d24/d49/d3b/f45 590687 0 2024-04-24T22:45:21.686 INFO:tasks.workunit.client.0.smithi063.stdout:3/273: write da/f24 [448814,24437] 0 2024-04-24T22:45:21.687 INFO:tasks.workunit.client.0.smithi063.stdout:3/274: readlink da/d18/d47/l51 0 2024-04-24T22:45:21.689 INFO:tasks.workunit.client.0.smithi063.stdout:0/256: mkdir d6/d5b 0 2024-04-24T22:45:21.690 INFO:tasks.workunit.client.0.smithi063.stdout:2/271: truncate d0/d2/d27/f37 1764140 0 2024-04-24T22:45:21.691 INFO:tasks.workunit.client.0.smithi063.stdout:2/272: write d0/d2/d35/f52 [905034,63147] 0 2024-04-24T22:45:21.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/253: fsync d2/d8/f11 0 2024-04-24T22:45:21.696 INFO:tasks.workunit.client.0.smithi063.stdout:5/273: mkdir d5/d11/d23/d2d/d62 0 2024-04-24T22:45:21.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/275: chown da/d18/l35 12478744 1 2024-04-24T22:45:21.698 INFO:tasks.workunit.client.0.smithi063.stdout:1/268: creat d3/da/dc/d39/d5a/f5b x:0 0 0 2024-04-24T22:45:21.698 INFO:tasks.workunit.client.0.smithi063.stdout:3/276: stat da 0 2024-04-24T22:45:21.700 INFO:tasks.workunit.client.0.smithi063.stdout:2/273: fdatasync d0/d2/d54/d7/f53 0 2024-04-24T22:45:21.701 INFO:tasks.workunit.client.0.smithi063.stdout:8/281: dread d1/f1f [0,4194304] 0 2024-04-24T22:45:21.703 INFO:tasks.workunit.client.0.smithi063.stdout:4/254: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:21.704 INFO:tasks.workunit.client.0.smithi063.stdout:6/252: getdents db/d1d/d20/d3f 0 2024-04-24T22:45:21.706 INFO:tasks.workunit.client.0.smithi063.stdout:8/282: write d1/f5 [1663370,19728] 0 2024-04-24T22:45:21.711 INFO:tasks.workunit.client.0.smithi063.stdout:5/274: dwrite d5/d24/d49/d3b/f48 [0,4194304] 0 2024-04-24T22:45:21.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/255: mknod d2/d8/d36/c47 0 2024-04-24T22:45:21.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/277: creat da/f60 x:0 0 0 2024-04-24T22:45:21.713 INFO:tasks.workunit.client.0.smithi063.stdout:6/253: rename f8 to db/d1d/d2d/f52 0 2024-04-24T22:45:21.715 INFO:tasks.workunit.client.0.smithi063.stdout:5/275: creat d5/d24/d36/f63 x:0 0 0 2024-04-24T22:45:21.717 INFO:tasks.workunit.client.0.smithi063.stdout:6/254: unlink db/c30 0 2024-04-24T22:45:21.719 INFO:tasks.workunit.client.0.smithi063.stdout:2/274: dwrite d0/d2/d54/d1c/f2a [4194304,4194304] 0 2024-04-24T22:45:21.720 INFO:tasks.workunit.client.0.smithi063.stdout:1/269: write d3/f17 [31250,49479] 0 2024-04-24T22:45:21.721 INFO:tasks.workunit.client.0.smithi063.stdout:3/278: creat da/d18/f61 x:0 0 0 2024-04-24T22:45:21.722 INFO:tasks.workunit.client.0.smithi063.stdout:5/276: truncate d5/d11/d23/f4e 64382 0 2024-04-24T22:45:21.724 INFO:tasks.workunit.client.0.smithi063.stdout:5/277: stat d5/d11/f3c 0 2024-04-24T22:45:21.727 INFO:tasks.workunit.client.0.smithi063.stdout:5/278: creat d5/d1d/f64 x:0 0 0 2024-04-24T22:45:21.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/279: mkdir da/d18/d47/d4d/d62 0 2024-04-24T22:45:21.730 INFO:tasks.workunit.client.0.smithi063.stdout:6/255: rename db/c16 to db/d1d/d20/c53 0 2024-04-24T22:45:21.732 INFO:tasks.workunit.client.0.smithi063.stdout:2/275: truncate d0/d2/d54/d7/f19 843120 0 2024-04-24T22:45:21.732 INFO:tasks.workunit.client.0.smithi063.stdout:2/276: readlink d0/l4a 0 2024-04-24T22:45:21.733 INFO:tasks.workunit.client.0.smithi063.stdout:4/256: truncate d2/ff 4148321 0 2024-04-24T22:45:21.736 INFO:tasks.workunit.client.0.smithi063.stdout:3/280: write da/f21 [607529,20472] 0 2024-04-24T22:45:21.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/270: rename d3/da/dc/d39/d5a/f5b to d3/da/f5c 0 2024-04-24T22:45:21.738 INFO:tasks.workunit.client.0.smithi063.stdout:5/279: dwrite d5/d24/d2c/f50 [0,4194304] 0 2024-04-24T22:45:21.739 INFO:tasks.workunit.client.0.smithi063.stdout:2/277: mknod d0/d2/d35/c55 0 2024-04-24T22:45:21.742 INFO:tasks.workunit.client.0.smithi063.stdout:4/257: unlink d2/d8/l1e 0 2024-04-24T22:45:21.746 INFO:tasks.workunit.client.0.smithi063.stdout:5/280: write d5/f5b [496079,122460] 0 2024-04-24T22:45:21.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/281: mkdir da/d18/d3a/d63 0 2024-04-24T22:45:21.747 INFO:tasks.workunit.client.0.smithi063.stdout:2/278: write d0/dd/f22 [3063986,91667] 0 2024-04-24T22:45:21.747 INFO:tasks.workunit.client.0.smithi063.stdout:6/256: dwrite db/d1d/f4c [0,4194304] 0 2024-04-24T22:45:21.747 INFO:tasks.workunit.client.0.smithi063.stdout:8/283: dread d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:21.747 INFO:tasks.workunit.client.0.smithi063.stdout:4/258: symlink d2/d19/l48 0 2024-04-24T22:45:21.748 INFO:tasks.workunit.client.0.smithi063.stdout:1/271: truncate d3/da/dc/d1f/d21/d42/f58 1037098 0 2024-04-24T22:45:21.749 INFO:tasks.workunit.client.0.smithi063.stdout:5/281: fsync d5/d24/d36/f63 0 2024-04-24T22:45:21.754 INFO:tasks.workunit.client.0.smithi063.stdout:8/284: creat d1/d21/f61 x:0 0 0 2024-04-24T22:45:21.755 INFO:tasks.workunit.client.0.smithi063.stdout:5/282: chown d5/d24/d36/c4d 262687 1 2024-04-24T22:45:21.756 INFO:tasks.workunit.client.0.smithi063.stdout:4/259: mknod d2/d8/d40/c49 0 2024-04-24T22:45:21.757 INFO:tasks.workunit.client.0.smithi063.stdout:1/272: dread d3/da/f36 [0,4194304] 0 2024-04-24T22:45:21.758 INFO:tasks.workunit.client.0.smithi063.stdout:3/282: getdents da/d40 0 2024-04-24T22:45:21.759 INFO:tasks.workunit.client.0.smithi063.stdout:8/285: dread d1/d2/f2f [0,4194304] 0 2024-04-24T22:45:21.761 INFO:tasks.workunit.client.0.smithi063.stdout:5/283: chown d5/d24/d36/c4d 3433 1 2024-04-24T22:45:21.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/273: fdatasync d3/da/dc/d11/f46 0 2024-04-24T22:45:21.762 INFO:tasks.workunit.client.0.smithi063.stdout:4/260: mknod d2/d8/d40/c4a 0 2024-04-24T22:45:21.763 INFO:tasks.workunit.client.0.smithi063.stdout:2/279: dwrite d0/d2/f1d [0,4194304] 0 2024-04-24T22:45:21.763 INFO:tasks.workunit.client.0.smithi063.stdout:4/261: write d2/fb [2908454,95679] 0 2024-04-24T22:45:21.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/284: truncate d5/d24/d44/f46 5005385 0 2024-04-24T22:45:21.765 INFO:tasks.workunit.client.0.smithi063.stdout:2/280: chown d0/d2/d35/f52 0 1 2024-04-24T22:45:21.765 INFO:tasks.workunit.client.0.smithi063.stdout:4/262: write d2/d19/f2b [665240,7487] 0 2024-04-24T22:45:21.766 INFO:tasks.workunit.client.0.smithi063.stdout:1/274: unlink d3/c31 0 2024-04-24T22:45:21.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/263: read - d2/f2e zero size 2024-04-24T22:45:21.768 INFO:tasks.workunit.client.0.smithi063.stdout:8/286: rmdir d1/d2/d3/d5c 0 2024-04-24T22:45:21.769 INFO:tasks.workunit.client.1.smithi192.stdout:2/280: chown d4/d9/l4b 78260524 1 2024-04-24T22:45:21.769 INFO:tasks.workunit.client.0.smithi063.stdout:8/287: dread - d1/d11/d3c/f55 zero size 2024-04-24T22:45:21.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/181: rename d2/d9/d1e/d28/c2c to d2/d9/d1e/d28/c43 0 2024-04-24T22:45:21.770 INFO:tasks.workunit.client.1.smithi192.stdout:6/205: creat d0/d8/d3d/d3e/f4b x:0 0 0 2024-04-24T22:45:21.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/175: mknod d3/dd/c40 0 2024-04-24T22:45:21.770 INFO:tasks.workunit.client.1.smithi192.stdout:0/299: rmdir d3/d27/d28/d2e/d4a/d5c 39 2024-04-24T22:45:21.771 INFO:tasks.workunit.client.0.smithi063.stdout:3/283: link da/f14 da/d18/d3a/d63/f64 0 2024-04-24T22:45:21.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/264: unlink d2/d19/f1f 0 2024-04-24T22:45:21.774 INFO:tasks.workunit.client.1.smithi192.stdout:7/201: dwrite d2/db/f15 [0,4194304] 0 2024-04-24T22:45:21.775 INFO:tasks.workunit.client.1.smithi192.stdout:3/219: dread d2/d4/d7/f2c [0,4194304] 0 2024-04-24T22:45:21.777 INFO:tasks.workunit.client.1.smithi192.stdout:4/182: dread d2/d9/d25/f23 [0,4194304] 0 2024-04-24T22:45:21.778 INFO:tasks.workunit.client.0.smithi063.stdout:5/285: dread d5/f5f [0,4194304] 0 2024-04-24T22:45:21.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/275: creat d3/da/dc/d1f/d21/d49/f5d x:0 0 0 2024-04-24T22:45:21.780 INFO:tasks.workunit.client.0.smithi063.stdout:1/276: dread - d3/da/dc/f3a zero size 2024-04-24T22:45:21.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/176: rmdir d3 39 2024-04-24T22:45:21.784 INFO:tasks.workunit.client.0.smithi063.stdout:6/257: dread db/f2a [0,4194304] 0 2024-04-24T22:45:21.785 INFO:tasks.workunit.client.0.smithi063.stdout:6/258: truncate db/d1d/f49 968594 0 2024-04-24T22:45:21.785 INFO:tasks.workunit.client.1.smithi192.stdout:6/206: mkdir d0/d8/d29/d4c 0 2024-04-24T22:45:21.791 INFO:tasks.workunit.client.1.smithi192.stdout:6/207: dwrite d0/d8/f17 [0,4194304] 0 2024-04-24T22:45:21.792 INFO:tasks.workunit.client.0.smithi063.stdout:4/265: rename d2/d8/f16 to d2/d8/d36/d3a/f4b 0 2024-04-24T22:45:21.793 INFO:tasks.workunit.client.1.smithi192.stdout:7/202: chown d2/f6 5 1 2024-04-24T22:45:21.794 INFO:tasks.workunit.client.0.smithi063.stdout:5/286: mkdir d5/d24/d36/d65 0 2024-04-24T22:45:21.795 INFO:tasks.workunit.client.1.smithi192.stdout:3/220: fdatasync d2/f13 0 2024-04-24T22:45:21.798 INFO:tasks.workunit.client.0.smithi063.stdout:1/277: mkdir d3/da/dc/d11/d5e 0 2024-04-24T22:45:21.798 INFO:tasks.workunit.client.1.smithi192.stdout:4/183: mkdir d2/d9/d44 0 2024-04-24T22:45:21.801 INFO:tasks.workunit.client.1.smithi192.stdout:2/281: dread d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:21.802 INFO:tasks.workunit.client.0.smithi063.stdout:3/284: creat da/d18/d47/d4d/d62/f65 x:0 0 0 2024-04-24T22:45:21.810 INFO:tasks.workunit.client.0.smithi063.stdout:7/213: sync 2024-04-24T22:45:21.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/287: creat d5/d11/d23/f66 x:0 0 0 2024-04-24T22:45:21.810 INFO:tasks.workunit.client.0.smithi063.stdout:7/214: fdatasync d0/d4/f19 0 2024-04-24T22:45:21.811 INFO:tasks.workunit.client.1.smithi192.stdout:7/203: dwrite d2/db/d25/f27 [4194304,4194304] 0 2024-04-24T22:45:21.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/288: creat d1/f62 x:0 0 0 2024-04-24T22:45:21.813 INFO:tasks.workunit.client.0.smithi063.stdout:5/288: dread f1 [0,4194304] 0 2024-04-24T22:45:21.814 INFO:tasks.workunit.client.1.smithi192.stdout:0/300: creat d3/d27/f68 x:0 0 0 2024-04-24T22:45:21.816 INFO:tasks.workunit.client.0.smithi063.stdout:1/278: unlink d3/da/dc/d1f/d21/l3b 0 2024-04-24T22:45:21.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/208: mkdir d0/d8/d29/d4c/d4d 0 2024-04-24T22:45:21.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/209: chown d0/d8/fe 318956 1 2024-04-24T22:45:21.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/279: dread d3/da/f3d [0,4194304] 0 2024-04-24T22:45:21.819 INFO:tasks.workunit.client.0.smithi063.stdout:6/259: getdents db/d1d/d20/d3e/d4b 0 2024-04-24T22:45:21.820 INFO:tasks.workunit.client.0.smithi063.stdout:6/260: write db/d1d/d20/d3e/f4f [615217,4354] 0 2024-04-24T22:45:21.823 INFO:tasks.workunit.client.1.smithi192.stdout:7/204: mkdir d2/db/d3a 0 2024-04-24T22:45:21.824 INFO:tasks.workunit.client.1.smithi192.stdout:7/205: readlink d2/db/d21/l35 0 2024-04-24T22:45:21.825 INFO:tasks.workunit.client.1.smithi192.stdout:0/301: mknod d3/d40/c69 0 2024-04-24T22:45:21.826 INFO:tasks.workunit.client.1.smithi192.stdout:0/302: fdatasync d3/f10 0 2024-04-24T22:45:21.826 INFO:tasks.workunit.client.0.smithi063.stdout:8/289: creat d1/d11/f63 x:0 0 0 2024-04-24T22:45:21.828 INFO:tasks.workunit.client.0.smithi063.stdout:8/290: write d1/d2/fe [1252493,6563] 0 2024-04-24T22:45:21.828 INFO:tasks.workunit.client.1.smithi192.stdout:7/206: dread d2/db/f15 [0,4194304] 0 2024-04-24T22:45:21.829 INFO:tasks.workunit.client.0.smithi063.stdout:8/291: stat d1/d11/l16 0 2024-04-24T22:45:21.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/289: mkdir d5/d11/d33/d67 0 2024-04-24T22:45:21.833 INFO:tasks.workunit.client.1.smithi192.stdout:3/221: creat d2/d4/d12/f49 x:0 0 0 2024-04-24T22:45:21.834 INFO:tasks.workunit.client.0.smithi063.stdout:2/281: getdents d0/d2/d54/d48 0 2024-04-24T22:45:21.837 INFO:tasks.workunit.client.1.smithi192.stdout:4/184: truncate d2/f1f 1708254 0 2024-04-24T22:45:21.837 INFO:tasks.workunit.client.1.smithi192.stdout:4/185: stat d2/d9 0 2024-04-24T22:45:21.838 INFO:tasks.workunit.client.0.smithi063.stdout:3/285: symlink da/l66 0 2024-04-24T22:45:21.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/266: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:21.840 INFO:tasks.workunit.client.0.smithi063.stdout:9/251: sync 2024-04-24T22:45:21.840 INFO:tasks.workunit.client.0.smithi063.stdout:0/257: sync 2024-04-24T22:45:21.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/261: dwrite db/d1d/d2d/f42 [0,4194304] 0 2024-04-24T22:45:21.845 INFO:tasks.workunit.client.1.smithi192.stdout:7/207: symlink d2/db/d25/l3b 0 2024-04-24T22:45:21.847 INFO:tasks.workunit.client.1.smithi192.stdout:4/186: dwrite d2/d9/f2d [0,4194304] 0 2024-04-24T22:45:21.849 INFO:tasks.workunit.client.1.smithi192.stdout:6/210: getdents d0/d8/d3d/d3e/d40/d44 0 2024-04-24T22:45:21.849 INFO:tasks.workunit.client.1.smithi192.stdout:7/208: fsync d2/db/d25/f27 0 2024-04-24T22:45:21.857 INFO:tasks.workunit.client.0.smithi063.stdout:5/290: rmdir d5/d11/d23/d2d 39 2024-04-24T22:45:21.858 INFO:tasks.workunit.client.1.smithi192.stdout:2/282: dwrite d4/d9/d37/f3b [0,4194304] 0 2024-04-24T22:45:21.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/222: creat d2/d27/f4a x:0 0 0 2024-04-24T22:45:21.858 INFO:tasks.workunit.client.1.smithi192.stdout:4/187: write d2/d9/d1e/f24 [609276,57063] 0 2024-04-24T22:45:21.858 INFO:tasks.workunit.client.1.smithi192.stdout:0/303: mkdir d3/d27/d6a 0 2024-04-24T22:45:21.858 INFO:tasks.workunit.client.1.smithi192.stdout:2/283: read d4/d9/d10/f11 [2969733,73656] 0 2024-04-24T22:45:21.860 INFO:tasks.workunit.client.1.smithi192.stdout:2/284: write d4/d29/f51 [1008122,42525] 0 2024-04-24T22:45:21.862 INFO:tasks.workunit.client.1.smithi192.stdout:7/209: readlink d2/l19 0 2024-04-24T22:45:21.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/211: dwrite d0/d8/f34 [0,4194304] 0 2024-04-24T22:45:21.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/286: dwrite da/f29 [0,4194304] 0 2024-04-24T22:45:21.866 INFO:tasks.workunit.client.1.smithi192.stdout:6/212: fdatasync d0/d8/d3d/d3e/d40/f47 0 2024-04-24T22:45:21.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/287: write da/f3b [2171722,38843] 0 2024-04-24T22:45:21.871 INFO:tasks.workunit.client.1.smithi192.stdout:4/188: rmdir d2/d9/d25 39 2024-04-24T22:45:21.871 INFO:tasks.workunit.client.1.smithi192.stdout:6/213: dread d0/d8/f17 [0,4194304] 0 2024-04-24T22:45:21.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/262: fsync db/d1d/d2d/f52 0 2024-04-24T22:45:21.872 INFO:tasks.workunit.client.0.smithi063.stdout:2/282: write d0/d2/d27/f34 [262128,67927] 0 2024-04-24T22:45:21.874 INFO:tasks.workunit.client.1.smithi192.stdout:2/285: truncate d4/d9/f35 693015 0 2024-04-24T22:45:21.875 INFO:tasks.workunit.client.0.smithi063.stdout:8/292: symlink d1/d22/l64 0 2024-04-24T22:45:21.875 INFO:tasks.workunit.client.1.smithi192.stdout:7/210: creat d2/db/d2a/f3c x:0 0 0 2024-04-24T22:45:21.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/283: fdatasync d0/d2/d35/f52 0 2024-04-24T22:45:21.877 INFO:tasks.workunit.client.0.smithi063.stdout:5/291: mknod d5/d11/d33/c68 0 2024-04-24T22:45:21.878 INFO:tasks.workunit.client.0.smithi063.stdout:1/280: truncate d3/da/dc/d1f/d21/d42/f58 1035103 0 2024-04-24T22:45:21.879 INFO:tasks.workunit.client.0.smithi063.stdout:3/288: dwrite da/d48/f5d [0,4194304] 0 2024-04-24T22:45:21.880 INFO:tasks.workunit.client.0.smithi063.stdout:2/284: fdatasync d0/d2/d27/f29 0 2024-04-24T22:45:21.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/223: mkdir d2/d4/d7/d4b 0 2024-04-24T22:45:21.883 INFO:tasks.workunit.client.1.smithi192.stdout:7/211: dwrite d2/f6 [4194304,4194304] 0 2024-04-24T22:45:21.884 INFO:tasks.workunit.client.1.smithi192.stdout:4/189: chown d2/c5 1566 1 2024-04-24T22:45:21.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/252: rename d6/d2f/f51 to d6/d8/d1f/f52 0 2024-04-24T22:45:21.886 INFO:tasks.workunit.client.0.smithi063.stdout:2/285: dread d0/d2/d27/f34 [0,4194304] 0 2024-04-24T22:45:21.887 INFO:tasks.workunit.client.1.smithi192.stdout:6/214: unlink d0/d8/l39 0 2024-04-24T22:45:21.889 INFO:tasks.workunit.client.0.smithi063.stdout:2/286: dread - d0/d2/d54/d48/f4f zero size 2024-04-24T22:45:21.889 INFO:tasks.workunit.client.1.smithi192.stdout:4/190: truncate d2/d9/d1e/d28/d2f/f3c 241768 0 2024-04-24T22:45:21.889 INFO:tasks.workunit.client.1.smithi192.stdout:4/191: chown d2/db/f3d 0 1 2024-04-24T22:45:21.890 INFO:tasks.workunit.client.0.smithi063.stdout:7/215: creat d0/d26/f53 x:0 0 0 2024-04-24T22:45:21.890 INFO:tasks.workunit.client.0.smithi063.stdout:3/289: dwrite da/d18/d47/d4d/f5a [0,4194304] 0 2024-04-24T22:45:21.891 INFO:tasks.workunit.client.1.smithi192.stdout:4/192: readlink d2/db/l1b 0 2024-04-24T22:45:21.891 INFO:tasks.workunit.client.0.smithi063.stdout:2/287: write d0/d2/f47 [569272,97136] 0 2024-04-24T22:45:21.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/224: dwrite d2/d4/d7/f2c [0,4194304] 0 2024-04-24T22:45:21.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/212: rmdir d2/db 39 2024-04-24T22:45:21.898 INFO:tasks.workunit.client.0.smithi063.stdout:8/293: creat d1/d11/d32/f65 x:0 0 0 2024-04-24T22:45:21.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/304: creat d3/d27/d28/d2e/d4a/f6b x:0 0 0 2024-04-24T22:45:21.906 INFO:tasks.workunit.client.0.smithi063.stdout:1/281: creat d3/da/dc/d11/f5f x:0 0 0 2024-04-24T22:45:21.910 INFO:tasks.workunit.client.1.smithi192.stdout:0/305: rmdir d3/d27/d28/d2e/d3f 39 2024-04-24T22:45:21.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/193: stat d2/d9/d25/d2e/d31 0 2024-04-24T22:45:21.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/194: dread - d2/db/f3d zero size 2024-04-24T22:45:21.916 INFO:tasks.workunit.client.0.smithi063.stdout:6/263: creat db/d1d/d20/d3e/d47/f54 x:0 0 0 2024-04-24T22:45:21.917 INFO:tasks.workunit.client.1.smithi192.stdout:5/190: sync 2024-04-24T22:45:21.917 INFO:tasks.workunit.client.0.smithi063.stdout:6/264: fsync db/d1d/d20/f40 0 2024-04-24T22:45:21.927 INFO:tasks.workunit.client.0.smithi063.stdout:0/258: getdents d6/dd 0 2024-04-24T22:45:21.928 INFO:tasks.workunit.client.0.smithi063.stdout:0/259: write d6/dd/d20/d21/d25/f4e [158204,36867] 0 2024-04-24T22:45:21.929 INFO:tasks.workunit.client.0.smithi063.stdout:0/260: fdatasync d6/da/dc/f13 0 2024-04-24T22:45:21.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/253: mkdir d6/d8/d53 0 2024-04-24T22:45:21.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/254: chown d6/d2f/d42 102817 1 2024-04-24T22:45:21.932 INFO:tasks.workunit.client.0.smithi063.stdout:2/288: dread d0/d2/d54/d7/f13 [0,4194304] 0 2024-04-24T22:45:21.932 INFO:tasks.workunit.client.0.smithi063.stdout:9/255: chown d6/d8/l41 231 1 2024-04-24T22:45:21.932 INFO:tasks.workunit.client.1.smithi192.stdout:4/195: link d2/d9/d1e/f24 d2/d9/d1e/d28/f45 0 2024-04-24T22:45:21.933 INFO:tasks.workunit.client.1.smithi192.stdout:2/286: truncate d4/f41 380030 0 2024-04-24T22:45:21.936 INFO:tasks.workunit.client.1.smithi192.stdout:6/215: creat d0/d8/d3d/d3e/f4e x:0 0 0 2024-04-24T22:45:21.938 INFO:tasks.workunit.client.1.smithi192.stdout:3/225: dread d2/d4/d12/d1a/d22/f24 [0,4194304] 0 2024-04-24T22:45:21.938 INFO:tasks.workunit.client.0.smithi063.stdout:3/290: mknod da/d18/d3a/d63/c67 0 2024-04-24T22:45:21.938 INFO:tasks.workunit.client.0.smithi063.stdout:3/291: chown da/d48 7456846 1 2024-04-24T22:45:21.939 INFO:tasks.workunit.client.1.smithi192.stdout:4/196: rmdir d2/d9/d1e/d28 39 2024-04-24T22:45:21.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/213: truncate d2/f17 3877556 0 2024-04-24T22:45:21.942 INFO:tasks.workunit.client.0.smithi063.stdout:6/265: rmdir db/d1d/d2d 39 2024-04-24T22:45:21.943 INFO:tasks.workunit.client.0.smithi063.stdout:6/266: chown db/d1d/d20/d3e/f43 61356 1 2024-04-24T22:45:21.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/191: dread d0/f10 [0,4194304] 0 2024-04-24T22:45:21.949 INFO:tasks.workunit.client.0.smithi063.stdout:5/292: creat d5/d11/d23/f69 x:0 0 0 2024-04-24T22:45:21.950 INFO:tasks.workunit.client.1.smithi192.stdout:0/306: dwrite d3/d40/f50 [0,4194304] 0 2024-04-24T22:45:21.953 INFO:tasks.workunit.client.1.smithi192.stdout:2/287: link d4/d9/d10/f53 d4/d9/d37/f55 0 2024-04-24T22:45:21.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/216: mknod d0/d8/d18/c4f 0 2024-04-24T22:45:21.955 INFO:tasks.workunit.client.0.smithi063.stdout:5/293: dwrite d5/f10 [0,4194304] 0 2024-04-24T22:45:21.956 INFO:tasks.workunit.client.1.smithi192.stdout:3/226: symlink d2/l4c 0 2024-04-24T22:45:21.957 INFO:tasks.workunit.client.0.smithi063.stdout:5/294: read - d5/d24/d2c/f55 zero size 2024-04-24T22:45:21.957 INFO:tasks.workunit.client.0.smithi063.stdout:9/256: read d6/d8/db/f10 [860771,115456] 0 2024-04-24T22:45:21.958 INFO:tasks.workunit.client.0.smithi063.stdout:9/257: dread - d6/d8/db/d2b/f49 zero size 2024-04-24T22:45:21.958 INFO:tasks.workunit.client.0.smithi063.stdout:5/295: chown d5/d24/d44/f46 2296919 1 2024-04-24T22:45:21.960 INFO:tasks.workunit.client.1.smithi192.stdout:8/196: sync 2024-04-24T22:45:21.960 INFO:tasks.workunit.client.1.smithi192.stdout:9/177: sync 2024-04-24T22:45:21.960 INFO:tasks.workunit.client.1.smithi192.stdout:1/203: sync 2024-04-24T22:45:21.962 INFO:tasks.workunit.client.1.smithi192.stdout:8/197: truncate db/dc/d22/f39 131718 0 2024-04-24T22:45:21.964 INFO:tasks.workunit.client.1.smithi192.stdout:0/307: mkdir d3/d27/d3c/d6c 0 2024-04-24T22:45:21.966 INFO:tasks.workunit.client.1.smithi192.stdout:2/288: stat d4/c1b 0 2024-04-24T22:45:21.968 INFO:tasks.workunit.client.1.smithi192.stdout:2/289: read d4/d9/f2c [2666732,44847] 0 2024-04-24T22:45:21.976 INFO:tasks.workunit.client.1.smithi192.stdout:0/308: dwrite d3/d27/d28/d2e/d4a/f4b [0,4194304] 0 2024-04-24T22:45:21.977 INFO:tasks.workunit.client.0.smithi063.stdout:6/267: symlink db/d1d/l55 0 2024-04-24T22:45:21.979 INFO:tasks.workunit.client.1.smithi192.stdout:0/309: dread - d3/d27/d3c/f4e zero size 2024-04-24T22:45:21.979 INFO:tasks.workunit.client.1.smithi192.stdout:6/217: dwrite d0/d8/f2b [0,4194304] 0 2024-04-24T22:45:21.981 INFO:tasks.workunit.client.1.smithi192.stdout:0/310: read - d3/d27/d28/d31/f52 zero size 2024-04-24T22:45:21.981 INFO:tasks.workunit.client.0.smithi063.stdout:7/216: rename d0/d4/de/f1c to d0/d4/de/d17/d35/f54 0 2024-04-24T22:45:21.982 INFO:tasks.workunit.client.0.smithi063.stdout:8/294: link d1/d11/d3c/l40 d1/d22/l66 0 2024-04-24T22:45:21.982 INFO:tasks.workunit.client.1.smithi192.stdout:1/204: dread f3 [4194304,4194304] 0 2024-04-24T22:45:21.985 INFO:tasks.workunit.client.1.smithi192.stdout:2/290: dread d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:21.986 INFO:tasks.workunit.client.0.smithi063.stdout:0/261: link d6/da/dc/f13 d6/dd/d20/f5c 0 2024-04-24T22:45:21.986 INFO:tasks.workunit.client.0.smithi063.stdout:7/217: dwrite d0/f4f [0,4194304] 0 2024-04-24T22:45:21.986 INFO:tasks.workunit.client.1.smithi192.stdout:2/291: write d4/f19 [3273242,106567] 0 2024-04-24T22:45:21.988 INFO:tasks.workunit.client.1.smithi192.stdout:4/197: link d2/d9/l1c d2/l46 0 2024-04-24T22:45:21.988 INFO:tasks.workunit.client.1.smithi192.stdout:2/292: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:21.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/293: stat d4/l30 0 2024-04-24T22:45:22.003 INFO:tasks.workunit.client.1.smithi192.stdout:1/205: read d9/f2c [3136971,88543] 0 2024-04-24T22:45:22.004 INFO:tasks.workunit.client.1.smithi192.stdout:4/198: rmdir d2/d9/d25 39 2024-04-24T22:45:22.006 INFO:tasks.workunit.client.1.smithi192.stdout:7/214: dwrite d2/db/f20 [0,4194304] 0 2024-04-24T22:45:22.008 INFO:tasks.workunit.client.1.smithi192.stdout:9/178: truncate d3/dd/de/d17/f34 4782277 0 2024-04-24T22:45:22.013 INFO:tasks.workunit.client.1.smithi192.stdout:0/311: mkdir d3/d6d 0 2024-04-24T22:45:22.015 INFO:tasks.workunit.client.0.smithi063.stdout:9/258: mknod d6/d8/c54 0 2024-04-24T22:45:22.016 INFO:tasks.workunit.client.1.smithi192.stdout:3/227: getdents d2/d4/d12/d39/d3a 0 2024-04-24T22:45:22.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/295: truncate d1/d2/d3/f46 4318121 0 2024-04-24T22:45:22.018 INFO:tasks.workunit.client.0.smithi063.stdout:1/282: chown d3/da/dc/d1f/d21/d42/f58 78 1 2024-04-24T22:45:22.018 INFO:tasks.workunit.client.0.smithi063.stdout:3/292: truncate da/f2c 1071295 0 2024-04-24T22:45:22.019 INFO:tasks.workunit.client.0.smithi063.stdout:3/293: chown da/d18/f27 20615960 1 2024-04-24T22:45:22.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/192: dread d0/f18 [0,4194304] 0 2024-04-24T22:45:22.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/193: write d0/f31 [862900,22327] 0 2024-04-24T22:45:22.025 INFO:tasks.workunit.client.1.smithi192.stdout:5/194: chown d0/d1b 1772860 1 2024-04-24T22:45:22.025 INFO:tasks.workunit.client.1.smithi192.stdout:8/198: dwrite f9 [0,4194304] 0 2024-04-24T22:45:22.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/268: mknod db/d1d/d2d/c56 0 2024-04-24T22:45:22.026 INFO:tasks.workunit.client.0.smithi063.stdout:6/269: truncate db/d1d/d2d/f36 391228 0 2024-04-24T22:45:22.028 INFO:tasks.workunit.client.1.smithi192.stdout:7/215: creat d2/db/d36/f3d x:0 0 0 2024-04-24T22:45:22.033 INFO:tasks.workunit.client.1.smithi192.stdout:0/312: symlink d3/d27/d3c/d3b/l6e 0 2024-04-24T22:45:22.035 INFO:tasks.workunit.client.1.smithi192.stdout:0/313: chown d3/f2c 29325066 1 2024-04-24T22:45:22.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/228: symlink d2/d4/d12/d1a/d1d/l4d 0 2024-04-24T22:45:22.041 INFO:tasks.workunit.client.1.smithi192.stdout:9/179: dread d3/fa [0,4194304] 0 2024-04-24T22:45:22.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/296: rename d5/f5f to d5/d11/f6a 0 2024-04-24T22:45:22.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/297: dread - d5/d24/d36/f63 zero size 2024-04-24T22:45:22.042 INFO:tasks.workunit.client.1.smithi192.stdout:2/294: dwrite d4/d9/d10/f11 [0,4194304] 0 2024-04-24T22:45:22.042 INFO:tasks.workunit.client.1.smithi192.stdout:9/180: fsync d3/f21 0 2024-04-24T22:45:22.046 INFO:tasks.workunit.client.0.smithi063.stdout:5/298: dread d5/d11/f15 [0,4194304] 0 2024-04-24T22:45:22.046 INFO:tasks.workunit.client.1.smithi192.stdout:6/218: dwrite d0/d8/f17 [0,4194304] 0 2024-04-24T22:45:22.046 INFO:tasks.workunit.client.0.smithi063.stdout:5/299: fdatasync d5/d11/f2f 0 2024-04-24T22:45:22.049 INFO:tasks.workunit.client.0.smithi063.stdout:9/259: mknod d6/d8/d1f/d20/c55 0 2024-04-24T22:45:22.053 INFO:tasks.workunit.client.0.smithi063.stdout:1/283: write d3/f7 [3157024,129381] 0 2024-04-24T22:45:22.054 INFO:tasks.workunit.client.0.smithi063.stdout:1/284: dread - d3/d2d/f52 zero size 2024-04-24T22:45:22.060 INFO:tasks.workunit.client.1.smithi192.stdout:7/216: write d2/f1a [1240644,16169] 0 2024-04-24T22:45:22.061 INFO:tasks.workunit.client.1.smithi192.stdout:5/195: dread d0/d1b/d1c/f20 [0,4194304] 0 2024-04-24T22:45:22.061 INFO:tasks.workunit.client.1.smithi192.stdout:5/196: write d0/d1b/d1c/f33 [489395,27309] 0 2024-04-24T22:45:22.061 INFO:tasks.workunit.client.0.smithi063.stdout:7/218: mkdir d0/d26/d2e/d55 0 2024-04-24T22:45:22.062 INFO:tasks.workunit.client.1.smithi192.stdout:5/197: read - d0/d1b/f24 zero size 2024-04-24T22:45:22.064 INFO:tasks.workunit.client.1.smithi192.stdout:3/229: write d2/d4/d12/d1a/d22/f24 [1447783,128911] 0 2024-04-24T22:45:22.066 INFO:tasks.workunit.client.0.smithi063.stdout:9/260: symlink d6/d8/d1f/l56 0 2024-04-24T22:45:22.066 INFO:tasks.workunit.client.1.smithi192.stdout:9/181: creat d3/d24/f41 x:0 0 0 2024-04-24T22:45:22.067 INFO:tasks.workunit.client.0.smithi063.stdout:8/296: rmdir d1/d11/d32/d38/d50 0 2024-04-24T22:45:22.069 INFO:tasks.workunit.client.1.smithi192.stdout:8/199: dwrite db/f1d [0,4194304] 0 2024-04-24T22:45:22.069 INFO:tasks.workunit.client.1.smithi192.stdout:3/230: chown d2/d4/d12/d1a 15149673 1 2024-04-24T22:45:22.071 INFO:tasks.workunit.client.1.smithi192.stdout:7/217: dwrite d2/f1a [0,4194304] 0 2024-04-24T22:45:22.072 INFO:tasks.workunit.client.0.smithi063.stdout:3/294: creat da/d18/f68 x:0 0 0 2024-04-24T22:45:22.073 INFO:tasks.workunit.client.0.smithi063.stdout:7/219: write d0/d26/f47 [529620,73098] 0 2024-04-24T22:45:22.074 INFO:tasks.workunit.client.0.smithi063.stdout:6/270: rmdir db/d1d/d20/d3e/d4b 0 2024-04-24T22:45:22.078 INFO:tasks.workunit.client.0.smithi063.stdout:9/261: creat d6/d8/db/d1c/f57 x:0 0 0 2024-04-24T22:45:22.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/314: mknod d3/d27/d28/d2e/c6f 0 2024-04-24T22:45:22.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/262: write d6/d8/d1f/d20/f3f [86116,33786] 0 2024-04-24T22:45:22.081 INFO:tasks.workunit.client.0.smithi063.stdout:8/297: creat d1/d11/d32/d38/f67 x:0 0 0 2024-04-24T22:45:22.086 INFO:tasks.workunit.client.1.smithi192.stdout:2/295: truncate d4/d9/d37/f55 2275029 0 2024-04-24T22:45:22.087 INFO:tasks.workunit.client.0.smithi063.stdout:7/220: chown d0/d4/de/d17/f22 650066403 1 2024-04-24T22:45:22.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/300: creat d5/d24/d49/f6b x:0 0 0 2024-04-24T22:45:22.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/271: rename db/d1d/d20/d29/l32 to db/d1d/d20/d3e/d47/l57 0 2024-04-24T22:45:22.092 INFO:tasks.workunit.client.0.smithi063.stdout:6/272: write db/d1d/d2d/f4d [1126461,31874] 0 2024-04-24T22:45:22.094 INFO:tasks.workunit.client.0.smithi063.stdout:9/263: rmdir d6/d8/d1f/d20/d22/d31 39 2024-04-24T22:45:22.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/264: chown d6/d8/d1f/d20/f3f 148532659 1 2024-04-24T22:45:22.096 INFO:tasks.workunit.client.1.smithi192.stdout:8/200: symlink db/dc/l3a 0 2024-04-24T22:45:22.100 INFO:tasks.workunit.client.1.smithi192.stdout:3/231: dwrite d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:22.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/199: link d2/d9/d1e/d28/d2f/f34 d2/d9/d1e/f47 0 2024-04-24T22:45:22.105 INFO:tasks.workunit.client.0.smithi063.stdout:5/301: unlink d5/d11/d18/f41 0 2024-04-24T22:45:22.108 INFO:tasks.workunit.client.1.smithi192.stdout:1/206: getdents d9/d15/d2d 0 2024-04-24T22:45:22.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/298: dread d1/f5 [0,4194304] 0 2024-04-24T22:45:22.112 INFO:tasks.workunit.client.0.smithi063.stdout:6/273: creat db/d1c/f58 x:0 0 0 2024-04-24T22:45:22.115 INFO:tasks.workunit.client.1.smithi192.stdout:9/182: dwrite d3/d2c/f2d [0,4194304] 0 2024-04-24T22:45:22.117 INFO:tasks.workunit.client.0.smithi063.stdout:1/285: getdents d3/da/dc/d11 0 2024-04-24T22:45:22.119 INFO:tasks.workunit.client.0.smithi063.stdout:3/295: write da/d18/d3a/d63/f64 [685614,103447] 0 2024-04-24T22:45:22.119 INFO:tasks.workunit.client.1.smithi192.stdout:6/219: dwrite d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:22.121 INFO:tasks.workunit.client.1.smithi192.stdout:8/201: unlink db/dc/l37 0 2024-04-24T22:45:22.123 INFO:tasks.workunit.client.0.smithi063.stdout:5/302: creat d5/d24/d36/f6c x:0 0 0 2024-04-24T22:45:22.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/218: symlink d2/db/d3a/l3e 0 2024-04-24T22:45:22.124 INFO:tasks.workunit.client.0.smithi063.stdout:8/299: fsync d1/f5 0 2024-04-24T22:45:22.125 INFO:tasks.workunit.client.1.smithi192.stdout:3/232: readlink d2/d4/l21 0 2024-04-24T22:45:22.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/300: chown d1/f5 907 1 2024-04-24T22:45:22.125 INFO:tasks.workunit.client.1.smithi192.stdout:7/219: chown d2/db/d25/l31 0 1 2024-04-24T22:45:22.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/274: mkdir db/d1d/d2d/d59 0 2024-04-24T22:45:22.127 INFO:tasks.workunit.client.0.smithi063.stdout:8/301: write d1/d11/f63 [734656,33242] 0 2024-04-24T22:45:22.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/220: truncate d2/db/d2a/f3c 277630 0 2024-04-24T22:45:22.127 INFO:tasks.workunit.client.1.smithi192.stdout:0/315: symlink d3/d27/d3c/d6c/l70 0 2024-04-24T22:45:22.128 INFO:tasks.workunit.client.0.smithi063.stdout:8/302: chown d1/d2/f39 625556 1 2024-04-24T22:45:22.130 INFO:tasks.workunit.client.0.smithi063.stdout:1/286: unlink d3/da/dc/d1f/d21/d49/f5d 0 2024-04-24T22:45:22.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/296: symlink d4/d9/d36/l56 0 2024-04-24T22:45:22.133 INFO:tasks.workunit.client.0.smithi063.stdout:3/296: creat da/d18/d3a/f69 x:0 0 0 2024-04-24T22:45:22.134 INFO:tasks.workunit.client.1.smithi192.stdout:5/198: creat d0/d1b/d1c/d2b/f3a x:0 0 0 2024-04-24T22:45:22.134 INFO:tasks.workunit.client.1.smithi192.stdout:7/221: dwrite d2/f1a [4194304,4194304] 0 2024-04-24T22:45:22.134 INFO:tasks.workunit.client.0.smithi063.stdout:8/303: dwrite d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:22.136 INFO:tasks.workunit.client.0.smithi063.stdout:9/265: symlink d6/d8/d1f/d20/d2a/d4b/l58 0 2024-04-24T22:45:22.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/183: truncate d3/f4 2438826 0 2024-04-24T22:45:22.138 INFO:tasks.workunit.client.0.smithi063.stdout:9/266: stat d6/d8/d1f/d20 0 2024-04-24T22:45:22.139 INFO:tasks.workunit.client.0.smithi063.stdout:9/267: write d6/d8/d1f/f28 [598019,93070] 0 2024-04-24T22:45:22.140 INFO:tasks.workunit.client.1.smithi192.stdout:6/220: mknod d0/d8/d29/c50 0 2024-04-24T22:45:22.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/297: dwrite da/d18/f61 [0,4194304] 0 2024-04-24T22:45:22.142 INFO:tasks.workunit.client.0.smithi063.stdout:5/303: truncate d5/d11/f6a 1723138 0 2024-04-24T22:45:22.148 INFO:tasks.workunit.client.1.smithi192.stdout:8/202: dwrite db/dc/d29/f2b [0,4194304] 0 2024-04-24T22:45:22.148 INFO:tasks.workunit.client.1.smithi192.stdout:4/200: mknod d2/db/c48 0 2024-04-24T22:45:22.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/268: mkdir d6/d8/d4d/d4f/d59 0 2024-04-24T22:45:22.158 INFO:tasks.workunit.client.1.smithi192.stdout:8/203: dwrite db/dc/d22/f28 [0,4194304] 0 2024-04-24T22:45:22.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/297: creat d4/d9/d36/f57 x:0 0 0 2024-04-24T22:45:22.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/304: rename d1/d22/f2e to d1/f68 0 2024-04-24T22:45:22.164 INFO:tasks.workunit.client.0.smithi063.stdout:9/269: creat d6/d8/d4d/d4f/f5a x:0 0 0 2024-04-24T22:45:22.165 INFO:tasks.workunit.client.1.smithi192.stdout:8/204: dwrite f8 [0,4194304] 0 2024-04-24T22:45:22.166 INFO:tasks.workunit.client.1.smithi192.stdout:8/205: readlink db/dc/d29/l2a 0 2024-04-24T22:45:22.170 INFO:tasks.workunit.client.1.smithi192.stdout:6/221: dread d0/d8/f33 [0,4194304] 0 2024-04-24T22:45:22.172 INFO:tasks.workunit.client.1.smithi192.stdout:4/201: readlink d2/d9/l1c 0 2024-04-24T22:45:22.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/304: creat d5/f6d x:0 0 0 2024-04-24T22:45:22.177 INFO:tasks.workunit.client.0.smithi063.stdout:5/305: dread d5/d24/d49/d3b/f45 [0,4194304] 0 2024-04-24T22:45:22.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/270: creat d6/d8/f5b x:0 0 0 2024-04-24T22:45:22.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/271: stat d6 0 2024-04-24T22:45:22.178 INFO:tasks.workunit.client.0.smithi063.stdout:8/305: dread d1/f2c [0,4194304] 0 2024-04-24T22:45:22.180 INFO:tasks.workunit.client.0.smithi063.stdout:5/306: write d5/d11/d33/f3f [547107,124069] 0 2024-04-24T22:45:22.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/206: unlink db/f15 0 2024-04-24T22:45:22.180 INFO:tasks.workunit.client.0.smithi063.stdout:5/307: chown d5/d24/d2c/l34 1622266 1 2024-04-24T22:45:22.181 INFO:tasks.workunit.client.0.smithi063.stdout:9/272: rmdir d6/d8/d4d 39 2024-04-24T22:45:22.183 INFO:tasks.workunit.client.1.smithi192.stdout:4/202: mknod d2/d9/d1e/d28/c49 0 2024-04-24T22:45:22.185 INFO:tasks.workunit.client.1.smithi192.stdout:0/316: creat d3/d27/f71 x:0 0 0 2024-04-24T22:45:22.185 INFO:tasks.workunit.client.0.smithi063.stdout:8/306: creat d1/d11/d3c/f69 x:0 0 0 2024-04-24T22:45:22.189 INFO:tasks.workunit.client.1.smithi192.stdout:8/207: mkdir db/dc/d3b 0 2024-04-24T22:45:22.190 INFO:tasks.workunit.client.1.smithi192.stdout:8/208: write db/f34 [719922,73331] 0 2024-04-24T22:45:22.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/298: dread da/f1a [0,4194304] 0 2024-04-24T22:45:22.190 INFO:tasks.workunit.client.0.smithi063.stdout:5/308: symlink d5/d11/d23/d2d/l6e 0 2024-04-24T22:45:22.191 INFO:tasks.workunit.client.1.smithi192.stdout:3/233: getdents d2/d4/d12/d1a 0 2024-04-24T22:45:22.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/273: creat d6/d8/d4d/f5c x:0 0 0 2024-04-24T22:45:22.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/274: stat d6/ce 0 2024-04-24T22:45:22.194 INFO:tasks.workunit.client.1.smithi192.stdout:8/209: dread db/f34 [0,4194304] 0 2024-04-24T22:45:22.196 INFO:tasks.workunit.client.0.smithi063.stdout:8/307: dread d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:22.197 INFO:tasks.workunit.client.1.smithi192.stdout:7/222: link d2/db/d3a/l3e d2/l3f 0 2024-04-24T22:45:22.199 INFO:tasks.workunit.client.0.smithi063.stdout:5/309: creat d5/f6f x:0 0 0 2024-04-24T22:45:22.199 INFO:tasks.workunit.client.1.smithi192.stdout:3/234: mkdir d2/d4/d4e 0 2024-04-24T22:45:22.200 INFO:tasks.workunit.client.0.smithi063.stdout:5/310: write d5/f6f [646702,44399] 0 2024-04-24T22:45:22.200 INFO:tasks.workunit.client.0.smithi063.stdout:8/308: dread d1/d22/f31 [0,4194304] 0 2024-04-24T22:45:22.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/210: creat db/dc/f3c x:0 0 0 2024-04-24T22:45:22.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/211: fsync f6 0 2024-04-24T22:45:22.202 INFO:tasks.workunit.client.1.smithi192.stdout:2/298: getdents d4/d9 0 2024-04-24T22:45:22.204 INFO:tasks.workunit.client.1.smithi192.stdout:2/299: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:22.206 INFO:tasks.workunit.client.1.smithi192.stdout:4/203: rmdir d2/d9/d44 0 2024-04-24T22:45:22.209 INFO:tasks.workunit.client.1.smithi192.stdout:7/223: dwrite f1 [0,4194304] 0 2024-04-24T22:45:22.209 INFO:tasks.workunit.client.1.smithi192.stdout:0/317: creat d3/d27/d28/d2e/d4a/f72 x:0 0 0 2024-04-24T22:45:22.211 INFO:tasks.workunit.client.1.smithi192.stdout:2/300: unlink d4/f19 0 2024-04-24T22:45:22.212 INFO:tasks.workunit.client.1.smithi192.stdout:3/235: mkdir d2/d4/d7/d4f 0 2024-04-24T22:45:22.213 INFO:tasks.workunit.client.1.smithi192.stdout:3/236: dread d2/d4/d7/f47 [0,4194304] 0 2024-04-24T22:45:22.214 INFO:tasks.workunit.client.1.smithi192.stdout:3/237: fsync d2/d4/d12/d39/f44 0 2024-04-24T22:45:22.215 INFO:tasks.workunit.client.1.smithi192.stdout:3/238: chown d2/d4/d12/d39/f44 117201426 1 2024-04-24T22:45:22.216 INFO:tasks.workunit.client.1.smithi192.stdout:3/239: read d2/f33 [908441,104699] 0 2024-04-24T22:45:22.216 INFO:tasks.workunit.client.1.smithi192.stdout:7/224: unlink d2/db/c34 0 2024-04-24T22:45:22.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/204: creat d2/d9/d25/d2e/d31/f4a x:0 0 0 2024-04-24T22:45:22.220 INFO:tasks.workunit.client.1.smithi192.stdout:8/212: creat db/f3d x:0 0 0 2024-04-24T22:45:22.221 INFO:tasks.workunit.client.1.smithi192.stdout:4/205: chown d2/d9/l2a 125135618 1 2024-04-24T22:45:22.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/225: unlink d2/db/c33 0 2024-04-24T22:45:22.225 INFO:tasks.workunit.client.1.smithi192.stdout:3/240: creat d2/d4/d7/d4b/f50 x:0 0 0 2024-04-24T22:45:22.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/318: dread d3/f47 [0,4194304] 0 2024-04-24T22:45:22.228 INFO:tasks.workunit.client.1.smithi192.stdout:4/206: symlink d2/d9/l4b 0 2024-04-24T22:45:22.229 INFO:tasks.workunit.client.1.smithi192.stdout:3/241: write d2/d4/d12/d1a/d1d/f25 [1679837,106662] 0 2024-04-24T22:45:22.230 INFO:tasks.workunit.client.1.smithi192.stdout:0/319: read d3/f1c [120647,34073] 0 2024-04-24T22:45:22.230 INFO:tasks.workunit.client.1.smithi192.stdout:8/213: rmdir db/dc/d3b 0 2024-04-24T22:45:22.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/207: write d2/d9/d1e/d28/d2f/f3c [857736,108851] 0 2024-04-24T22:45:22.233 INFO:tasks.workunit.client.1.smithi192.stdout:7/226: dwrite d2/db/d2a/f3c [0,4194304] 0 2024-04-24T22:45:22.234 INFO:tasks.workunit.client.1.smithi192.stdout:3/242: write d2/d4/d12/f42 [171471,39324] 0 2024-04-24T22:45:22.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/214: mknod db/dc/d22/c3e 0 2024-04-24T22:45:22.237 INFO:tasks.workunit.client.1.smithi192.stdout:3/243: write d2/d4/d7/f2e [3219295,92234] 0 2024-04-24T22:45:22.239 INFO:tasks.workunit.client.1.smithi192.stdout:8/215: chown db/dc/f17 3232883 1 2024-04-24T22:45:22.239 INFO:tasks.workunit.client.1.smithi192.stdout:7/227: symlink d2/db/d21/l40 0 2024-04-24T22:45:22.241 INFO:tasks.workunit.client.1.smithi192.stdout:4/208: rename d2/d9/f2d to d2/d9/d25/d2e/d31/f4c 0 2024-04-24T22:45:22.242 INFO:tasks.workunit.client.1.smithi192.stdout:0/320: mknod d3/d27/d28/d2e/d4a/d5c/c73 0 2024-04-24T22:45:22.243 INFO:tasks.workunit.client.1.smithi192.stdout:0/321: fdatasync d3/f39 0 2024-04-24T22:45:22.244 INFO:tasks.workunit.client.1.smithi192.stdout:3/244: fdatasync d2/d4/d12/f1b 0 2024-04-24T22:45:22.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/228: mknod d2/db/d21/c41 0 2024-04-24T22:45:22.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/229: chown d2/db/d2a/f2b 119731260 1 2024-04-24T22:45:22.251 INFO:tasks.workunit.client.1.smithi192.stdout:3/245: unlink d2/f3b 0 2024-04-24T22:45:22.253 INFO:tasks.workunit.client.1.smithi192.stdout:8/216: mknod db/c3f 0 2024-04-24T22:45:22.254 INFO:tasks.workunit.client.1.smithi192.stdout:4/209: symlink d2/db/l4d 0 2024-04-24T22:45:22.256 INFO:tasks.workunit.client.1.smithi192.stdout:0/322: symlink d3/l74 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:3/246: dwrite d2/d4/d12/f49 [0,4194304] 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:7/230: fdatasync f1 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:0/323: mknod d3/d6d/c75 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:0/324: dread - d3/d27/d3c/f4e zero size 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:4/210: dread d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:7/231: mkdir d2/db/d21/d42 0 2024-04-24T22:45:22.265 INFO:tasks.workunit.client.1.smithi192.stdout:9/184: write d3/dd/de/d17/f1d [4670229,29369] 0 2024-04-24T22:45:22.266 INFO:tasks.workunit.client.1.smithi192.stdout:7/232: chown d2/db/d36/f3d 1976 1 2024-04-24T22:45:22.267 INFO:tasks.workunit.client.1.smithi192.stdout:7/233: write d2/db/d36/f3d [643534,58555] 0 2024-04-24T22:45:22.269 INFO:tasks.workunit.client.1.smithi192.stdout:3/247: symlink d2/d27/d28/d38/l51 0 2024-04-24T22:45:22.270 INFO:tasks.workunit.client.1.smithi192.stdout:3/248: fdatasync d2/f1e 0 2024-04-24T22:45:22.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/185: creat d3/d2c/f42 x:0 0 0 2024-04-24T22:45:22.271 INFO:tasks.workunit.client.1.smithi192.stdout:9/186: chown d3/f3f 98 1 2024-04-24T22:45:22.272 INFO:tasks.workunit.client.1.smithi192.stdout:3/249: chown d2/d27/d28/d38/l51 230 1 2024-04-24T22:45:22.273 INFO:tasks.workunit.client.1.smithi192.stdout:9/187: write d3/dd/de/d17/f23 [378305,18507] 0 2024-04-24T22:45:22.280 INFO:tasks.workunit.client.1.smithi192.stdout:3/250: creat d2/d4/d7/d4f/f52 x:0 0 0 2024-04-24T22:45:22.281 INFO:tasks.workunit.client.1.smithi192.stdout:3/251: chown d2/d4/d12 92807226 1 2024-04-24T22:45:22.283 INFO:tasks.workunit.client.1.smithi192.stdout:9/188: creat d3/f43 x:0 0 0 2024-04-24T22:45:22.284 INFO:tasks.workunit.client.1.smithi192.stdout:7/234: link d2/db/d25/l31 d2/l43 0 2024-04-24T22:45:22.285 INFO:tasks.workunit.client.1.smithi192.stdout:9/189: creat d3/d24/f44 x:0 0 0 2024-04-24T22:45:22.286 INFO:tasks.workunit.client.1.smithi192.stdout:9/190: write d3/dd/de/d17/f23 [164243,60660] 0 2024-04-24T22:45:22.288 INFO:tasks.workunit.client.0.smithi063.stdout:9/275: write d6/d8/db/f10 [914994,56453] 0 2024-04-24T22:45:22.290 INFO:tasks.workunit.client.1.smithi192.stdout:3/252: creat d2/d4/d12/f53 x:0 0 0 2024-04-24T22:45:22.291 INFO:tasks.workunit.client.1.smithi192.stdout:9/191: fdatasync d3/dd/de/f1a 0 2024-04-24T22:45:22.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/222: dwrite d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:22.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/223: stat d0/d8/d22/c23 0 2024-04-24T22:45:22.294 INFO:tasks.workunit.client.1.smithi192.stdout:6/224: write d0/d8/d18/f3f [230077,42374] 0 2024-04-24T22:45:22.296 INFO:tasks.workunit.client.0.smithi063.stdout:9/276: rmdir d6/d8 39 2024-04-24T22:45:22.301 INFO:tasks.workunit.client.1.smithi192.stdout:3/253: mkdir d2/d4/d7/d4b/d54 0 2024-04-24T22:45:22.304 INFO:tasks.workunit.client.1.smithi192.stdout:9/192: chown d3/dd/de/c1c 29 1 2024-04-24T22:45:22.304 INFO:tasks.workunit.client.1.smithi192.stdout:9/193: stat d3/dd/de/l1b 0 2024-04-24T22:45:22.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/309: dwrite d1/d22/f26 [0,4194304] 0 2024-04-24T22:45:22.307 INFO:tasks.workunit.client.1.smithi192.stdout:6/225: creat d0/d8/d18/d2e/f51 x:0 0 0 2024-04-24T22:45:22.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/289: sync 2024-04-24T22:45:22.308 INFO:tasks.workunit.client.0.smithi063.stdout:4/267: sync 2024-04-24T22:45:22.311 INFO:tasks.workunit.client.0.smithi063.stdout:9/277: rename d6/d8/d1f/d20/d2a/l2d to d6/d2f/d42/l5d 0 2024-04-24T22:45:22.312 INFO:tasks.workunit.client.0.smithi063.stdout:2/290: dread d0/d2/f47 [0,4194304] 0 2024-04-24T22:45:22.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/278: fdatasync d6/d8/d4d/f1e 0 2024-04-24T22:45:22.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/279: truncate d6/d8/d1f/f28 789880 0 2024-04-24T22:45:22.314 INFO:tasks.workunit.client.0.smithi063.stdout:8/310: mkdir d1/d11/d32/d6a 0 2024-04-24T22:45:22.317 INFO:tasks.workunit.client.1.smithi192.stdout:7/235: getdents d2/db 0 2024-04-24T22:45:22.318 INFO:tasks.workunit.client.1.smithi192.stdout:6/226: mknod d0/d8/d29/c52 0 2024-04-24T22:45:22.320 INFO:tasks.workunit.client.0.smithi063.stdout:4/268: mknod d2/d19/d33/c4c 0 2024-04-24T22:45:22.325 INFO:tasks.workunit.client.1.smithi192.stdout:6/227: dwrite d0/d8/d3d/d3e/f4b [0,4194304] 0 2024-04-24T22:45:22.327 INFO:tasks.workunit.client.0.smithi063.stdout:2/291: symlink d0/d2/l56 0 2024-04-24T22:45:22.329 INFO:tasks.workunit.client.0.smithi063.stdout:8/311: rename d1/f68 to d1/d11/d3c/f6b 0 2024-04-24T22:45:22.331 INFO:tasks.workunit.client.0.smithi063.stdout:9/280: mkdir d6/d8/d1f/d20/d2a/d4b/d5e 0 2024-04-24T22:45:22.332 INFO:tasks.workunit.client.1.smithi192.stdout:9/194: creat d3/f45 x:0 0 0 2024-04-24T22:45:22.332 INFO:tasks.workunit.client.0.smithi063.stdout:9/281: fsync d6/d8/db/d1c/f57 0 2024-04-24T22:45:22.337 INFO:tasks.workunit.client.0.smithi063.stdout:4/269: read d2/d8/f1d [1706205,26404] 0 2024-04-24T22:45:22.338 INFO:tasks.workunit.client.1.smithi192.stdout:6/228: rename d0/d8/d3d/d3e/f4e to d0/d8/d29/f53 0 2024-04-24T22:45:22.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/282: dwrite d6/d8/d4d/f23 [0,4194304] 0 2024-04-24T22:45:22.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/292: unlink d0/d2/d54/d7/c46 0 2024-04-24T22:45:22.342 INFO:tasks.workunit.client.1.smithi192.stdout:3/254: getdents d2/d4/d12/d1a 0 2024-04-24T22:45:22.343 INFO:tasks.workunit.client.0.smithi063.stdout:2/293: readlink d0/d2/d39/l4c 0 2024-04-24T22:45:22.343 INFO:tasks.workunit.client.0.smithi063.stdout:8/312: readlink d1/d2/l28 0 2024-04-24T22:45:22.344 INFO:tasks.workunit.client.1.smithi192.stdout:2/301: write d4/d9/d10/f53 [2071142,64320] 0 2024-04-24T22:45:22.346 INFO:tasks.workunit.client.0.smithi063.stdout:9/283: dwrite d6/d8/f5b [0,4194304] 0 2024-04-24T22:45:22.348 INFO:tasks.workunit.client.0.smithi063.stdout:9/284: write d6/d8/d1f/d20/f21 [5177846,105681] 0 2024-04-24T22:45:22.353 INFO:tasks.workunit.client.0.smithi063.stdout:4/270: rename f1 to d2/d19/f4d 0 2024-04-24T22:45:22.355 INFO:tasks.workunit.client.1.smithi192.stdout:2/302: dwrite d4/d9/d10/f53 [0,4194304] 0 2024-04-24T22:45:22.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/255: chown d2/d4/d7/f9 71 1 2024-04-24T22:45:22.358 INFO:tasks.workunit.client.1.smithi192.stdout:3/256: chown d2/d4/d12/d39/c43 1281 1 2024-04-24T22:45:22.362 INFO:tasks.workunit.client.0.smithi063.stdout:0/262: sync 2024-04-24T22:45:22.362 INFO:tasks.workunit.client.0.smithi063.stdout:7/221: sync 2024-04-24T22:45:22.365 INFO:tasks.workunit.client.1.smithi192.stdout:9/195: rmdir d3/d2c 39 2024-04-24T22:45:22.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/211: dwrite d2/d9/d1e/f47 [0,4194304] 0 2024-04-24T22:45:22.368 INFO:tasks.workunit.client.1.smithi192.stdout:0/325: truncate d3/d27/d28/d2e/f4c 3788048 0 2024-04-24T22:45:22.376 INFO:tasks.workunit.client.0.smithi063.stdout:4/271: dread - d2/d8/f23 zero size 2024-04-24T22:45:22.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/303: rename d4/d9/f35 to d4/d9/d10/f58 0 2024-04-24T22:45:22.379 INFO:tasks.workunit.client.1.smithi192.stdout:6/229: dread d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:22.381 INFO:tasks.workunit.client.1.smithi192.stdout:9/196: fsync d3/fb 0 2024-04-24T22:45:22.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/197: dread - d3/dd/de/f19 zero size 2024-04-24T22:45:22.382 INFO:tasks.workunit.client.0.smithi063.stdout:7/222: fsync d0/f7 0 2024-04-24T22:45:22.385 INFO:tasks.workunit.client.0.smithi063.stdout:0/263: rmdir d6/da/dc/d26/d33 39 2024-04-24T22:45:22.385 INFO:tasks.workunit.client.0.smithi063.stdout:7/223: dread d0/d4/de/d17/f43 [0,4194304] 0 2024-04-24T22:45:22.388 INFO:tasks.workunit.client.0.smithi063.stdout:7/224: truncate d0/d4/de/d17/f43 383526 0 2024-04-24T22:45:22.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/294: link d0/d2/d54/f25 d0/f57 0 2024-04-24T22:45:22.392 INFO:tasks.workunit.client.0.smithi063.stdout:8/313: fsync d1/d11/d3c/f6b 0 2024-04-24T22:45:22.392 INFO:tasks.workunit.client.0.smithi063.stdout:0/264: dwrite d6/dd/f4d [0,4194304] 0 2024-04-24T22:45:22.394 INFO:tasks.workunit.client.1.smithi192.stdout:3/257: rename d2/d4/d12/d1a/d22 to d2/d4/d12/d1a/d1d/d55 0 2024-04-24T22:45:22.398 INFO:tasks.workunit.client.0.smithi063.stdout:9/285: rename d6/d2f/d42/l5d to d6/d8/d4d/l5f 0 2024-04-24T22:45:22.400 INFO:tasks.workunit.client.0.smithi063.stdout:4/272: creat d2/d8/d3c/f4e x:0 0 0 2024-04-24T22:45:22.401 INFO:tasks.workunit.client.0.smithi063.stdout:4/273: stat d2/d19/c2f 0 2024-04-24T22:45:22.401 INFO:tasks.workunit.client.0.smithi063.stdout:4/274: stat d2/f29 0 2024-04-24T22:45:22.405 INFO:tasks.workunit.client.1.smithi192.stdout:7/236: dwrite d2/f17 [4194304,4194304] 0 2024-04-24T22:45:22.406 INFO:tasks.workunit.client.1.smithi192.stdout:6/230: rename d0/d8/d18/l38 to d0/d8/d18/d2e/l54 0 2024-04-24T22:45:22.406 INFO:tasks.workunit.client.1.smithi192.stdout:9/198: rename d3/dd/de to d3/dd/de/d39/d46 22 2024-04-24T22:45:22.408 INFO:tasks.workunit.client.1.smithi192.stdout:9/199: read d3/dd/de/d17/f34 [3341159,21907] 0 2024-04-24T22:45:22.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/295: chown d0/d2/d54/l38 29 1 2024-04-24T22:45:22.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/200: truncate d3/dd/de/d17/f1d 4920950 0 2024-04-24T22:45:22.413 INFO:tasks.workunit.client.1.smithi192.stdout:7/237: dwrite d2/db/d2a/f3c [4194304,4194304] 0 2024-04-24T22:45:22.414 INFO:tasks.workunit.client.0.smithi063.stdout:0/265: unlink d6/dd/d20/d21/d25/f4e 0 2024-04-24T22:45:22.415 INFO:tasks.workunit.client.0.smithi063.stdout:7/225: dread d0/d4/de/d17/f42 [0,4194304] 0 2024-04-24T22:45:22.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/226: truncate d0/f48 166758 0 2024-04-24T22:45:22.421 INFO:tasks.workunit.client.1.smithi192.stdout:6/231: dwrite d0/d8/f34 [0,4194304] 0 2024-04-24T22:45:22.424 INFO:tasks.workunit.client.1.smithi192.stdout:4/212: truncate d2/d9/d1e/f37 121054 0 2024-04-24T22:45:22.425 INFO:tasks.workunit.client.1.smithi192.stdout:7/238: mknod d2/db/d25/c44 0 2024-04-24T22:45:22.426 INFO:tasks.workunit.client.0.smithi063.stdout:0/266: dwrite d6/dd/d20/d21/f48 [0,4194304] 0 2024-04-24T22:45:22.429 INFO:tasks.workunit.client.1.smithi192.stdout:2/304: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:22.430 INFO:tasks.workunit.client.0.smithi063.stdout:7/227: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:22.437 INFO:tasks.workunit.client.0.smithi063.stdout:9/286: dwrite d6/d8/d1f/d20/d22/d31/f33 [0,4194304] 0 2024-04-24T22:45:22.438 INFO:tasks.workunit.client.0.smithi063.stdout:9/287: chown d6/d8/db/d1c 124 1 2024-04-24T22:45:22.461 INFO:tasks.workunit.client.0.smithi063.stdout:6/275: sync 2024-04-24T22:45:22.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/311: sync 2024-04-24T22:45:22.461 INFO:tasks.workunit.client.0.smithi063.stdout:3/299: sync 2024-04-24T22:45:22.461 INFO:tasks.workunit.client.0.smithi063.stdout:1/287: sync 2024-04-24T22:45:22.525 INFO:tasks.workunit.client.0.smithi063.stdout:0/267: sync 2024-04-24T22:45:22.525 INFO:tasks.workunit.client.0.smithi063.stdout:8/314: sync 2024-04-24T22:45:22.527 INFO:tasks.workunit.client.1.smithi192.stdout:0/326: truncate d3/f5e 1851414 0 2024-04-24T22:45:22.529 INFO:tasks.workunit.client.1.smithi192.stdout:9/201: symlink d3/dd/de/d39/l47 0 2024-04-24T22:45:22.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/315: dread d1/d11/d32/d38/f42 [0,4194304] 0 2024-04-24T22:45:22.542 INFO:tasks.workunit.client.1.smithi192.stdout:7/239: chown d2/l3 5 1 2024-04-24T22:45:22.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/296: creat d0/d2/d2f/f58 x:0 0 0 2024-04-24T22:45:22.545 INFO:tasks.workunit.client.1.smithi192.stdout:3/258: getdents d2/d4 0 2024-04-24T22:45:22.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/275: write d2/d8/f1d [2023179,117158] 0 2024-04-24T22:45:22.547 INFO:tasks.workunit.client.0.smithi063.stdout:9/288: readlink d6/d8/d1f/d20/d22/l3e 0 2024-04-24T22:45:22.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/276: chown d2/d8/d36/l46 7815911 1 2024-04-24T22:45:22.550 INFO:tasks.workunit.client.1.smithi192.stdout:9/202: creat d3/dd/de/d17/f48 x:0 0 0 2024-04-24T22:45:22.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/259: dwrite d2/d4/f16 [0,4194304] 0 2024-04-24T22:45:22.553 INFO:tasks.workunit.client.0.smithi063.stdout:8/316: dread d1/d11/d32/d38/f3f [0,4194304] 0 2024-04-24T22:45:22.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/300: rmdir da 39 2024-04-24T22:45:22.554 INFO:tasks.workunit.client.0.smithi063.stdout:4/277: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:22.554 INFO:tasks.workunit.client.0.smithi063.stdout:4/278: fsync d2/f26 0 2024-04-24T22:45:22.555 INFO:tasks.workunit.client.0.smithi063.stdout:4/279: write d2/fd [9053191,28909] 0 2024-04-24T22:45:22.557 INFO:tasks.workunit.client.1.smithi192.stdout:7/240: creat d2/db/f45 x:0 0 0 2024-04-24T22:45:22.560 INFO:tasks.workunit.client.1.smithi192.stdout:3/260: dwrite d2/d4/d12/d1a/d1d/f3f [0,4194304] 0 2024-04-24T22:45:22.566 INFO:tasks.workunit.client.1.smithi192.stdout:9/203: rmdir d3/dd/de 39 2024-04-24T22:45:22.571 INFO:tasks.workunit.client.1.smithi192.stdout:4/213: creat d2/db/f4e x:0 0 0 2024-04-24T22:45:22.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/241: write d2/db/f15 [3201132,43254] 0 2024-04-24T22:45:22.573 INFO:tasks.workunit.client.0.smithi063.stdout:9/289: mknod d6/d8/d1f/c60 0 2024-04-24T22:45:22.573 INFO:tasks.workunit.client.0.smithi063.stdout:9/290: chown d6 5171348 1 2024-04-24T22:45:22.574 INFO:tasks.workunit.client.1.smithi192.stdout:2/305: creat d4/f59 x:0 0 0 2024-04-24T22:45:22.575 INFO:tasks.workunit.client.1.smithi192.stdout:3/261: mknod d2/d4/d7/d4b/c56 0 2024-04-24T22:45:22.576 INFO:tasks.workunit.client.1.smithi192.stdout:3/262: write d2/f33 [1820789,36554] 0 2024-04-24T22:45:22.578 INFO:tasks.workunit.client.1.smithi192.stdout:0/327: rename d3/d27/d3c/c54 to d3/d27/d28/d2e/c76 0 2024-04-24T22:45:22.578 INFO:tasks.workunit.client.1.smithi192.stdout:0/328: chown d3/d27 41 1 2024-04-24T22:45:22.579 INFO:tasks.workunit.client.1.smithi192.stdout:0/329: dread - d3/d27/d28/d2e/d4a/d5c/f46 zero size 2024-04-24T22:45:22.581 INFO:tasks.workunit.client.1.smithi192.stdout:0/330: write d3/d27/d28/d2e/d4a/d5c/f59 [500438,127802] 0 2024-04-24T22:45:22.582 INFO:tasks.workunit.client.1.smithi192.stdout:0/331: write d3/d27/d28/d2e/d4a/f4d [1541907,64767] 0 2024-04-24T22:45:22.583 INFO:tasks.workunit.client.0.smithi063.stdout:4/280: unlink d2/c2a 0 2024-04-24T22:45:22.584 INFO:tasks.workunit.client.0.smithi063.stdout:4/281: chown d2/d8/d36/l38 12770 1 2024-04-24T22:45:22.586 INFO:tasks.workunit.client.1.smithi192.stdout:7/242: unlink d2/db/d21/c41 0 2024-04-24T22:45:22.588 INFO:tasks.workunit.client.0.smithi063.stdout:8/317: write d1/d2/fd [4290587,110511] 0 2024-04-24T22:45:22.589 INFO:tasks.workunit.client.0.smithi063.stdout:8/318: fdatasync d1/f58 0 2024-04-24T22:45:22.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/268: mkdir d6/dd/d20/d5d 0 2024-04-24T22:45:22.592 INFO:tasks.workunit.client.1.smithi192.stdout:3/263: mknod d2/d4/d7/d4f/c57 0 2024-04-24T22:45:22.594 INFO:tasks.workunit.client.0.smithi063.stdout:8/319: dwrite d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:22.594 INFO:tasks.workunit.client.1.smithi192.stdout:7/243: dwrite d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:22.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/297: unlink d0/dd/c30 0 2024-04-24T22:45:22.600 INFO:tasks.workunit.client.0.smithi063.stdout:5/312: truncate d5/d24/d44/f46 2457669 0 2024-04-24T22:45:22.606 INFO:tasks.workunit.client.1.smithi192.stdout:4/214: link d2/db/f3d d2/db/d10/f4f 0 2024-04-24T22:45:22.607 INFO:tasks.workunit.client.1.smithi192.stdout:4/215: dread - d2/db/f3d zero size 2024-04-24T22:45:22.608 INFO:tasks.workunit.client.0.smithi063.stdout:3/301: creat da/d18/d47/f6a x:0 0 0 2024-04-24T22:45:22.612 INFO:tasks.workunit.client.1.smithi192.stdout:9/204: creat d3/dd/f49 x:0 0 0 2024-04-24T22:45:22.613 INFO:tasks.workunit.client.0.smithi063.stdout:4/282: chown d2/d8/f31 1589 1 2024-04-24T22:45:22.615 INFO:tasks.workunit.client.1.smithi192.stdout:7/244: mknod d2/db/d36/c46 0 2024-04-24T22:45:22.616 INFO:tasks.workunit.client.0.smithi063.stdout:9/291: dwrite d6/d2f/f35 [0,4194304] 0 2024-04-24T22:45:22.616 INFO:tasks.workunit.client.1.smithi192.stdout:0/332: dwrite d3/d27/d28/f17 [0,4194304] 0 2024-04-24T22:45:22.619 INFO:tasks.workunit.client.0.smithi063.stdout:0/269: chown d6/da/d2b 29 1 2024-04-24T22:45:22.620 INFO:tasks.workunit.client.1.smithi192.stdout:0/333: dread d3/f39 [0,4194304] 0 2024-04-24T22:45:22.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/298: symlink d0/d2/d27/l59 0 2024-04-24T22:45:22.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/228: getdents d0 0 2024-04-24T22:45:22.627 INFO:tasks.workunit.client.0.smithi063.stdout:7/229: read d0/d4/de/d17/f42 [278092,19463] 0 2024-04-24T22:45:22.627 INFO:tasks.workunit.client.1.smithi192.stdout:2/306: creat d4/d9/f5a x:0 0 0 2024-04-24T22:45:22.628 INFO:tasks.workunit.client.0.smithi063.stdout:8/320: dwrite d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:22.628 INFO:tasks.workunit.client.0.smithi063.stdout:8/321: chown d1/d11/d32/d38 217487 1 2024-04-24T22:45:22.628 INFO:tasks.workunit.client.0.smithi063.stdout:5/313: creat d5/d11/d23/d2d/f70 x:0 0 0 2024-04-24T22:45:22.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/322: write d1/d2/fd [3161202,104785] 0 2024-04-24T22:45:22.632 INFO:tasks.workunit.client.0.smithi063.stdout:6/276: getdents db 0 2024-04-24T22:45:22.633 INFO:tasks.workunit.client.0.smithi063.stdout:4/283: mknod d2/d19/d33/c4f 0 2024-04-24T22:45:22.633 INFO:tasks.workunit.client.1.smithi192.stdout:0/334: fsync d3/d27/d28/f34 0 2024-04-24T22:45:22.634 INFO:tasks.workunit.client.0.smithi063.stdout:1/288: getdents d3/da 0 2024-04-24T22:45:22.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/289: write d3/da/dc/d11/f41 [5600105,112151] 0 2024-04-24T22:45:22.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/292: creat d6/d8/d1f/f61 x:0 0 0 2024-04-24T22:45:22.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/205: rename d3/dd/de/d17/f34 to d3/d24/f4a 0 2024-04-24T22:45:22.642 INFO:tasks.workunit.client.1.smithi192.stdout:3/264: link d2/d27/d28/d38/l51 d2/d4/d12/d1a/d1d/l58 0 2024-04-24T22:45:22.643 INFO:tasks.workunit.client.0.smithi063.stdout:0/270: mknod d6/da/dc/c5e 0 2024-04-24T22:45:22.643 INFO:tasks.workunit.client.0.smithi063.stdout:0/271: chown d6/l56 794 1 2024-04-24T22:45:22.644 INFO:tasks.workunit.client.1.smithi192.stdout:3/265: dread d2/d4/d12/d1a/d1d/d55/f32 [0,4194304] 0 2024-04-24T22:45:22.644 INFO:tasks.workunit.client.0.smithi063.stdout:0/272: write f5 [4521130,19354] 0 2024-04-24T22:45:22.645 INFO:tasks.workunit.client.1.smithi192.stdout:3/266: chown d2/d4/d12/d1a/d1d/f25 2845 1 2024-04-24T22:45:22.646 INFO:tasks.workunit.client.1.smithi192.stdout:3/267: stat d2/d4/d12/d1a/d1d/l58 0 2024-04-24T22:45:22.646 INFO:tasks.workunit.client.0.smithi063.stdout:7/230: mknod d0/d4/de/d17/d35/c56 0 2024-04-24T22:45:22.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/216: creat d2/f50 x:0 0 0 2024-04-24T22:45:22.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/323: unlink d1/d22/f26 0 2024-04-24T22:45:22.657 INFO:tasks.workunit.client.0.smithi063.stdout:6/277: rename db/d1d/d20/d3f/c41 to db/d1c/c5a 0 2024-04-24T22:45:22.657 INFO:tasks.workunit.client.0.smithi063.stdout:2/299: dwrite d0/d2/f47 [0,4194304] 0 2024-04-24T22:45:22.657 INFO:tasks.workunit.client.0.smithi063.stdout:5/314: dwrite d5/d24/d49/d3b/f48 [0,4194304] 0 2024-04-24T22:45:22.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/245: rmdir d2/db/d21/d37 0 2024-04-24T22:45:22.659 INFO:tasks.workunit.client.1.smithi192.stdout:0/335: symlink d3/d27/d28/d2e/d3f/l77 0 2024-04-24T22:45:22.660 INFO:tasks.workunit.client.0.smithi063.stdout:2/300: chown d0/dd/f11 939341 1 2024-04-24T22:45:22.660 INFO:tasks.workunit.client.0.smithi063.stdout:1/290: dread d3/da/dc/d11/f41 [4194304,4194304] 0 2024-04-24T22:45:22.660 INFO:tasks.workunit.client.0.smithi063.stdout:5/315: write d5/d24/d2c/f3e [3742777,67130] 0 2024-04-24T22:45:22.661 INFO:tasks.workunit.client.0.smithi063.stdout:9/293: unlink d6/d8/db/d2b/f44 0 2024-04-24T22:45:22.662 INFO:tasks.workunit.client.1.smithi192.stdout:3/268: rename d2/d27/d28/c35 to d2/d4/d7/d4b/c59 0 2024-04-24T22:45:22.662 INFO:tasks.workunit.client.0.smithi063.stdout:9/294: chown d6/d8/db/d1c/f57 7472 1 2024-04-24T22:45:22.664 INFO:tasks.workunit.client.1.smithi192.stdout:4/217: symlink d2/d9/d25/d2e/l51 0 2024-04-24T22:45:22.664 INFO:tasks.workunit.client.1.smithi192.stdout:3/269: write d2/d4/d12/d1a/d1d/d55/f24 [621311,44803] 0 2024-04-24T22:45:22.665 INFO:tasks.workunit.client.1.smithi192.stdout:2/307: creat d4/d9/f5b x:0 0 0 2024-04-24T22:45:22.671 INFO:tasks.workunit.client.0.smithi063.stdout:2/301: dwrite d0/d2/f1e [0,4194304] 0 2024-04-24T22:45:22.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/231: read d0/d4/de/f10 [5219795,102037] 0 2024-04-24T22:45:22.671 INFO:tasks.workunit.client.1.smithi192.stdout:7/246: creat d2/db/d25/f47 x:0 0 0 2024-04-24T22:45:22.674 INFO:tasks.workunit.client.0.smithi063.stdout:7/232: dread d0/f3d [0,4194304] 0 2024-04-24T22:45:22.675 INFO:tasks.workunit.client.0.smithi063.stdout:7/233: write d0/f3d [5063240,35457] 0 2024-04-24T22:45:22.678 INFO:tasks.workunit.client.0.smithi063.stdout:8/324: fsync d1/d2/d3/f46 0 2024-04-24T22:45:22.687 INFO:tasks.workunit.client.1.smithi192.stdout:0/336: mkdir d3/d27/d6a/d78 0 2024-04-24T22:45:22.688 INFO:tasks.workunit.client.1.smithi192.stdout:0/337: dread - d3/d27/f71 zero size 2024-04-24T22:45:22.689 INFO:tasks.workunit.client.0.smithi063.stdout:1/291: symlink d3/da/dc/l60 0 2024-04-24T22:45:22.689 INFO:tasks.workunit.client.0.smithi063.stdout:1/292: dread - d3/da/dc/f3a zero size 2024-04-24T22:45:22.691 INFO:tasks.workunit.client.1.smithi192.stdout:7/247: creat d2/db/d3a/f48 x:0 0 0 2024-04-24T22:45:22.693 INFO:tasks.workunit.client.0.smithi063.stdout:9/295: rmdir d6 39 2024-04-24T22:45:22.693 INFO:tasks.workunit.client.1.smithi192.stdout:7/248: truncate d2/db/f20 9226389 0 2024-04-24T22:45:22.693 INFO:tasks.workunit.client.1.smithi192.stdout:0/338: dread d3/f47 [0,4194304] 0 2024-04-24T22:45:22.695 INFO:tasks.workunit.client.0.smithi063.stdout:6/278: dread db/f10 [0,4194304] 0 2024-04-24T22:45:22.695 INFO:tasks.workunit.client.0.smithi063.stdout:6/279: readlink db/d1d/l55 0 2024-04-24T22:45:22.697 INFO:tasks.workunit.client.1.smithi192.stdout:9/206: dwrite d3/dd/de/f16 [0,4194304] 0 2024-04-24T22:45:22.697 INFO:tasks.workunit.client.0.smithi063.stdout:2/302: symlink d0/d2/d2d/l5a 0 2024-04-24T22:45:22.698 INFO:tasks.workunit.client.0.smithi063.stdout:2/303: write d0/dd/f22 [447699,32202] 0 2024-04-24T22:45:22.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/218: creat d2/db/f52 x:0 0 0 2024-04-24T22:45:22.699 INFO:tasks.workunit.client.0.smithi063.stdout:3/302: getdents da/d18/d3a 0 2024-04-24T22:45:22.700 INFO:tasks.workunit.client.1.smithi192.stdout:3/270: write d2/d4/d7/f47 [672276,61428] 0 2024-04-24T22:45:22.703 INFO:tasks.workunit.client.1.smithi192.stdout:7/249: mkdir d2/db/d49 0 2024-04-24T22:45:22.705 INFO:tasks.workunit.client.1.smithi192.stdout:2/308: unlink d4/d9/d10/c1a 0 2024-04-24T22:45:22.706 INFO:tasks.workunit.client.0.smithi063.stdout:4/284: sync 2024-04-24T22:45:22.706 INFO:tasks.workunit.client.0.smithi063.stdout:7/234: fsync d0/f7 0 2024-04-24T22:45:22.708 INFO:tasks.workunit.client.1.smithi192.stdout:0/339: rename d3/d27/d6a/d78 to d3/d6d/d79 0 2024-04-24T22:45:22.709 INFO:tasks.workunit.client.1.smithi192.stdout:3/271: dwrite d2/d27/d28/f30 [0,4194304] 0 2024-04-24T22:45:22.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/207: rmdir d3/dd/de 39 2024-04-24T22:45:22.710 INFO:tasks.workunit.client.1.smithi192.stdout:3/272: chown d2/d4/d7/d4b/d54 25052 1 2024-04-24T22:45:22.710 INFO:tasks.workunit.client.1.smithi192.stdout:0/340: write d3/d27/d3c/d3b/f65 [688272,58009] 0 2024-04-24T22:45:22.712 INFO:tasks.workunit.client.1.smithi192.stdout:4/219: mkdir d2/d9/d25/d2e/d31/d53 0 2024-04-24T22:45:22.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/220: fsync d2/d9/d1e/d28/d2f/f3c 0 2024-04-24T22:45:22.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/293: creat d3/da/dc/d11/f61 x:0 0 0 2024-04-24T22:45:22.718 INFO:tasks.workunit.client.0.smithi063.stdout:3/303: dread da/f14 [0,4194304] 0 2024-04-24T22:45:22.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/273: creat d6/da/dc/d26/f5f x:0 0 0 2024-04-24T22:45:22.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/280: dread db/f37 [0,4194304] 0 2024-04-24T22:45:22.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/221: rmdir d2 39 2024-04-24T22:45:22.728 INFO:tasks.workunit.client.1.smithi192.stdout:2/309: symlink d4/d9/l5c 0 2024-04-24T22:45:22.731 INFO:tasks.workunit.client.1.smithi192.stdout:9/208: mknod d3/dd/de/d39/c4b 0 2024-04-24T22:45:22.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/222: write d2/d9/f3f [429299,12359] 0 2024-04-24T22:45:22.733 INFO:tasks.workunit.client.0.smithi063.stdout:4/285: mkdir d2/d8/d36/d3a/d50 0 2024-04-24T22:45:22.735 INFO:tasks.workunit.client.0.smithi063.stdout:7/235: rename d0/d4/de/d17/l44 to d0/d4/de/d17/l57 0 2024-04-24T22:45:22.737 INFO:tasks.workunit.client.0.smithi063.stdout:1/294: rmdir d3/da/dc/d39 39 2024-04-24T22:45:22.737 INFO:tasks.workunit.client.0.smithi063.stdout:1/295: chown d3/da/dc/d1f/d21/d42 230517392 1 2024-04-24T22:45:22.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/209: mknod d3/c4c 0 2024-04-24T22:45:22.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/210: write d3/dd/f49 [570371,18177] 0 2024-04-24T22:45:22.740 INFO:tasks.workunit.client.0.smithi063.stdout:9/296: creat d6/d8/db/f62 x:0 0 0 2024-04-24T22:45:22.741 INFO:tasks.workunit.client.0.smithi063.stdout:1/296: dread d3/da/f3d [0,4194304] 0 2024-04-24T22:45:22.742 INFO:tasks.workunit.client.1.smithi192.stdout:4/223: creat d2/d9/d1e/f54 x:0 0 0 2024-04-24T22:45:22.745 INFO:tasks.workunit.client.1.smithi192.stdout:9/211: write d3/d2c/f2e [543739,60176] 0 2024-04-24T22:45:22.747 INFO:tasks.workunit.client.1.smithi192.stdout:9/212: readlink d3/dd/de/l3c 0 2024-04-24T22:45:22.749 INFO:tasks.workunit.client.1.smithi192.stdout:2/310: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:22.749 INFO:tasks.workunit.client.1.smithi192.stdout:9/213: dread - d3/dd/de/f2f zero size 2024-04-24T22:45:22.750 INFO:tasks.workunit.client.1.smithi192.stdout:4/224: dread d2/d9/d25/d2e/d31/f4c [0,4194304] 0 2024-04-24T22:45:22.754 INFO:tasks.workunit.client.1.smithi192.stdout:4/225: mknod d2/db/c55 0 2024-04-24T22:45:22.754 INFO:tasks.workunit.client.1.smithi192.stdout:4/226: chown d2/db/d10 115 1 2024-04-24T22:45:22.755 INFO:tasks.workunit.client.1.smithi192.stdout:2/311: creat d4/d9/d10/f5d x:0 0 0 2024-04-24T22:45:22.755 INFO:tasks.workunit.client.1.smithi192.stdout:4/227: readlink d2/d9/d1e/l30 0 2024-04-24T22:45:22.756 INFO:tasks.workunit.client.1.smithi192.stdout:1/207: sync 2024-04-24T22:45:22.756 INFO:tasks.workunit.client.1.smithi192.stdout:4/228: fdatasync d2/d9/d25/d2e/d31/f4a 0 2024-04-24T22:45:22.758 INFO:tasks.workunit.client.1.smithi192.stdout:9/214: link d3/dd/de/l1b d3/dd/l4d 0 2024-04-24T22:45:22.758 INFO:tasks.workunit.client.1.smithi192.stdout:2/312: write d4/d9/f52 [845144,117034] 0 2024-04-24T22:45:22.761 INFO:tasks.workunit.client.1.smithi192.stdout:4/229: symlink d2/d9/d1e/d28/l56 0 2024-04-24T22:45:22.762 INFO:tasks.workunit.client.0.smithi063.stdout:2/304: creat d0/d2/d2d/d31/f5b x:0 0 0 2024-04-24T22:45:22.762 INFO:tasks.workunit.client.0.smithi063.stdout:6/281: unlink db/c11 0 2024-04-24T22:45:22.762 INFO:tasks.workunit.client.1.smithi192.stdout:4/230: dread - d2/f50 zero size 2024-04-24T22:45:22.763 INFO:tasks.workunit.client.0.smithi063.stdout:5/316: getdents d5/d1d 0 2024-04-24T22:45:22.763 INFO:tasks.workunit.client.0.smithi063.stdout:1/297: write d3/da/dc/d1f/d2b/f43 [3259365,2583] 0 2024-04-24T22:45:22.768 INFO:tasks.workunit.client.1.smithi192.stdout:9/215: truncate d3/fb 3601627 0 2024-04-24T22:45:22.768 INFO:tasks.workunit.client.1.smithi192.stdout:1/208: dwrite d9/d15/d2d/d31/f32 [0,4194304] 0 2024-04-24T22:45:22.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/216: stat d3/dd/de/d17/d33 0 2024-04-24T22:45:22.771 INFO:tasks.workunit.client.1.smithi192.stdout:2/313: dwrite d4/d9/d37/f55 [0,4194304] 0 2024-04-24T22:45:22.771 INFO:tasks.workunit.client.1.smithi192.stdout:4/231: rename d2/db/d10/l33 to d2/d9/d1e/d28/l57 0 2024-04-24T22:45:22.789 INFO:tasks.workunit.client.1.smithi192.stdout:1/209: creat d9/d15/f47 x:0 0 0 2024-04-24T22:45:22.790 INFO:tasks.workunit.client.1.smithi192.stdout:1/210: write d9/f1a [3847459,73974] 0 2024-04-24T22:45:22.794 INFO:tasks.workunit.client.1.smithi192.stdout:2/314: creat d4/f5e x:0 0 0 2024-04-24T22:45:22.795 INFO:tasks.workunit.client.1.smithi192.stdout:4/232: link d2/d9/d1e/d28/d2f/c42 d2/d9/d25/d2e/d31/c58 0 2024-04-24T22:45:22.798 INFO:tasks.workunit.client.1.smithi192.stdout:9/217: dwrite d3/f4 [0,4194304] 0 2024-04-24T22:45:22.800 INFO:tasks.workunit.client.0.smithi063.stdout:7/236: link d0/d4/de/d17/l1f d0/d2f/l58 0 2024-04-24T22:45:22.802 INFO:tasks.workunit.client.0.smithi063.stdout:2/305: creat d0/d2/d54/d7/f5c x:0 0 0 2024-04-24T22:45:22.802 INFO:tasks.workunit.client.0.smithi063.stdout:1/298: symlink d3/da/dc/d1f/l62 0 2024-04-24T22:45:22.803 INFO:tasks.workunit.client.0.smithi063.stdout:3/304: dwrite da/f1a [4194304,4194304] 0 2024-04-24T22:45:22.804 INFO:tasks.workunit.client.1.smithi192.stdout:9/218: truncate d3/f21 477430 0 2024-04-24T22:45:22.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/219: creat d3/d24/f4e x:0 0 0 2024-04-24T22:45:22.806 INFO:tasks.workunit.client.1.smithi192.stdout:9/220: fdatasync d3/dd/f30 0 2024-04-24T22:45:22.806 INFO:tasks.workunit.client.1.smithi192.stdout:9/221: dread - d3/d24/f4e zero size 2024-04-24T22:45:22.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/325: link d1/c3e d1/d22/c6c 0 2024-04-24T22:45:22.807 INFO:tasks.workunit.client.1.smithi192.stdout:9/222: chown d3/d24/f41 0 1 2024-04-24T22:45:22.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/223: write d3/d24/f41 [843723,51973] 0 2024-04-24T22:45:22.809 INFO:tasks.workunit.client.0.smithi063.stdout:6/282: dwrite db/d1d/d2d/f52 [0,4194304] 0 2024-04-24T22:45:22.809 INFO:tasks.workunit.client.0.smithi063.stdout:1/299: rmdir d3 39 2024-04-24T22:45:22.809 INFO:tasks.workunit.client.0.smithi063.stdout:6/283: stat db/l31 0 2024-04-24T22:45:22.810 INFO:tasks.workunit.client.0.smithi063.stdout:6/284: dread - db/d1d/d20/f50 zero size 2024-04-24T22:45:22.811 INFO:tasks.workunit.client.1.smithi192.stdout:9/224: creat d3/dd/de/d17/f4f x:0 0 0 2024-04-24T22:45:22.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/306: creat d0/d2/d54/d48/f5d x:0 0 0 2024-04-24T22:45:22.813 INFO:tasks.workunit.client.0.smithi063.stdout:2/307: rename d0 to d0/d2/d5e 22 2024-04-24T22:45:22.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/285: write db/d1c/f58 [418708,72965] 0 2024-04-24T22:45:22.814 INFO:tasks.workunit.client.1.smithi192.stdout:1/211: write d9/dd/ff [331642,98327] 0 2024-04-24T22:45:22.815 INFO:tasks.workunit.client.0.smithi063.stdout:8/326: mknod d1/d22/c6d 0 2024-04-24T22:45:22.816 INFO:tasks.workunit.client.0.smithi063.stdout:6/286: dread db/d1d/d20/d3e/f43 [0,4194304] 0 2024-04-24T22:45:22.817 INFO:tasks.workunit.client.1.smithi192.stdout:2/315: truncate d4/d29/f51 191837 0 2024-04-24T22:45:22.819 INFO:tasks.workunit.client.0.smithi063.stdout:2/308: symlink d0/d2/d35/l5f 0 2024-04-24T22:45:22.820 INFO:tasks.workunit.client.1.smithi192.stdout:9/225: dwrite d3/f3f [0,4194304] 0 2024-04-24T22:45:22.820 INFO:tasks.workunit.client.1.smithi192.stdout:2/316: write d4/d9/f52 [1171047,29235] 0 2024-04-24T22:45:22.822 INFO:tasks.workunit.client.1.smithi192.stdout:2/317: chown d4/d29/f34 432665 1 2024-04-24T22:45:22.823 INFO:tasks.workunit.client.1.smithi192.stdout:2/318: fdatasync d4/d9/d37/f44 0 2024-04-24T22:45:22.824 INFO:tasks.workunit.client.0.smithi063.stdout:8/327: truncate d1/d2/d17/f3d 4744378 0 2024-04-24T22:45:22.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/226: unlink d3/d2c/f2d 0 2024-04-24T22:45:22.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/212: read d9/f13 [837080,114827] 0 2024-04-24T22:45:22.828 INFO:tasks.workunit.client.0.smithi063.stdout:7/237: getdents d0 0 2024-04-24T22:45:22.830 INFO:tasks.workunit.client.0.smithi063.stdout:6/287: rename db/d1d/l24 to db/d1d/d20/d3f/l5b 0 2024-04-24T22:45:22.832 INFO:tasks.workunit.client.0.smithi063.stdout:3/305: dread f7 [0,4194304] 0 2024-04-24T22:45:22.833 INFO:tasks.workunit.client.1.smithi192.stdout:4/233: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:22.839 INFO:tasks.workunit.client.1.smithi192.stdout:4/234: dread d2/d9/d25/d2e/d31/f4c [0,4194304] 0 2024-04-24T22:45:22.841 INFO:tasks.workunit.client.1.smithi192.stdout:4/235: read d2/d9/d1e/d28/f45 [90255,63310] 0 2024-04-24T22:45:22.841 INFO:tasks.workunit.client.0.smithi063.stdout:1/300: creat d3/f63 x:0 0 0 2024-04-24T22:45:22.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/288: dwrite db/d1d/d20/d29/f2e [0,4194304] 0 2024-04-24T22:45:22.842 INFO:tasks.workunit.client.0.smithi063.stdout:7/238: symlink d0/d26/d2e/l59 0 2024-04-24T22:45:22.843 INFO:tasks.workunit.client.1.smithi192.stdout:2/319: dread d4/d29/f51 [0,4194304] 0 2024-04-24T22:45:22.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/320: write d4/f59 [905076,73523] 0 2024-04-24T22:45:22.845 INFO:tasks.workunit.client.0.smithi063.stdout:1/301: write d3/f3c [2247433,30016] 0 2024-04-24T22:45:22.846 INFO:tasks.workunit.client.0.smithi063.stdout:2/309: dwrite d0/d2/f4b [0,4194304] 0 2024-04-24T22:45:22.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/328: dread d1/f1f [0,4194304] 0 2024-04-24T22:45:22.850 INFO:tasks.workunit.client.0.smithi063.stdout:6/289: dread db/d1d/d2d/f36 [0,4194304] 0 2024-04-24T22:45:22.852 INFO:tasks.workunit.client.0.smithi063.stdout:8/329: write d1/d11/d49/d54/f60 [10758,94480] 0 2024-04-24T22:45:22.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/236: unlink d2/c16 0 2024-04-24T22:45:22.856 INFO:tasks.workunit.client.0.smithi063.stdout:3/306: rename c6 to da/d18/d47/d4d/c6b 0 2024-04-24T22:45:22.858 INFO:tasks.workunit.client.0.smithi063.stdout:3/307: readlink da/d40/l42 0 2024-04-24T22:45:22.858 INFO:tasks.workunit.client.0.smithi063.stdout:1/302: dwrite d3/da/dc/d11/f46 [0,4194304] 0 2024-04-24T22:45:22.859 INFO:tasks.workunit.client.1.smithi192.stdout:9/227: dread d3/dd/de/d17/f1d [0,4194304] 0 2024-04-24T22:45:22.859 INFO:tasks.workunit.client.0.smithi063.stdout:1/303: dread - d3/da/dc/d11/f5f zero size 2024-04-24T22:45:22.860 INFO:tasks.workunit.client.1.smithi192.stdout:9/228: dread - d3/dd/f3b zero size 2024-04-24T22:45:22.862 INFO:tasks.workunit.client.0.smithi063.stdout:6/290: creat db/d1d/d20/d3e/d47/f5c x:0 0 0 2024-04-24T22:45:22.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/237: mknod d2/d9/d25/d2e/c59 0 2024-04-24T22:45:22.865 INFO:tasks.workunit.client.1.smithi192.stdout:1/213: getdents d9/d15 0 2024-04-24T22:45:22.866 INFO:tasks.workunit.client.0.smithi063.stdout:8/330: unlink d1/d2/d17/l44 0 2024-04-24T22:45:22.868 INFO:tasks.workunit.client.0.smithi063.stdout:6/291: dwrite db/d1d/f4c [4194304,4194304] 0 2024-04-24T22:45:22.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/239: dread d0/d4/f3a [0,4194304] 0 2024-04-24T22:45:22.886 INFO:tasks.workunit.client.0.smithi063.stdout:7/240: stat d0/d2f 0 2024-04-24T22:45:22.888 INFO:tasks.workunit.client.0.smithi063.stdout:0/274: sync 2024-04-24T22:45:22.888 INFO:tasks.workunit.client.0.smithi063.stdout:5/317: sync 2024-04-24T22:45:22.888 INFO:tasks.workunit.client.0.smithi063.stdout:9/297: sync 2024-04-24T22:45:22.888 INFO:tasks.workunit.client.0.smithi063.stdout:4/286: sync 2024-04-24T22:45:22.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/298: rename d6/d8/db to d6/d8/db/d2b/d48/d63 22 2024-04-24T22:45:22.908 INFO:tasks.workunit.client.1.smithi192.stdout:6/232: sync 2024-04-24T22:45:22.908 INFO:tasks.workunit.client.1.smithi192.stdout:8/217: sync 2024-04-24T22:45:22.908 INFO:tasks.workunit.client.1.smithi192.stdout:5/199: sync 2024-04-24T22:45:22.909 INFO:tasks.workunit.client.1.smithi192.stdout:5/200: stat d0/d13/d19/f1a 0 2024-04-24T22:45:22.910 INFO:tasks.workunit.client.1.smithi192.stdout:9/229: creat d3/dd/de/d17/d33/f50 x:0 0 0 2024-04-24T22:45:22.911 INFO:tasks.workunit.client.1.smithi192.stdout:5/201: fdatasync d0/d1b/d1c/d2b/f3a 0 2024-04-24T22:45:22.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/233: creat d0/d8/d3d/f55 x:0 0 0 2024-04-24T22:45:22.917 INFO:tasks.workunit.client.0.smithi063.stdout:1/304: unlink d3/da/f3d 0 2024-04-24T22:45:22.918 INFO:tasks.workunit.client.0.smithi063.stdout:1/305: stat d3/d2d/f52 0 2024-04-24T22:45:22.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/218: mknod db/dc/d29/c40 0 2024-04-24T22:45:22.919 INFO:tasks.workunit.client.0.smithi063.stdout:3/308: write da/d18/f19 [443810,9505] 0 2024-04-24T22:45:22.919 INFO:tasks.workunit.client.1.smithi192.stdout:8/219: write db/dc/f1c [3746904,105629] 0 2024-04-24T22:45:22.922 INFO:tasks.workunit.client.1.smithi192.stdout:9/230: rmdir d3/dd/de 39 2024-04-24T22:45:22.922 INFO:tasks.workunit.client.0.smithi063.stdout:3/309: dread da/f31 [0,4194304] 0 2024-04-24T22:45:22.925 INFO:tasks.workunit.client.1.smithi192.stdout:5/202: rename d0/d1b/d1c/d2b to d0/d13/d3b 0 2024-04-24T22:45:22.927 INFO:tasks.workunit.client.1.smithi192.stdout:8/220: dwrite db/dc/d22/f28 [0,4194304] 0 2024-04-24T22:45:22.927 INFO:tasks.workunit.client.1.smithi192.stdout:8/221: fdatasync f3 0 2024-04-24T22:45:22.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/321: link d4/d9/d10/c43 d4/d9/d10/c5f 0 2024-04-24T22:45:22.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/222: read db/dc/d22/f39 [43636,2538] 0 2024-04-24T22:45:22.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/322: chown d4/d9/d10/c5f 821846691 1 2024-04-24T22:45:22.930 INFO:tasks.workunit.client.0.smithi063.stdout:4/287: mkdir d2/d8/d40/d51 0 2024-04-24T22:45:22.931 INFO:tasks.workunit.client.0.smithi063.stdout:0/275: mkdir d6/da/d2b/d60 0 2024-04-24T22:45:22.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/318: mkdir d5/d24/d44/d71 0 2024-04-24T22:45:22.933 INFO:tasks.workunit.client.0.smithi063.stdout:6/292: write db/d1d/d20/d3e/f43 [1149414,125129] 0 2024-04-24T22:45:22.935 INFO:tasks.workunit.client.0.smithi063.stdout:9/299: symlink d6/d8/db/d2b/l64 0 2024-04-24T22:45:22.936 INFO:tasks.workunit.client.0.smithi063.stdout:9/300: chown d6/d8/db/d2b/f3d 55808927 1 2024-04-24T22:45:22.937 INFO:tasks.workunit.client.1.smithi192.stdout:9/231: truncate d3/dd/de/f2f 916597 0 2024-04-24T22:45:22.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/232: write d3/dd/de/d17/d33/f50 [231526,112146] 0 2024-04-24T22:45:22.939 INFO:tasks.workunit.client.0.smithi063.stdout:1/306: symlink d3/da/dc/d11/l64 0 2024-04-24T22:45:22.945 INFO:tasks.workunit.client.1.smithi192.stdout:6/234: creat d0/d8/d29/d4c/d4d/f56 x:0 0 0 2024-04-24T22:45:22.946 INFO:tasks.workunit.client.0.smithi063.stdout:3/310: unlink da/f21 0 2024-04-24T22:45:22.948 INFO:tasks.workunit.client.1.smithi192.stdout:5/203: mkdir d0/d1b/d1c/d2c/d3c 0 2024-04-24T22:45:22.950 INFO:tasks.workunit.client.1.smithi192.stdout:9/233: fdatasync d3/fa 0 2024-04-24T22:45:22.951 INFO:tasks.workunit.client.1.smithi192.stdout:9/234: read - d3/dd/de/f19 zero size 2024-04-24T22:45:22.954 INFO:tasks.workunit.client.0.smithi063.stdout:3/311: dwrite da/f29 [0,4194304] 0 2024-04-24T22:45:22.956 INFO:tasks.workunit.client.1.smithi192.stdout:2/323: rename d4/d9/d36 to d4/d29/d60 0 2024-04-24T22:45:22.957 INFO:tasks.workunit.client.0.smithi063.stdout:2/310: getdents d0/d2/d2d 0 2024-04-24T22:45:22.958 INFO:tasks.workunit.client.1.smithi192.stdout:6/235: rmdir d0/d8/d29/d4c 39 2024-04-24T22:45:22.959 INFO:tasks.workunit.client.0.smithi063.stdout:2/311: write d0/d2/d2d/f4d [924159,26184] 0 2024-04-24T22:45:22.960 INFO:tasks.workunit.client.1.smithi192.stdout:8/223: dwrite f8 [0,4194304] 0 2024-04-24T22:45:22.960 INFO:tasks.workunit.client.1.smithi192.stdout:9/235: rmdir d3/d24 39 2024-04-24T22:45:22.966 INFO:tasks.workunit.client.1.smithi192.stdout:2/324: dwrite d4/d9/d10/f5d [0,4194304] 0 2024-04-24T22:45:22.967 INFO:tasks.workunit.client.1.smithi192.stdout:6/236: write d0/d8/f14 [5000713,20879] 0 2024-04-24T22:45:22.970 INFO:tasks.workunit.client.1.smithi192.stdout:9/236: rmdir d3/dd 39 2024-04-24T22:45:22.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/224: unlink db/f14 0 2024-04-24T22:45:22.976 INFO:tasks.workunit.client.0.smithi063.stdout:0/276: creat d6/d5b/f61 x:0 0 0 2024-04-24T22:45:22.977 INFO:tasks.workunit.client.1.smithi192.stdout:8/225: dread f8 [0,4194304] 0 2024-04-24T22:45:22.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/319: creat d5/d24/d44/d71/f72 x:0 0 0 2024-04-24T22:45:22.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/325: readlink d4/d9/l2e 0 2024-04-24T22:45:22.980 INFO:tasks.workunit.client.0.smithi063.stdout:3/312: truncate da/d39/f41 927846 0 2024-04-24T22:45:22.982 INFO:tasks.workunit.client.0.smithi063.stdout:1/307: getdents d3/da/dc/d11/d5e 0 2024-04-24T22:45:22.984 INFO:tasks.workunit.client.0.smithi063.stdout:2/312: write d0/d2/d54/d1c/f2a [2996830,90635] 0 2024-04-24T22:45:22.985 INFO:tasks.workunit.client.0.smithi063.stdout:2/313: readlink d0/d2/d27/l59 0 2024-04-24T22:45:22.986 INFO:tasks.workunit.client.0.smithi063.stdout:8/331: getdents d1/d22/d33 0 2024-04-24T22:45:22.988 INFO:tasks.workunit.client.1.smithi192.stdout:8/226: unlink db/dc/d22/c30 0 2024-04-24T22:45:22.990 INFO:tasks.workunit.client.0.smithi063.stdout:4/288: rename d2/ca to d2/d8/c52 0 2024-04-24T22:45:22.990 INFO:tasks.workunit.client.0.smithi063.stdout:4/289: stat d2/d8/d36/d3a/d50 0 2024-04-24T22:45:22.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/237: symlink d0/d8/l57 0 2024-04-24T22:45:22.994 INFO:tasks.workunit.client.0.smithi063.stdout:3/313: creat da/d18/d47/f6c x:0 0 0 2024-04-24T22:45:22.996 INFO:tasks.workunit.client.0.smithi063.stdout:4/290: dwrite d2/fb [0,4194304] 0 2024-04-24T22:45:22.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/291: chown d2/d19/f25 1466 1 2024-04-24T22:45:23.000 INFO:tasks.workunit.client.1.smithi192.stdout:6/238: dread d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:23.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/239: readlink d0/d8/l57 0 2024-04-24T22:45:23.003 INFO:tasks.workunit.client.0.smithi063.stdout:4/292: dwrite d2/d8/d36/f44 [0,4194304] 0 2024-04-24T22:45:23.007 INFO:tasks.workunit.client.1.smithi192.stdout:2/326: link d4/d9/cd d4/d29/d60/c61 0 2024-04-24T22:45:23.011 INFO:tasks.workunit.client.0.smithi063.stdout:5/320: mkdir d5/d11/d23/d2d/d62/d73 0 2024-04-24T22:45:23.013 INFO:tasks.workunit.client.0.smithi063.stdout:3/314: symlink da/d18/d47/l6d 0 2024-04-24T22:45:23.015 INFO:tasks.workunit.client.1.smithi192.stdout:2/327: rmdir d4/d9/d10 39 2024-04-24T22:45:23.016 INFO:tasks.workunit.client.0.smithi063.stdout:7/241: sync 2024-04-24T22:45:23.016 INFO:tasks.workunit.client.0.smithi063.stdout:6/293: sync 2024-04-24T22:45:23.017 INFO:tasks.workunit.client.1.smithi192.stdout:8/227: dwrite db/dc/f18 [0,4194304] 0 2024-04-24T22:45:23.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/228: write db/f3d [468907,20552] 0 2024-04-24T22:45:23.019 INFO:tasks.workunit.client.1.smithi192.stdout:6/240: mkdir d0/d8/d29/d58 0 2024-04-24T22:45:23.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/277: truncate d6/dd/f4d 2485013 0 2024-04-24T22:45:23.020 INFO:tasks.workunit.client.1.smithi192.stdout:6/241: readlink d0/l15 0 2024-04-24T22:45:23.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/229: readlink db/dc/d29/l36 0 2024-04-24T22:45:23.024 INFO:tasks.workunit.client.1.smithi192.stdout:2/328: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:23.027 INFO:tasks.workunit.client.1.smithi192.stdout:8/230: creat db/dc/f41 x:0 0 0 2024-04-24T22:45:23.029 INFO:tasks.workunit.client.1.smithi192.stdout:2/329: chown d4/d9/f18 6952740 1 2024-04-24T22:45:23.032 INFO:tasks.workunit.client.0.smithi063.stdout:3/315: creat da/d18/d3a/d63/f6e x:0 0 0 2024-04-24T22:45:23.033 INFO:tasks.workunit.client.0.smithi063.stdout:3/316: chown f7 876 1 2024-04-24T22:45:23.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/231: dwrite db/f3d [0,4194304] 0 2024-04-24T22:45:23.034 INFO:tasks.workunit.client.0.smithi063.stdout:2/314: creat d0/f60 x:0 0 0 2024-04-24T22:45:23.035 INFO:tasks.workunit.client.0.smithi063.stdout:2/315: chown d0/d2/d54/d1c/d3f/f42 466 1 2024-04-24T22:45:23.037 INFO:tasks.workunit.client.1.smithi192.stdout:2/330: dread d4/d9/d10/f58 [0,4194304] 0 2024-04-24T22:45:23.040 INFO:tasks.workunit.client.1.smithi192.stdout:2/331: write d4/d9/d10/f1e [1402708,120839] 0 2024-04-24T22:45:23.040 INFO:tasks.workunit.client.0.smithi063.stdout:6/294: mkdir db/d1d/d20/d29/d5d 0 2024-04-24T22:45:23.041 INFO:tasks.workunit.client.0.smithi063.stdout:2/316: dwrite d0/d2/d54/d1c/d3f/f42 [0,4194304] 0 2024-04-24T22:45:23.041 INFO:tasks.workunit.client.1.smithi192.stdout:2/332: readlink d4/d29/d60/l56 0 2024-04-24T22:45:23.042 INFO:tasks.workunit.client.0.smithi063.stdout:2/317: chown d0/d2/d54/d7/c2e 81151 1 2024-04-24T22:45:23.043 INFO:tasks.workunit.client.0.smithi063.stdout:0/278: stat d6/dd/d20/d21/d25/d31/l50 0 2024-04-24T22:45:23.043 INFO:tasks.workunit.client.1.smithi192.stdout:8/232: mknod db/c42 0 2024-04-24T22:45:23.047 INFO:tasks.workunit.client.1.smithi192.stdout:2/333: creat d4/d9/d37/f62 x:0 0 0 2024-04-24T22:45:23.049 INFO:tasks.workunit.client.0.smithi063.stdout:3/317: unlink da/d40/f53 0 2024-04-24T22:45:23.050 INFO:tasks.workunit.client.1.smithi192.stdout:8/233: mkdir db/d43 0 2024-04-24T22:45:23.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/279: rmdir d6/dd 39 2024-04-24T22:45:23.056 INFO:tasks.workunit.client.1.smithi192.stdout:2/334: dwrite d4/d9/d37/f4e [0,4194304] 0 2024-04-24T22:45:23.082 INFO:tasks.workunit.client.1.smithi192.stdout:8/234: creat db/dc/f44 x:0 0 0 2024-04-24T22:45:23.083 INFO:tasks.workunit.client.0.smithi063.stdout:8/332: link d1/l4e d1/d22/l6e 0 2024-04-24T22:45:23.083 INFO:tasks.workunit.client.1.smithi192.stdout:8/235: chown db/l32 114 1 2024-04-24T22:45:23.085 INFO:tasks.workunit.client.0.smithi063.stdout:9/301: rename d6/d8/d4d/l1a to d6/d8/d1f/d20/d22/l65 0 2024-04-24T22:45:23.089 INFO:tasks.workunit.client.0.smithi063.stdout:3/318: creat da/d18/d47/d4d/d62/f6f x:0 0 0 2024-04-24T22:45:23.095 INFO:tasks.workunit.client.0.smithi063.stdout:1/308: rename d3/da/dc/d1f/f3f to d3/da/dc/d39/f65 0 2024-04-24T22:45:23.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/318: fdatasync d0/d2/d54/d1c/d3f/f42 0 2024-04-24T22:45:23.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/293: getdents d2/d8/d40 0 2024-04-24T22:45:23.097 INFO:tasks.workunit.client.0.smithi063.stdout:1/309: dread - d3/da/dc/d39/f65 zero size 2024-04-24T22:45:23.097 INFO:tasks.workunit.client.0.smithi063.stdout:3/319: symlink da/d18/d47/d4d/l70 0 2024-04-24T22:45:23.099 INFO:tasks.workunit.client.0.smithi063.stdout:3/320: write da/d18/d3a/d63/f6e [815796,129010] 0 2024-04-24T22:45:23.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/319: dread d0/d2/f1d [0,4194304] 0 2024-04-24T22:45:23.104 INFO:tasks.workunit.client.1.smithi192.stdout:8/236: dwrite f7 [4194304,4194304] 0 2024-04-24T22:45:23.105 INFO:tasks.workunit.client.0.smithi063.stdout:8/333: truncate d1/d11/d49/d54/f60 74185 0 2024-04-24T22:45:23.106 INFO:tasks.workunit.client.1.smithi192.stdout:2/335: dwrite d4/f41 [0,4194304] 0 2024-04-24T22:45:23.106 INFO:tasks.workunit.client.1.smithi192.stdout:2/336: readlink d4/d29/l4d 0 2024-04-24T22:45:23.107 INFO:tasks.workunit.client.0.smithi063.stdout:0/280: mkdir d6/dd/d20/d21/d25/d62 0 2024-04-24T22:45:23.107 INFO:tasks.workunit.client.0.smithi063.stdout:8/334: chown d1/d2/d17/c2d 4676072 1 2024-04-24T22:45:23.109 INFO:tasks.workunit.client.0.smithi063.stdout:1/310: creat d3/da/dc/d39/f66 x:0 0 0 2024-04-24T22:45:23.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/237: creat db/dc/d22/d31/f45 x:0 0 0 2024-04-24T22:45:23.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/294: dwrite d2/f5 [4194304,4194304] 0 2024-04-24T22:45:23.111 INFO:tasks.workunit.client.0.smithi063.stdout:0/281: write d6/da/f14 [4906220,9292] 0 2024-04-24T22:45:23.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/238: dread - db/dc/f44 zero size 2024-04-24T22:45:23.112 INFO:tasks.workunit.client.1.smithi192.stdout:8/239: readlink db/l19 0 2024-04-24T22:45:23.112 INFO:tasks.workunit.client.1.smithi192.stdout:8/240: stat db/dc/d22/f35 0 2024-04-24T22:45:23.113 INFO:tasks.workunit.client.0.smithi063.stdout:1/311: write d3/da/dc/d1f/f30 [4429410,52253] 0 2024-04-24T22:45:23.113 INFO:tasks.workunit.client.0.smithi063.stdout:3/321: fdatasync da/d18/d3a/d63/f64 0 2024-04-24T22:45:23.115 INFO:tasks.workunit.client.1.smithi192.stdout:2/337: rename d4/f59 to d4/d29/d60/f63 0 2024-04-24T22:45:23.118 INFO:tasks.workunit.client.0.smithi063.stdout:1/312: chown d3/da/l55 606555 1 2024-04-24T22:45:23.119 INFO:tasks.workunit.client.1.smithi192.stdout:8/241: symlink db/dc/d29/l46 0 2024-04-24T22:45:23.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/322: chown da/f3b 8094025 1 2024-04-24T22:45:23.121 INFO:tasks.workunit.client.0.smithi063.stdout:5/321: rename d5/d24/d49/f4f to d5/d11/d23/d2d/d62/d73/f74 0 2024-04-24T22:45:23.122 INFO:tasks.workunit.client.1.smithi192.stdout:2/338: fdatasync d4/f13 0 2024-04-24T22:45:23.122 INFO:tasks.workunit.client.0.smithi063.stdout:8/335: dwrite d1/f58 [0,4194304] 0 2024-04-24T22:45:23.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/313: write d3/da/dc/d1f/f4d [5160342,25309] 0 2024-04-24T22:45:23.122 INFO:tasks.workunit.client.1.smithi192.stdout:2/339: fsync d4/d9/d10/f5d 0 2024-04-24T22:45:23.124 INFO:tasks.workunit.client.1.smithi192.stdout:2/340: dread d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:23.124 INFO:tasks.workunit.client.0.smithi063.stdout:1/314: chown d3/f17 81 1 2024-04-24T22:45:23.125 INFO:tasks.workunit.client.1.smithi192.stdout:8/242: creat db/dc/d29/f47 x:0 0 0 2024-04-24T22:45:23.126 INFO:tasks.workunit.client.0.smithi063.stdout:1/315: dread - d3/da/dc/d11/f61 zero size 2024-04-24T22:45:23.128 INFO:tasks.workunit.client.0.smithi063.stdout:4/295: symlink d2/d19/l53 0 2024-04-24T22:45:23.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/341: creat d4/d9/d37/f64 x:0 0 0 2024-04-24T22:45:23.133 INFO:tasks.workunit.client.0.smithi063.stdout:9/302: getdents d6/d8 0 2024-04-24T22:45:23.134 INFO:tasks.workunit.client.0.smithi063.stdout:3/323: dwrite da/d18/f19 [0,4194304] 0 2024-04-24T22:45:23.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/342: rename d4/d9/l5c to d4/d9/d37/l65 0 2024-04-24T22:45:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/336: creat d1/d21/f6f x:0 0 0 2024-04-24T22:45:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:9/303: chown d6/d8/db/d2b/l64 2593892 1 2024-04-24T22:45:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/337: chown d1/f1f 575435 1 2024-04-24T22:45:23.138 INFO:tasks.workunit.client.0.smithi063.stdout:9/304: chown d6/d8/l16 1070500927 1 2024-04-24T22:45:23.138 INFO:tasks.workunit.client.0.smithi063.stdout:8/338: dread - d1/d22/f5b zero size 2024-04-24T22:45:23.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/295: rename db/d1d/d20/d3e/d47/f5c to db/d1d/d20/d3e/f5e 0 2024-04-24T22:45:23.140 INFO:tasks.workunit.client.1.smithi192.stdout:2/343: mknod d4/d9/d37/c66 0 2024-04-24T22:45:23.141 INFO:tasks.workunit.client.0.smithi063.stdout:8/339: truncate d1/d11/f29 4627607 0 2024-04-24T22:45:23.141 INFO:tasks.workunit.client.1.smithi192.stdout:2/344: chown d4/d9/d10/l32 1 1 2024-04-24T22:45:23.142 INFO:tasks.workunit.client.0.smithi063.stdout:4/296: write d2/d8/d36/d3a/f4b [1285262,59328] 0 2024-04-24T22:45:23.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/340: write d1/d11/d3c/f55 [585234,74495] 0 2024-04-24T22:45:23.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/282: mkdir d6/dd/d20/d21/d25/d62/d63 0 2024-04-24T22:45:23.146 INFO:tasks.workunit.client.0.smithi063.stdout:3/324: symlink da/d40/l71 0 2024-04-24T22:45:23.146 INFO:tasks.workunit.client.0.smithi063.stdout:0/283: chown d6/da/d2b/l4a 6435 1 2024-04-24T22:45:23.148 INFO:tasks.workunit.client.1.smithi192.stdout:2/345: dread d4/d9/f52 [0,4194304] 0 2024-04-24T22:45:23.148 INFO:tasks.workunit.client.0.smithi063.stdout:1/316: symlink d3/da/d34/l67 0 2024-04-24T22:45:23.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/317: chown d3/da/dc/c16 72 1 2024-04-24T22:45:23.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/318: write d3/da/dc/d1f/f30 [19632,60272] 0 2024-04-24T22:45:23.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/305: creat d6/d8/d1f/d20/d2a/f66 x:0 0 0 2024-04-24T22:45:23.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/346: rmdir d4/d29 39 2024-04-24T22:45:23.157 INFO:tasks.workunit.client.0.smithi063.stdout:7/242: sync 2024-04-24T22:45:23.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/347: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:23.158 INFO:tasks.workunit.client.0.smithi063.stdout:0/284: unlink d6/da/dc/f17 0 2024-04-24T22:45:23.159 INFO:tasks.workunit.client.0.smithi063.stdout:0/285: stat d6/da/dc/d26/l3c 0 2024-04-24T22:45:23.159 INFO:tasks.workunit.client.0.smithi063.stdout:3/325: creat da/d18/d47/d4d/f72 x:0 0 0 2024-04-24T22:45:23.160 INFO:tasks.workunit.client.0.smithi063.stdout:6/296: dwrite db/d1d/d20/d3e/f5e [0,4194304] 0 2024-04-24T22:45:23.161 INFO:tasks.workunit.client.0.smithi063.stdout:6/297: chown db/d1d/d20 108914 1 2024-04-24T22:45:23.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/286: truncate d6/da/f3d 174310 0 2024-04-24T22:45:23.162 INFO:tasks.workunit.client.0.smithi063.stdout:1/319: mkdir d3/da/d34/d68 0 2024-04-24T22:45:23.162 INFO:tasks.workunit.client.0.smithi063.stdout:6/298: chown db/f2a 1841657 1 2024-04-24T22:45:23.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/348: mknod d4/d9/d37/c67 0 2024-04-24T22:45:23.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/349: read - d4/d9/d37/f44 zero size 2024-04-24T22:45:23.165 INFO:tasks.workunit.client.0.smithi063.stdout:9/306: unlink d6/d8/d1f/d20/f3f 0 2024-04-24T22:45:23.166 INFO:tasks.workunit.client.0.smithi063.stdout:4/297: symlink d2/d8/d36/d3a/d50/l54 0 2024-04-24T22:45:23.168 INFO:tasks.workunit.client.0.smithi063.stdout:5/322: getdents d5/d24/d44/d71 0 2024-04-24T22:45:23.170 INFO:tasks.workunit.client.1.smithi192.stdout:2/350: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:23.171 INFO:tasks.workunit.client.0.smithi063.stdout:5/323: chown d5/d11/f2f 936 1 2024-04-24T22:45:23.171 INFO:tasks.workunit.client.0.smithi063.stdout:9/307: dread d6/d8/db/d2b/f3d [0,4194304] 0 2024-04-24T22:45:23.174 INFO:tasks.workunit.client.0.smithi063.stdout:7/243: write d0/d4/de/d17/f42 [3926431,91470] 0 2024-04-24T22:45:23.174 INFO:tasks.workunit.client.0.smithi063.stdout:9/308: readlink d6/d8/db/d1c/l43 0 2024-04-24T22:45:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:22 smithi192 ceph-mon[33991]: pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 9.7 MiB/s rd, 138 MiB/s wr, 259 op/s 2024-04-24T22:45:23.179 INFO:tasks.workunit.client.0.smithi063.stdout:6/299: mkdir db/d1d/d20/d3f/d5f 0 2024-04-24T22:45:23.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/300: truncate db/f1a 5148655 0 2024-04-24T22:45:23.180 INFO:tasks.workunit.client.1.smithi192.stdout:2/351: symlink d4/d9/l68 0 2024-04-24T22:45:23.184 INFO:tasks.workunit.client.0.smithi063.stdout:4/298: creat d2/d8/d40/f55 x:0 0 0 2024-04-24T22:45:23.188 INFO:tasks.workunit.client.1.smithi192.stdout:2/352: link d4/d9/d10/f53 d4/f69 0 2024-04-24T22:45:23.188 INFO:tasks.workunit.client.1.smithi192.stdout:2/353: dread - d4/d9/f5a zero size 2024-04-24T22:45:23.189 INFO:tasks.workunit.client.0.smithi063.stdout:8/341: truncate d1/d11/d49/d54/f60 634689 0 2024-04-24T22:45:23.189 INFO:tasks.workunit.client.0.smithi063.stdout:7/244: rmdir d0/d4/de/d17 39 2024-04-24T22:45:23.190 INFO:tasks.workunit.client.0.smithi063.stdout:9/309: readlink d6/d8/d1f/d20/d22/l4c 0 2024-04-24T22:45:23.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/310: stat d6/l13 0 2024-04-24T22:45:23.192 INFO:tasks.workunit.client.0.smithi063.stdout:6/301: creat db/d1d/d2d/f60 x:0 0 0 2024-04-24T22:45:23.194 INFO:tasks.workunit.client.0.smithi063.stdout:4/299: rmdir d2/d19 39 2024-04-24T22:45:23.197 INFO:tasks.workunit.client.0.smithi063.stdout:5/324: link d5/f9 d5/d11/d23/d2d/d62/d73/f75 0 2024-04-24T22:45:23.197 INFO:tasks.workunit.client.0.smithi063.stdout:5/325: stat d5/d11/d33/f3f 0 2024-04-24T22:45:23.199 INFO:tasks.workunit.client.0.smithi063.stdout:5/326: truncate d5/d24/f51 429703 0 2024-04-24T22:45:23.199 INFO:tasks.workunit.client.0.smithi063.stdout:8/342: write d1/d11/f29 [3209331,37886] 0 2024-04-24T22:45:23.199 INFO:tasks.workunit.client.0.smithi063.stdout:9/311: symlink d6/d8/db/d1c/l67 0 2024-04-24T22:45:23.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/354: getdents d4/d29 0 2024-04-24T22:45:23.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/320: creat d3/da/dc/d1f/d21/f69 x:0 0 0 2024-04-24T22:45:23.202 INFO:tasks.workunit.client.0.smithi063.stdout:8/343: readlink d1/l1d 0 2024-04-24T22:45:23.203 INFO:tasks.workunit.client.0.smithi063.stdout:0/287: getdents d6/dd/d20/d21/d25/d62 0 2024-04-24T22:45:23.205 INFO:tasks.workunit.client.0.smithi063.stdout:9/312: creat d6/d8/d1f/d20/d22/d31/f68 x:0 0 0 2024-04-24T22:45:23.205 INFO:tasks.workunit.client.1.smithi192.stdout:3/273: sync 2024-04-24T22:45:23.205 INFO:tasks.workunit.client.1.smithi192.stdout:7/250: sync 2024-04-24T22:45:23.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/302: mknod db/d1d/d20/d3f/d5f/c61 0 2024-04-24T22:45:23.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/303: stat db/d1d/d20/d29 0 2024-04-24T22:45:23.207 INFO:tasks.workunit.client.1.smithi192.stdout:2/355: dwrite d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:23.207 INFO:tasks.workunit.client.0.smithi063.stdout:1/321: stat d3/da/d34/l50 0 2024-04-24T22:45:23.207 INFO:tasks.workunit.client.1.smithi192.stdout:7/251: write d2/db/d36/f38 [769027,72887] 0 2024-04-24T22:45:23.208 INFO:tasks.workunit.client.0.smithi063.stdout:1/322: stat d3/da/dc/d11/f35 0 2024-04-24T22:45:23.209 INFO:tasks.workunit.client.1.smithi192.stdout:2/356: read - d4/d29/d60/f57 zero size 2024-04-24T22:45:23.209 INFO:tasks.workunit.client.1.smithi192.stdout:7/252: fsync d2/db/f45 0 2024-04-24T22:45:23.209 INFO:tasks.workunit.client.0.smithi063.stdout:5/327: mkdir d5/d11/d76 0 2024-04-24T22:45:23.210 INFO:tasks.workunit.client.1.smithi192.stdout:7/253: chown d2/db/ld 3620522 1 2024-04-24T22:45:23.210 INFO:tasks.workunit.client.1.smithi192.stdout:2/357: write d4/d9/f5b [1001306,95759] 0 2024-04-24T22:45:23.211 INFO:tasks.workunit.client.1.smithi192.stdout:2/358: write d4/d29/d60/f57 [841998,50089] 0 2024-04-24T22:45:23.212 INFO:tasks.workunit.client.0.smithi063.stdout:6/304: creat db/d1d/d20/d3e/d47/f62 x:0 0 0 2024-04-24T22:45:23.213 INFO:tasks.workunit.client.0.smithi063.stdout:1/323: rename d3/da/f1b to d3/da/f6a 0 2024-04-24T22:45:23.214 INFO:tasks.workunit.client.0.smithi063.stdout:5/328: mknod d5/d11/d23/d2d/c77 0 2024-04-24T22:45:23.216 INFO:tasks.workunit.client.0.smithi063.stdout:9/313: link d6/d8/db/d2b/f34 d6/d8/db/d1c/f69 0 2024-04-24T22:45:23.216 INFO:tasks.workunit.client.0.smithi063.stdout:1/324: chown d3/da/dc/d39/f65 362 1 2024-04-24T22:45:23.216 INFO:tasks.workunit.client.1.smithi192.stdout:2/359: mknod d4/d9/d37/c6a 0 2024-04-24T22:45:23.217 INFO:tasks.workunit.client.0.smithi063.stdout:5/329: chown d5/d24/d44/f5c 5295 1 2024-04-24T22:45:23.217 INFO:tasks.workunit.client.0.smithi063.stdout:8/344: dwrite d1/d2/f1c [0,4194304] 0 2024-04-24T22:45:23.218 INFO:tasks.workunit.client.0.smithi063.stdout:4/300: getdents d2/d8/d36 0 2024-04-24T22:45:23.220 INFO:tasks.workunit.client.1.smithi192.stdout:2/360: dread d4/f69 [0,4194304] 0 2024-04-24T22:45:23.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/325: fdatasync d3/da/dc/d11/f41 0 2024-04-24T22:45:23.223 INFO:tasks.workunit.client.0.smithi063.stdout:6/305: dwrite db/d1d/d20/f38 [0,4194304] 0 2024-04-24T22:45:23.223 INFO:tasks.workunit.client.1.smithi192.stdout:2/361: chown d4/d9/l4b 7087599 1 2024-04-24T22:45:23.226 INFO:tasks.workunit.client.0.smithi063.stdout:5/330: rename d5/d24/f32 to d5/d11/d33/d67/f78 0 2024-04-24T22:45:23.227 INFO:tasks.workunit.client.0.smithi063.stdout:8/345: mkdir d1/d70 0 2024-04-24T22:45:23.228 INFO:tasks.workunit.client.0.smithi063.stdout:2/320: sync 2024-04-24T22:45:23.228 INFO:tasks.workunit.client.1.smithi192.stdout:2/362: unlink d4/d9/d10/f11 0 2024-04-24T22:45:23.235 INFO:tasks.workunit.client.1.smithi192.stdout:2/363: getdents d4/d9 0 2024-04-24T22:45:23.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/306: rename db/d1d/d2d/f52 to db/d1d/d2d/d48/f63 0 2024-04-24T22:45:23.241 INFO:tasks.workunit.client.0.smithi063.stdout:1/326: dwrite d3/da/dc/d1f/f30 [4194304,4194304] 0 2024-04-24T22:45:23.241 INFO:tasks.workunit.client.0.smithi063.stdout:9/314: link d6/d8/d1f/d20/d22/d31/f33 d6/d8/db/d2b/f6a 0 2024-04-24T22:45:23.244 INFO:tasks.workunit.client.0.smithi063.stdout:9/315: dread - d6/d8/db/f62 zero size 2024-04-24T22:45:23.245 INFO:tasks.workunit.client.0.smithi063.stdout:2/321: creat d0/d2/d51/f61 x:0 0 0 2024-04-24T22:45:23.246 INFO:tasks.workunit.client.1.smithi192.stdout:3/274: read d2/f1e [67675,28865] 0 2024-04-24T22:45:23.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/346: dwrite d1/d21/f6f [0,4194304] 0 2024-04-24T22:45:23.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/307: creat db/d1c/f64 x:0 0 0 2024-04-24T22:45:23.249 INFO:tasks.workunit.client.0.smithi063.stdout:5/331: creat d5/d11/d23/f79 x:0 0 0 2024-04-24T22:45:23.253 INFO:tasks.workunit.client.0.smithi063.stdout:5/332: chown d5/f5b 1176800951 1 2024-04-24T22:45:23.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/333: mknod d5/d11/d33/c7a 0 2024-04-24T22:45:23.261 INFO:tasks.workunit.client.0.smithi063.stdout:8/347: dwrite d1/d11/d3c/f55 [0,4194304] 0 2024-04-24T22:45:23.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/316: dread d6/d8/db/f10 [0,4194304] 0 2024-04-24T22:45:23.268 INFO:tasks.workunit.client.0.smithi063.stdout:2/322: dread d0/d2/d54/d7/f19 [0,4194304] 0 2024-04-24T22:45:23.270 INFO:tasks.workunit.client.0.smithi063.stdout:2/323: mkdir d0/dd/d62 0 2024-04-24T22:45:23.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/324: read - d0/d2/d51/f61 zero size 2024-04-24T22:45:23.275 INFO:tasks.workunit.client.0.smithi063.stdout:9/317: read d6/f14 [85852,7206] 0 2024-04-24T22:45:23.275 INFO:tasks.workunit.client.0.smithi063.stdout:2/325: getdents d0/d2/d2f 0 2024-04-24T22:45:23.276 INFO:tasks.workunit.client.0.smithi063.stdout:2/326: mkdir d0/d2/d63 0 2024-04-24T22:45:23.278 INFO:tasks.workunit.client.0.smithi063.stdout:2/327: write d0/d2/d54/d1c/d3f/f42 [2704486,130453] 0 2024-04-24T22:45:23.283 INFO:tasks.workunit.client.0.smithi063.stdout:2/328: chown d0/d2/d54/f25 228762907 1 2024-04-24T22:45:23.283 INFO:tasks.workunit.client.0.smithi063.stdout:9/318: dread d6/d8/d1f/d20/d22/d31/f33 [0,4194304] 0 2024-04-24T22:45:23.283 INFO:tasks.workunit.client.0.smithi063.stdout:2/329: rename d0/d2/f1d to d0/d2/d54/d7/f64 0 2024-04-24T22:45:23.285 INFO:tasks.workunit.client.0.smithi063.stdout:9/319: rename d6/d8/d4d to d6/d8/db/d6b 0 2024-04-24T22:45:23.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/330: truncate d0/d2/f1b 111090 0 2024-04-24T22:45:23.287 INFO:tasks.workunit.client.0.smithi063.stdout:9/320: creat d6/f6c x:0 0 0 2024-04-24T22:45:23.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/308: rmdir db/d1d/d20 39 2024-04-24T22:45:23.294 INFO:tasks.workunit.client.0.smithi063.stdout:6/309: symlink db/d1d/l65 0 2024-04-24T22:45:23.296 INFO:tasks.workunit.client.0.smithi063.stdout:6/310: mkdir db/d1d/d2d/d59/d66 0 2024-04-24T22:45:23.297 INFO:tasks.workunit.client.1.smithi192.stdout:2/364: rmdir d4/d9/d10 39 2024-04-24T22:45:23.299 INFO:tasks.workunit.client.0.smithi063.stdout:8/348: dread d1/d11/d3c/f6b [0,4194304] 0 2024-04-24T22:45:23.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/301: dwrite d2/d8/f2c [0,4194304] 0 2024-04-24T22:45:23.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/311: creat db/d1d/d2d/d59/d66/f67 x:0 0 0 2024-04-24T22:45:23.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:22 smithi063 ceph-mon[26912]: pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 9.7 MiB/s rd, 138 MiB/s wr, 259 op/s 2024-04-24T22:45:23.310 INFO:tasks.workunit.client.1.smithi192.stdout:2/365: rename d4/d9/d10/c22 to d4/d9/c6b 0 2024-04-24T22:45:23.311 INFO:tasks.workunit.client.1.smithi192.stdout:2/366: read d4/d29/d60/f63 [280518,10533] 0 2024-04-24T22:45:23.314 INFO:tasks.workunit.client.1.smithi192.stdout:2/367: mkdir d4/d9/d10/d6c 0 2024-04-24T22:45:23.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/368: truncate d4/d9/d37/f3b 3320096 0 2024-04-24T22:45:23.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/369: dread - d4/d9/d37/f44 zero size 2024-04-24T22:45:23.340 INFO:tasks.workunit.client.0.smithi063.stdout:3/326: sync 2024-04-24T22:45:23.353 INFO:tasks.workunit.client.0.smithi063.stdout:8/349: rename d1/d11/d3c/f6b to d1/d11/d49/d54/f71 0 2024-04-24T22:45:23.354 INFO:tasks.workunit.client.0.smithi063.stdout:1/327: write d3/f2c [1843925,34281] 0 2024-04-24T22:45:23.354 INFO:tasks.workunit.client.0.smithi063.stdout:3/327: write da/f3d [818089,58361] 0 2024-04-24T22:45:23.354 INFO:tasks.workunit.client.0.smithi063.stdout:8/350: stat d1/d11/d32 0 2024-04-24T22:45:23.355 INFO:tasks.workunit.client.0.smithi063.stdout:1/328: chown d3/da/f37 1 1 2024-04-24T22:45:23.356 INFO:tasks.workunit.client.0.smithi063.stdout:8/351: chown d1/d11/d3c/l52 0 1 2024-04-24T22:45:23.392 INFO:tasks.workunit.client.0.smithi063.stdout:6/312: fsync db/d1d/d20/d3e/f45 0 2024-04-24T22:45:23.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/302: truncate d2/d8/f1a 3545998 0 2024-04-24T22:45:23.396 INFO:tasks.workunit.client.0.smithi063.stdout:6/313: mkdir db/d1d/d2d/d59/d68 0 2024-04-24T22:45:23.398 INFO:tasks.workunit.client.0.smithi063.stdout:3/328: getdents da/d18/d47/d4d 0 2024-04-24T22:45:23.399 INFO:tasks.workunit.client.0.smithi063.stdout:1/329: getdents d3/da/dc/d11 0 2024-04-24T22:45:23.400 INFO:tasks.workunit.client.0.smithi063.stdout:6/314: mkdir db/d1d/d2d/d69 0 2024-04-24T22:45:23.401 INFO:tasks.workunit.client.0.smithi063.stdout:9/321: link d6/d8/db/f62 d6/d8/d1f/d20/d22/f6d 0 2024-04-24T22:45:23.403 INFO:tasks.workunit.client.0.smithi063.stdout:1/330: rename d3/f17 to d3/da/dc/d11/f6b 0 2024-04-24T22:45:23.404 INFO:tasks.workunit.client.0.smithi063.stdout:4/303: link d2/d19/d33/c4f d2/d19/c56 0 2024-04-24T22:45:23.404 INFO:tasks.workunit.client.0.smithi063.stdout:6/315: fsync db/d1d/d2d/d48/f51 0 2024-04-24T22:45:23.405 INFO:tasks.workunit.client.0.smithi063.stdout:6/316: stat c2 0 2024-04-24T22:45:23.405 INFO:tasks.workunit.client.0.smithi063.stdout:1/331: fsync d3/da/dc/d1f/f4d 0 2024-04-24T22:45:23.405 INFO:tasks.workunit.client.0.smithi063.stdout:1/332: fdatasync d3/f2c 0 2024-04-24T22:45:23.405 INFO:tasks.workunit.client.0.smithi063.stdout:6/317: chown db/ff 407472 1 2024-04-24T22:45:23.407 INFO:tasks.workunit.client.0.smithi063.stdout:9/322: rename d6/d8/db/f10 to d6/d8/d1f/d20/d2a/f6e 0 2024-04-24T22:45:23.411 INFO:tasks.workunit.client.0.smithi063.stdout:9/323: creat d6/d2f/d42/f6f x:0 0 0 2024-04-24T22:45:23.412 INFO:tasks.workunit.client.0.smithi063.stdout:9/324: chown d6/d8/db/d6b/f23 4065 1 2024-04-24T22:45:23.412 INFO:tasks.workunit.client.0.smithi063.stdout:3/329: dwrite da/d18/d3a/f3f [0,4194304] 0 2024-04-24T22:45:23.413 INFO:tasks.workunit.client.0.smithi063.stdout:9/325: readlink d6/d8/db/d1c/l67 0 2024-04-24T22:45:23.414 INFO:tasks.workunit.client.0.smithi063.stdout:3/330: write da/d18/f68 [717704,124157] 0 2024-04-24T22:45:23.414 INFO:tasks.workunit.client.0.smithi063.stdout:4/304: write d2/d8/f23 [383658,31801] 0 2024-04-24T22:45:23.416 INFO:tasks.workunit.client.0.smithi063.stdout:6/318: link db/d1d/d20/c4a db/d1d/d2d/d59/c6a 0 2024-04-24T22:45:23.416 INFO:tasks.workunit.client.0.smithi063.stdout:9/326: chown d6/d8/l29 14 1 2024-04-24T22:45:23.417 INFO:tasks.workunit.client.0.smithi063.stdout:1/333: truncate d3/f7 3227916 0 2024-04-24T22:45:23.418 INFO:tasks.workunit.client.0.smithi063.stdout:9/327: chown d6/d8/db/d6b/d4f/f5a 980308970 1 2024-04-24T22:45:23.419 INFO:tasks.workunit.client.0.smithi063.stdout:1/334: write d3/da/dc/f3a [774901,98611] 0 2024-04-24T22:45:23.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/328: stat d6/d2f/d42/f6f 0 2024-04-24T22:45:23.419 INFO:tasks.workunit.client.0.smithi063.stdout:4/305: mknod d2/c57 0 2024-04-24T22:45:23.420 INFO:tasks.workunit.client.0.smithi063.stdout:1/335: write d3/da/dc/d11/f61 [708789,8068] 0 2024-04-24T22:45:23.422 INFO:tasks.workunit.client.0.smithi063.stdout:6/319: creat db/d1d/d20/d29/d5d/f6b x:0 0 0 2024-04-24T22:45:23.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/320: dread - db/d1d/f4e zero size 2024-04-24T22:45:23.424 INFO:tasks.workunit.client.0.smithi063.stdout:4/306: symlink d2/d8/d36/l58 0 2024-04-24T22:45:23.425 INFO:tasks.workunit.client.0.smithi063.stdout:3/331: getdents da/d18/d3a/d63 0 2024-04-24T22:45:23.427 INFO:tasks.workunit.client.0.smithi063.stdout:3/332: write da/d18/d47/d4d/d62/f65 [676718,25284] 0 2024-04-24T22:45:23.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/307: mknod d2/d8/d36/c59 0 2024-04-24T22:45:23.430 INFO:tasks.workunit.client.0.smithi063.stdout:9/329: dread d6/d8/f5b [0,4194304] 0 2024-04-24T22:45:23.435 INFO:tasks.workunit.client.0.smithi063.stdout:3/333: dread da/d18/f61 [0,4194304] 0 2024-04-24T22:45:23.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/330: symlink d6/d8/d53/l70 0 2024-04-24T22:45:23.436 INFO:tasks.workunit.client.0.smithi063.stdout:4/308: rename d2/c12 to d2/d8/c5a 0 2024-04-24T22:45:23.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/336: write d3/da/dc/d1f/d21/d42/f58 [1064291,75185] 0 2024-04-24T22:45:23.437 INFO:tasks.workunit.client.0.smithi063.stdout:9/331: chown d6/d8/d1f/d20/c55 0 1 2024-04-24T22:45:23.437 INFO:tasks.workunit.client.0.smithi063.stdout:4/309: chown d2/d8/d36/d3a 19 1 2024-04-24T22:45:23.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/321: dread db/f2a [0,4194304] 0 2024-04-24T22:45:23.438 INFO:tasks.workunit.client.0.smithi063.stdout:1/337: truncate d3/f63 461677 0 2024-04-24T22:45:23.439 INFO:tasks.workunit.client.0.smithi063.stdout:9/332: rmdir d6/d8/db/d1c 39 2024-04-24T22:45:23.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/322: chown db/d1d/d20/d3e/d47/f62 69680197 1 2024-04-24T22:45:23.439 INFO:tasks.workunit.client.0.smithi063.stdout:1/338: rmdir d3/d2d 39 2024-04-24T22:45:23.441 INFO:tasks.workunit.client.0.smithi063.stdout:4/310: creat d2/d8/f5b x:0 0 0 2024-04-24T22:45:23.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/339: write d3/da/dc/f3a [295623,39430] 0 2024-04-24T22:45:23.441 INFO:tasks.workunit.client.0.smithi063.stdout:9/333: rmdir d6/d2f 39 2024-04-24T22:45:23.443 INFO:tasks.workunit.client.0.smithi063.stdout:7/245: sync 2024-04-24T22:45:23.443 INFO:tasks.workunit.client.0.smithi063.stdout:9/334: read d6/d8/db/d2b/f3d [415659,121284] 0 2024-04-24T22:45:23.443 INFO:tasks.workunit.client.0.smithi063.stdout:6/323: getdents db/d1d/d2d/d69 0 2024-04-24T22:45:23.445 INFO:tasks.workunit.client.0.smithi063.stdout:6/324: chown c4 1598089 1 2024-04-24T22:45:23.449 INFO:tasks.workunit.client.0.smithi063.stdout:4/311: dwrite d2/fd [4194304,4194304] 0 2024-04-24T22:45:23.451 INFO:tasks.workunit.client.0.smithi063.stdout:6/325: rename db/d1c/f64 to db/d1d/d2d/d69/f6c 0 2024-04-24T22:45:23.453 INFO:tasks.workunit.client.0.smithi063.stdout:7/246: dread d0/d4/fb [0,4194304] 0 2024-04-24T22:45:23.453 INFO:tasks.workunit.client.0.smithi063.stdout:9/335: dread d6/f30 [0,4194304] 0 2024-04-24T22:45:23.454 INFO:tasks.workunit.client.0.smithi063.stdout:6/326: fdatasync db/d1d/d20/f40 0 2024-04-24T22:45:23.454 INFO:tasks.workunit.client.0.smithi063.stdout:1/340: dwrite d3/da/dc/d1f/d2b/f32 [0,4194304] 0 2024-04-24T22:45:23.457 INFO:tasks.workunit.client.0.smithi063.stdout:9/336: fsync d6/f14 0 2024-04-24T22:45:23.458 INFO:tasks.workunit.client.0.smithi063.stdout:9/337: write d6/d8/db/d2b/d36/f40 [4110696,130102] 0 2024-04-24T22:45:23.463 INFO:tasks.workunit.client.0.smithi063.stdout:9/338: dread d6/d8/db/d6b/f23 [0,4194304] 0 2024-04-24T22:45:23.466 INFO:tasks.workunit.client.0.smithi063.stdout:6/327: creat db/d1d/d2d/d59/d68/f6d x:0 0 0 2024-04-24T22:45:23.466 INFO:tasks.workunit.client.0.smithi063.stdout:4/312: getdents d2/d8/d36/d3a 0 2024-04-24T22:45:23.468 INFO:tasks.workunit.client.0.smithi063.stdout:9/339: rmdir d6/d8/db/d2b 39 2024-04-24T22:45:23.472 INFO:tasks.workunit.client.0.smithi063.stdout:1/341: rename d3/da/f36 to d3/f6c 0 2024-04-24T22:45:23.474 INFO:tasks.workunit.client.0.smithi063.stdout:6/328: creat db/d1d/d2d/d48/f6e x:0 0 0 2024-04-24T22:45:23.474 INFO:tasks.workunit.client.0.smithi063.stdout:7/247: dwrite d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:23.476 INFO:tasks.workunit.client.0.smithi063.stdout:1/342: mkdir d3/da/d34/d6d 0 2024-04-24T22:45:23.477 INFO:tasks.workunit.client.0.smithi063.stdout:4/313: mkdir d2/d8/d5c 0 2024-04-24T22:45:23.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/314: chown d2/d8/d5c 3379 1 2024-04-24T22:45:23.478 INFO:tasks.workunit.client.0.smithi063.stdout:6/329: mkdir db/d1d/d20/d29/d5d/d6f 0 2024-04-24T22:45:23.478 INFO:tasks.workunit.client.0.smithi063.stdout:1/343: write d3/da/dc/d11/f41 [4823550,48626] 0 2024-04-24T22:45:23.480 INFO:tasks.workunit.client.0.smithi063.stdout:7/248: creat d0/d2f/f5a x:0 0 0 2024-04-24T22:45:23.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/344: dread - d3/da/dc/d39/f66 zero size 2024-04-24T22:45:23.487 INFO:tasks.workunit.client.0.smithi063.stdout:4/315: dread d2/fd [0,4194304] 0 2024-04-24T22:45:23.487 INFO:tasks.workunit.client.0.smithi063.stdout:4/316: read d2/ff [1666153,121599] 0 2024-04-24T22:45:23.487 INFO:tasks.workunit.client.0.smithi063.stdout:6/330: rename db/d1d/d2d/d59/d66 to db/d1d/d2d/d59/d68/d70 0 2024-04-24T22:45:23.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/345: mkdir d3/da/dc/d1f/d21/d42/d6e 0 2024-04-24T22:45:23.491 INFO:tasks.workunit.client.0.smithi063.stdout:6/331: write db/f37 [4627858,3762] 0 2024-04-24T22:45:23.493 INFO:tasks.workunit.client.0.smithi063.stdout:1/346: mkdir d3/da/dc/d11/d5e/d6f 0 2024-04-24T22:45:23.494 INFO:tasks.workunit.client.0.smithi063.stdout:6/332: truncate db/ff 1715090 0 2024-04-24T22:45:23.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/249: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:23.499 INFO:tasks.workunit.client.0.smithi063.stdout:6/333: rename db/d1d/d2d/f60 to db/d1d/d20/f71 0 2024-04-24T22:45:23.500 INFO:tasks.workunit.client.0.smithi063.stdout:1/347: dwrite d3/da/dc/d11/f35 [4194304,4194304] 0 2024-04-24T22:45:23.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/334: symlink db/d1d/d20/d3f/l72 0 2024-04-24T22:45:23.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/335: stat l9 0 2024-04-24T22:45:23.507 INFO:tasks.workunit.client.0.smithi063.stdout:7/250: getdents d0/d2f 0 2024-04-24T22:45:23.508 INFO:tasks.workunit.client.0.smithi063.stdout:1/348: dwrite d3/da/dc/d1f/d21/f69 [0,4194304] 0 2024-04-24T22:45:23.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/336: dread db/f37 [0,4194304] 0 2024-04-24T22:45:23.517 INFO:tasks.workunit.client.0.smithi063.stdout:7/251: creat d0/d26/f5b x:0 0 0 2024-04-24T22:45:23.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/349: dwrite d3/da/dc/d11/f41 [4194304,4194304] 0 2024-04-24T22:45:23.519 INFO:tasks.workunit.client.0.smithi063.stdout:7/252: chown d0/d26/f5b 34909 1 2024-04-24T22:45:23.521 INFO:tasks.workunit.client.0.smithi063.stdout:6/337: dwrite db/d1d/f49 [0,4194304] 0 2024-04-24T22:45:23.525 INFO:tasks.workunit.client.0.smithi063.stdout:6/338: chown db/d1d/d20/d3e/f5e 9439 1 2024-04-24T22:45:23.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/350: symlink d3/da/d34/d68/l70 0 2024-04-24T22:45:23.527 INFO:tasks.workunit.client.0.smithi063.stdout:1/351: chown d3/da/dc/d1f/d21/f69 1415 1 2024-04-24T22:45:23.528 INFO:tasks.workunit.client.0.smithi063.stdout:6/339: mknod db/d1d/d20/d3e/c73 0 2024-04-24T22:45:23.530 INFO:tasks.workunit.client.1.smithi192.stdout:0/341: sync 2024-04-24T22:45:23.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/238: sync 2024-04-24T22:45:23.531 INFO:tasks.workunit.client.1.smithi192.stdout:1/214: sync 2024-04-24T22:45:23.531 INFO:tasks.workunit.client.0.smithi063.stdout:1/352: creat d3/da/dc/d1f/f71 x:0 0 0 2024-04-24T22:45:23.532 INFO:tasks.workunit.client.0.smithi063.stdout:6/340: creat db/d1d/d20/d3e/f74 x:0 0 0 2024-04-24T22:45:23.534 INFO:tasks.workunit.client.0.smithi063.stdout:6/341: truncate db/d1d/d20/d3e/f43 1649807 0 2024-04-24T22:45:23.535 INFO:tasks.workunit.client.1.smithi192.stdout:0/342: creat d3/d6d/d79/f7a x:0 0 0 2024-04-24T22:45:23.538 INFO:tasks.workunit.client.1.smithi192.stdout:4/239: creat d2/d9/d25/f5a x:0 0 0 2024-04-24T22:45:23.540 INFO:tasks.workunit.client.1.smithi192.stdout:0/343: rename d3/d27/d3c/d3b/l6e to d3/d27/d28/d2e/d3f/l7b 0 2024-04-24T22:45:23.540 INFO:tasks.workunit.client.1.smithi192.stdout:4/240: write d2/d9/d1e/d28/d2f/f3c [1608168,110958] 0 2024-04-24T22:45:23.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/215: dwrite d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:23.542 INFO:tasks.workunit.client.0.smithi063.stdout:6/342: getdents db/d1d/d2d/d59/d68 0 2024-04-24T22:45:23.544 INFO:tasks.workunit.client.1.smithi192.stdout:0/344: symlink d3/d27/d3c/d3b/l7c 0 2024-04-24T22:45:23.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/216: mkdir d9/d15/d2d/d48 0 2024-04-24T22:45:23.546 INFO:tasks.workunit.client.0.smithi063.stdout:7/253: write d0/fa [4267563,117704] 0 2024-04-24T22:45:23.547 INFO:tasks.workunit.client.1.smithi192.stdout:4/241: rename d2/d9/d1e to d2/d9/d25/d2e/d31/d36/d5b 0 2024-04-24T22:45:23.549 INFO:tasks.workunit.client.1.smithi192.stdout:4/242: write d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f34 [4594203,17323] 0 2024-04-24T22:45:23.550 INFO:tasks.workunit.client.1.smithi192.stdout:4/243: write d2/db/f4e [651855,30438] 0 2024-04-24T22:45:23.550 INFO:tasks.workunit.client.1.smithi192.stdout:1/217: creat d9/d15/f49 x:0 0 0 2024-04-24T22:45:23.551 INFO:tasks.workunit.client.0.smithi063.stdout:1/353: rmdir d3/da/dc/d1f 39 2024-04-24T22:45:23.551 INFO:tasks.workunit.client.1.smithi192.stdout:4/244: dread - d2/db/f3d zero size 2024-04-24T22:45:23.554 INFO:tasks.workunit.client.0.smithi063.stdout:7/254: mkdir d0/d26/d5c 0 2024-04-24T22:45:23.558 INFO:tasks.workunit.client.0.smithi063.stdout:7/255: readlink d0/d26/d2e/l59 0 2024-04-24T22:45:23.559 INFO:tasks.workunit.client.0.smithi063.stdout:1/354: mknod d3/da/dc/d11/d5e/d6f/c72 0 2024-04-24T22:45:23.559 INFO:tasks.workunit.client.1.smithi192.stdout:0/345: link d3/l4f d3/d27/d3c/d3b/l7d 0 2024-04-24T22:45:23.560 INFO:tasks.workunit.client.1.smithi192.stdout:0/346: chown d3/d27/d28/f3e 2018606 1 2024-04-24T22:45:23.561 INFO:tasks.workunit.client.1.smithi192.stdout:4/245: mkdir d2/d5c 0 2024-04-24T22:45:23.562 INFO:tasks.workunit.client.0.smithi063.stdout:7/256: getdents d0/d26/d5c 0 2024-04-24T22:45:23.564 INFO:tasks.workunit.client.1.smithi192.stdout:0/347: rename d3/d27/l36 to d3/d27/d3c/d6c/l7e 0 2024-04-24T22:45:23.565 INFO:tasks.workunit.client.0.smithi063.stdout:1/355: creat d3/da/dc/d1f/d21/d42/f73 x:0 0 0 2024-04-24T22:45:23.568 INFO:tasks.workunit.client.0.smithi063.stdout:7/257: dwrite d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:23.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/348: creat d3/d27/d28/d2e/d4a/f7f x:0 0 0 2024-04-24T22:45:23.575 INFO:tasks.workunit.client.1.smithi192.stdout:1/218: dread d9/f3c [0,4194304] 0 2024-04-24T22:45:23.576 INFO:tasks.workunit.client.1.smithi192.stdout:1/219: chown d9/dd/l36 16 1 2024-04-24T22:45:23.579 INFO:tasks.workunit.client.1.smithi192.stdout:1/220: chown d9/d15/c3d 187101705 1 2024-04-24T22:45:23.582 INFO:tasks.workunit.client.1.smithi192.stdout:1/221: getdents d9/dd 0 2024-04-24T22:45:23.584 INFO:tasks.workunit.client.1.smithi192.stdout:1/222: chown d9/d15/d2d/d31/d2f/f3f 3398 1 2024-04-24T22:45:23.584 INFO:tasks.workunit.client.1.smithi192.stdout:0/349: dread d3/d27/d28/f1d [0,4194304] 0 2024-04-24T22:45:23.585 INFO:tasks.workunit.client.1.smithi192.stdout:1/223: mkdir d9/d15/d2d/d4a 0 2024-04-24T22:45:23.586 INFO:tasks.workunit.client.1.smithi192.stdout:0/350: mkdir d3/d27/d28/d2e/d4a/d5c/d80 0 2024-04-24T22:45:23.587 INFO:tasks.workunit.client.1.smithi192.stdout:0/351: fsync d3/d27/d28/d2e/d4a/f7f 0 2024-04-24T22:45:23.587 INFO:tasks.workunit.client.1.smithi192.stdout:1/224: mknod d9/d15/d2d/d31/d2f/c4b 0 2024-04-24T22:45:23.587 INFO:tasks.workunit.client.1.smithi192.stdout:0/352: read - d3/d27/f68 zero size 2024-04-24T22:45:23.588 INFO:tasks.workunit.client.1.smithi192.stdout:1/225: creat d9/d15/d2d/d31/d2f/f4c x:0 0 0 2024-04-24T22:45:23.589 INFO:tasks.workunit.client.1.smithi192.stdout:1/226: readlink d9/dd/le 0 2024-04-24T22:45:23.591 INFO:tasks.workunit.client.1.smithi192.stdout:1/227: write d9/d15/f2e [982502,2678] 0 2024-04-24T22:45:23.592 INFO:tasks.workunit.client.1.smithi192.stdout:0/353: rmdir d3/d27/d28 39 2024-04-24T22:45:23.594 INFO:tasks.workunit.client.1.smithi192.stdout:0/354: mkdir d3/d27/d28/d81 0 2024-04-24T22:45:23.597 INFO:tasks.workunit.client.1.smithi192.stdout:0/355: rename d3/l67 to d3/d27/d28/d2e/l82 0 2024-04-24T22:45:23.598 INFO:tasks.workunit.client.1.smithi192.stdout:1/228: truncate d9/d15/d2d/f37 580963 0 2024-04-24T22:45:23.598 INFO:tasks.workunit.client.1.smithi192.stdout:0/356: write d3/d27/d28/d2e/d4a/d5c/f46 [840492,12374] 0 2024-04-24T22:45:23.598 INFO:tasks.workunit.client.1.smithi192.stdout:1/229: stat d9/dd/c1d 0 2024-04-24T22:45:23.599 INFO:tasks.workunit.client.1.smithi192.stdout:1/230: chown d9/dd/f29 510 1 2024-04-24T22:45:23.602 INFO:tasks.workunit.client.1.smithi192.stdout:1/231: link d9/d15/d2d/l23 d9/dd/l4d 0 2024-04-24T22:45:23.604 INFO:tasks.workunit.client.1.smithi192.stdout:0/357: write d3/f2c [65418,12449] 0 2024-04-24T22:45:23.608 INFO:tasks.workunit.client.1.smithi192.stdout:1/232: link d9/d15/c3d d9/d15/d2d/d31/c4e 0 2024-04-24T22:45:23.611 INFO:tasks.workunit.client.1.smithi192.stdout:1/233: truncate d9/dd/f41 23334 0 2024-04-24T22:45:23.611 INFO:tasks.workunit.client.1.smithi192.stdout:0/358: write d3/d27/f30 [1013369,74122] 0 2024-04-24T22:45:23.612 INFO:tasks.workunit.client.1.smithi192.stdout:0/359: read - d3/d27/d28/d2e/d4a/f7f zero size 2024-04-24T22:45:23.613 INFO:tasks.workunit.client.0.smithi063.stdout:0/288: sync 2024-04-24T22:45:23.613 INFO:tasks.workunit.client.0.smithi063.stdout:5/334: sync 2024-04-24T22:45:23.613 INFO:tasks.workunit.client.1.smithi192.stdout:1/234: creat d9/f4f x:0 0 0 2024-04-24T22:45:23.614 INFO:tasks.workunit.client.0.smithi063.stdout:1/356: dwrite d3/da/dc/d11/f61 [0,4194304] 0 2024-04-24T22:45:23.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/235: write d9/d15/d2d/d31/d2f/f3e [875672,53913] 0 2024-04-24T22:45:23.616 INFO:tasks.workunit.client.0.smithi063.stdout:7/258: dread d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:23.616 INFO:tasks.workunit.client.0.smithi063.stdout:5/335: truncate d5/d11/d33/f52 999539 0 2024-04-24T22:45:23.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/336: stat d5/d24/d36/d65 0 2024-04-24T22:45:23.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/337: stat d5/d11 0 2024-04-24T22:45:23.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/236: readlink d9/d15/d2d/l3b 0 2024-04-24T22:45:23.618 INFO:tasks.workunit.client.0.smithi063.stdout:1/357: readlink d3/d2d/l59 0 2024-04-24T22:45:23.619 INFO:tasks.workunit.client.0.smithi063.stdout:1/358: dread - d3/da/dc/d11/f5f zero size 2024-04-24T22:45:23.620 INFO:tasks.workunit.client.1.smithi192.stdout:0/360: dwrite d3/f2c [0,4194304] 0 2024-04-24T22:45:23.621 INFO:tasks.workunit.client.1.smithi192.stdout:0/361: chown d3/d27/d28/d2e/d3f 25695430 1 2024-04-24T22:45:23.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/259: mkdir d0/d26/d5d 0 2024-04-24T22:45:23.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/260: write d0/d26/f5b [685969,43403] 0 2024-04-24T22:45:23.624 INFO:tasks.workunit.client.0.smithi063.stdout:7/261: fdatasync d0/d4/de/d17/f42 0 2024-04-24T22:45:23.625 INFO:tasks.workunit.client.1.smithi192.stdout:1/237: dread d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:23.627 INFO:tasks.workunit.client.0.smithi063.stdout:7/262: dread d0/d4/f19 [0,4194304] 0 2024-04-24T22:45:23.629 INFO:tasks.workunit.client.0.smithi063.stdout:0/289: rename d6/dd/d20 to d6/d64 0 2024-04-24T22:45:23.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/263: read d0/fa [1958968,62983] 0 2024-04-24T22:45:23.631 INFO:tasks.workunit.client.0.smithi063.stdout:7/264: stat d0/d4/de/d17/d28/l4d 0 2024-04-24T22:45:23.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/359: mknod d3/da/d34/c74 0 2024-04-24T22:45:23.633 INFO:tasks.workunit.client.0.smithi063.stdout:1/360: mknod d3/da/d34/d68/c75 0 2024-04-24T22:45:23.635 INFO:tasks.workunit.client.0.smithi063.stdout:1/361: write d3/da/dc/d11/f46 [3433265,9275] 0 2024-04-24T22:45:23.635 INFO:tasks.workunit.client.0.smithi063.stdout:7/265: mkdir d0/d26/d5c/d5e 0 2024-04-24T22:45:23.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/362: chown d3/da/dc/d39/d5a 2 1 2024-04-24T22:45:23.640 INFO:tasks.workunit.client.0.smithi063.stdout:1/363: link d3/da/l4b d3/da/dc/l76 0 2024-04-24T22:45:23.642 INFO:tasks.workunit.client.0.smithi063.stdout:1/364: unlink d3/da/dc/d11/f6b 0 2024-04-24T22:45:23.642 INFO:tasks.workunit.client.0.smithi063.stdout:1/365: write d3/f63 [67784,103384] 0 2024-04-24T22:45:23.644 INFO:tasks.workunit.client.0.smithi063.stdout:5/338: write d5/d11/d23/d2d/d62/d73/f75 [350011,12415] 0 2024-04-24T22:45:23.644 INFO:tasks.workunit.client.0.smithi063.stdout:1/366: write d3/da/dc/d11/f35 [7551534,19965] 0 2024-04-24T22:45:23.646 INFO:tasks.workunit.client.0.smithi063.stdout:5/339: write d5/d24/d49/d3b/f48 [4615237,93999] 0 2024-04-24T22:45:23.647 INFO:tasks.workunit.client.0.smithi063.stdout:1/367: chown d3/da/dc/c15 251 1 2024-04-24T22:45:23.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/368: write d3/da/dc/d39/f65 [495361,25260] 0 2024-04-24T22:45:23.651 INFO:tasks.workunit.client.0.smithi063.stdout:0/290: truncate d6/dd/f12 5399146 0 2024-04-24T22:45:23.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/369: chown d3/da/d34/c74 1231455973 1 2024-04-24T22:45:23.653 INFO:tasks.workunit.client.0.smithi063.stdout:5/340: dwrite d5/f5b [0,4194304] 0 2024-04-24T22:45:23.656 INFO:tasks.workunit.client.0.smithi063.stdout:0/291: truncate d6/da/dc/d26/d33/f51 4545847 0 2024-04-24T22:45:23.658 INFO:tasks.workunit.client.0.smithi063.stdout:1/370: dwrite d3/f3c [0,4194304] 0 2024-04-24T22:45:23.660 INFO:tasks.workunit.client.0.smithi063.stdout:0/292: rename d6/d64/d21/c27 to d6/d3a/c65 0 2024-04-24T22:45:23.661 INFO:tasks.workunit.client.0.smithi063.stdout:1/371: creat d3/da/dc/d1f/f77 x:0 0 0 2024-04-24T22:45:23.670 INFO:tasks.workunit.client.0.smithi063.stdout:1/372: dwrite d3/da/dc/d11/f61 [4194304,4194304] 0 2024-04-24T22:45:23.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/373: symlink d3/da/dc/d1f/d21/d49/l78 0 2024-04-24T22:45:23.675 INFO:tasks.workunit.client.0.smithi063.stdout:1/374: dread - d3/da/dc/d1f/f77 zero size 2024-04-24T22:45:23.676 INFO:tasks.workunit.client.0.smithi063.stdout:7/266: dread d0/d4/f11 [0,4194304] 0 2024-04-24T22:45:23.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/267: mknod d0/d26/d2e/c5f 0 2024-04-24T22:45:23.680 INFO:tasks.workunit.client.0.smithi063.stdout:7/268: chown d0/d4/de 204178128 1 2024-04-24T22:45:23.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/269: creat d0/d4/de/f60 x:0 0 0 2024-04-24T22:45:23.689 INFO:tasks.workunit.client.0.smithi063.stdout:7/270: rename d0/d4/de/d17/f43 to d0/d4/de/f61 0 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/242: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:2/370: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:5/204: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/237: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:8/243: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:3/275: sync 2024-04-24T22:45:23.690 INFO:tasks.workunit.client.1.smithi192.stdout:7/254: sync 2024-04-24T22:45:23.691 INFO:tasks.workunit.client.0.smithi063.stdout:0/293: dread d6/d64/f3b [0,4194304] 0 2024-04-24T22:45:23.692 INFO:tasks.workunit.client.0.smithi063.stdout:7/271: mknod d0/d4/de/d17/d28/d31/c62 0 2024-04-24T22:45:23.695 INFO:tasks.workunit.client.0.smithi063.stdout:0/294: mknod d6/d64/d21/d25/d31/c66 0 2024-04-24T22:45:23.695 INFO:tasks.workunit.client.1.smithi192.stdout:9/238: write d3/dd/f30 [651904,16757] 0 2024-04-24T22:45:23.696 INFO:tasks.workunit.client.1.smithi192.stdout:8/244: dread f7 [4194304,4194304] 0 2024-04-24T22:45:23.696 INFO:tasks.workunit.client.1.smithi192.stdout:5/205: dread d0/d1b/d1c/f20 [0,4194304] 0 2024-04-24T22:45:23.696 INFO:tasks.workunit.client.1.smithi192.stdout:7/255: dread d2/f1a [4194304,4194304] 0 2024-04-24T22:45:23.697 INFO:tasks.workunit.client.0.smithi063.stdout:7/272: rmdir d0/d2f 39 2024-04-24T22:45:23.698 INFO:tasks.workunit.client.1.smithi192.stdout:8/245: write db/f1d [3432152,31102] 0 2024-04-24T22:45:23.698 INFO:tasks.workunit.client.0.smithi063.stdout:1/375: dwrite d3/f7 [0,4194304] 0 2024-04-24T22:45:23.699 INFO:tasks.workunit.client.0.smithi063.stdout:7/273: chown d0/d4/de/d17/d28 0 1 2024-04-24T22:45:23.700 INFO:tasks.workunit.client.0.smithi063.stdout:0/295: read d6/d64/d21/d25/f55 [507979,117820] 0 2024-04-24T22:45:23.702 INFO:tasks.workunit.client.1.smithi192.stdout:3/276: creat d2/d4/d7/d4f/f5a x:0 0 0 2024-04-24T22:45:23.703 INFO:tasks.workunit.client.0.smithi063.stdout:2/331: sync 2024-04-24T22:45:23.703 INFO:tasks.workunit.client.0.smithi063.stdout:8/352: sync 2024-04-24T22:45:23.703 INFO:tasks.workunit.client.0.smithi063.stdout:8/353: chown d1 13 1 2024-04-24T22:45:23.703 INFO:tasks.workunit.client.0.smithi063.stdout:2/332: stat d0/dd 0 2024-04-24T22:45:23.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/239: creat d3/dd/de/f51 x:0 0 0 2024-04-24T22:45:23.709 INFO:tasks.workunit.client.1.smithi192.stdout:3/277: dread d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:23.709 INFO:tasks.workunit.client.0.smithi063.stdout:7/274: dread d0/f7 [0,4194304] 0 2024-04-24T22:45:23.711 INFO:tasks.workunit.client.0.smithi063.stdout:8/354: creat d1/d11/d49/f72 x:0 0 0 2024-04-24T22:45:23.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/206: unlink d0/f10 0 2024-04-24T22:45:23.714 INFO:tasks.workunit.client.1.smithi192.stdout:5/207: truncate d0/d13/d3b/f3a 637683 0 2024-04-24T22:45:23.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/208: chown d0/d13/d3b/f32 1439 1 2024-04-24T22:45:23.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/243: mknod d0/d8/c59 0 2024-04-24T22:45:23.718 INFO:tasks.workunit.client.0.smithi063.stdout:0/296: symlink d6/da/d2b/d60/l67 0 2024-04-24T22:45:23.719 INFO:tasks.workunit.client.1.smithi192.stdout:8/246: creat db/dc/d29/f48 x:0 0 0 2024-04-24T22:45:23.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/376: truncate d3/da/dc/d39/f65 480296 0 2024-04-24T22:45:23.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/355: symlink d1/d11/d3c/l73 0 2024-04-24T22:45:23.723 INFO:tasks.workunit.client.0.smithi063.stdout:0/297: mknod d6/d64/d21/d25/d62/c68 0 2024-04-24T22:45:23.724 INFO:tasks.workunit.client.0.smithi063.stdout:7/275: dread d0/d4/de/f61 [0,4194304] 0 2024-04-24T22:45:23.726 INFO:tasks.workunit.client.0.smithi063.stdout:1/377: rename d3/d2d to d3/da/dc/d1f/d2b/d79 0 2024-04-24T22:45:23.727 INFO:tasks.workunit.client.1.smithi192.stdout:3/278: dread d2/d4/d7/f2c [0,4194304] 0 2024-04-24T22:45:23.728 INFO:tasks.workunit.client.1.smithi192.stdout:9/240: unlink d3/f3f 0 2024-04-24T22:45:23.728 INFO:tasks.workunit.client.1.smithi192.stdout:3/279: dread - d2/d27/f4a zero size 2024-04-24T22:45:23.729 INFO:tasks.workunit.client.1.smithi192.stdout:3/280: readlink d2/d4/d12/d1a/d1d/l4d 0 2024-04-24T22:45:23.732 INFO:tasks.workunit.client.0.smithi063.stdout:0/298: stat d6/d64/d24/d42 0 2024-04-24T22:45:23.732 INFO:tasks.workunit.client.1.smithi192.stdout:6/244: dread - d0/d8/d3d/d3e/d40/f47 zero size 2024-04-24T22:45:23.733 INFO:tasks.workunit.client.0.smithi063.stdout:0/299: write d6/da/dc/d26/d33/f51 [590560,89815] 0 2024-04-24T22:45:23.734 INFO:tasks.workunit.client.1.smithi192.stdout:5/209: unlink d0/d13/d3b/f32 0 2024-04-24T22:45:23.736 INFO:tasks.workunit.client.1.smithi192.stdout:2/371: truncate d4/f41 3715455 0 2024-04-24T22:45:23.736 INFO:tasks.workunit.client.1.smithi192.stdout:3/281: dwrite d2/d4/d12/f1c [0,4194304] 0 2024-04-24T22:45:23.736 INFO:tasks.workunit.client.0.smithi063.stdout:7/276: mkdir d0/d26/d5c/d63 0 2024-04-24T22:45:23.738 INFO:tasks.workunit.client.1.smithi192.stdout:7/256: symlink d2/db/d21/d42/l4a 0 2024-04-24T22:45:23.739 INFO:tasks.workunit.client.0.smithi063.stdout:0/300: dwrite d6/da/f10 [0,4194304] 0 2024-04-24T22:45:23.739 INFO:tasks.workunit.client.1.smithi192.stdout:7/257: write d2/db/d36/f38 [132363,120428] 0 2024-04-24T22:45:23.740 INFO:tasks.workunit.client.1.smithi192.stdout:7/258: stat d2/fa 0 2024-04-24T22:45:23.740 INFO:tasks.workunit.client.0.smithi063.stdout:0/301: write d6/d64/d24/f59 [769202,50254] 0 2024-04-24T22:45:23.740 INFO:tasks.workunit.client.0.smithi063.stdout:0/302: chown d6/d5b 1979216645 1 2024-04-24T22:45:23.741 INFO:tasks.workunit.client.0.smithi063.stdout:1/378: mknod d3/da/dc/d39/d5a/c7a 0 2024-04-24T22:45:23.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/356: mknod d1/d2/c74 0 2024-04-24T22:45:23.744 INFO:tasks.workunit.client.0.smithi063.stdout:8/357: chown d1/d21 14194691 1 2024-04-24T22:45:23.746 INFO:tasks.workunit.client.1.smithi192.stdout:9/241: rmdir d3/dd 39 2024-04-24T22:45:23.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/379: rename d3/l8 to d3/da/dc/d1f/d21/d42/l7b 0 2024-04-24T22:45:23.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/380: dread - d3/da/dc/d1f/d21/d42/f73 zero size 2024-04-24T22:45:23.752 INFO:tasks.workunit.client.0.smithi063.stdout:7/277: dread d0/d4/de/f25 [0,4194304] 0 2024-04-24T22:45:23.752 INFO:tasks.workunit.client.0.smithi063.stdout:0/303: symlink d6/d64/l69 0 2024-04-24T22:45:23.753 INFO:tasks.workunit.client.1.smithi192.stdout:5/210: creat d0/d1b/d1c/d2c/f3d x:0 0 0 2024-04-24T22:45:23.754 INFO:tasks.workunit.client.0.smithi063.stdout:8/358: rename d1/d11/d3c/l73 to d1/d11/d49/l75 0 2024-04-24T22:45:23.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/381: mknod d3/da/dc/d11/d5e/c7c 0 2024-04-24T22:45:23.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/304: rmdir d6/d64/d21/d25/d62 39 2024-04-24T22:45:23.756 INFO:tasks.workunit.client.0.smithi063.stdout:1/382: write d3/da/dc/d1f/d21/f69 [1923390,41452] 0 2024-04-24T22:45:23.757 INFO:tasks.workunit.client.0.smithi063.stdout:0/305: chown d6/da/d2b/l37 3 1 2024-04-24T22:45:23.757 INFO:tasks.workunit.client.0.smithi063.stdout:7/278: truncate d0/fa 1714249 0 2024-04-24T22:45:23.758 INFO:tasks.workunit.client.1.smithi192.stdout:8/247: link f9 db/dc/d22/d31/d33/f49 0 2024-04-24T22:45:23.758 INFO:tasks.workunit.client.0.smithi063.stdout:8/359: rmdir d1/d22/d33 39 2024-04-24T22:45:23.760 INFO:tasks.workunit.client.0.smithi063.stdout:0/306: mknod d6/d5b/c6a 0 2024-04-24T22:45:23.761 INFO:tasks.workunit.client.1.smithi192.stdout:6/245: mkdir d0/d8/d29/d5a 0 2024-04-24T22:45:23.761 INFO:tasks.workunit.client.0.smithi063.stdout:7/279: dread d0/d4/de/d17/d35/f54 [0,4194304] 0 2024-04-24T22:45:23.762 INFO:tasks.workunit.client.0.smithi063.stdout:7/280: write d0/d4/de/d17/f42 [704349,44875] 0 2024-04-24T22:45:23.762 INFO:tasks.workunit.client.0.smithi063.stdout:8/360: mknod d1/d11/c76 0 2024-04-24T22:45:23.764 INFO:tasks.workunit.client.1.smithi192.stdout:5/211: creat d0/d1b/d1c/f3e x:0 0 0 2024-04-24T22:45:23.766 INFO:tasks.workunit.client.1.smithi192.stdout:2/372: mknod d4/d9/d10/d6c/c6d 0 2024-04-24T22:45:23.766 INFO:tasks.workunit.client.1.smithi192.stdout:2/373: readlink d4/l39 0 2024-04-24T22:45:23.766 INFO:tasks.workunit.client.0.smithi063.stdout:3/334: sync 2024-04-24T22:45:23.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/281: creat d0/d4/de/d17/d28/f64 x:0 0 0 2024-04-24T22:45:23.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/307: truncate d6/da/dc/f13 3741256 0 2024-04-24T22:45:23.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/248: rmdir db/dc/d29 39 2024-04-24T22:45:23.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/374: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:23.773 INFO:tasks.workunit.client.1.smithi192.stdout:2/375: chown d4/d9/d10/l4f 15921 1 2024-04-24T22:45:23.778 INFO:tasks.workunit.client.0.smithi063.stdout:7/282: unlink d0/d26/d2e/l34 0 2024-04-24T22:45:23.780 INFO:tasks.workunit.client.1.smithi192.stdout:9/242: creat d3/d24/f52 x:0 0 0 2024-04-24T22:45:23.780 INFO:tasks.workunit.client.1.smithi192.stdout:2/376: dwrite d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:23.781 INFO:tasks.workunit.client.1.smithi192.stdout:9/243: fsync d3/d2c/f2e 0 2024-04-24T22:45:23.781 INFO:tasks.workunit.client.1.smithi192.stdout:2/377: chown d4/d9/d10/d6c/c6d 11685 1 2024-04-24T22:45:23.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/308: creat d6/d64/d24/f6b x:0 0 0 2024-04-24T22:45:23.784 INFO:tasks.workunit.client.0.smithi063.stdout:8/361: creat d1/d22/f77 x:0 0 0 2024-04-24T22:45:23.786 INFO:tasks.workunit.client.1.smithi192.stdout:5/212: creat d0/d13/f3f x:0 0 0 2024-04-24T22:45:23.787 INFO:tasks.workunit.client.0.smithi063.stdout:3/335: symlink da/d18/l73 0 2024-04-24T22:45:23.788 INFO:tasks.workunit.client.0.smithi063.stdout:0/309: dwrite d6/dd/f54 [0,4194304] 0 2024-04-24T22:45:23.793 INFO:tasks.workunit.client.0.smithi063.stdout:8/362: mknod d1/d2/d17/c78 0 2024-04-24T22:45:23.797 INFO:tasks.workunit.client.0.smithi063.stdout:3/336: mknod da/d48/c74 0 2024-04-24T22:45:23.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/363: mkdir d1/d11/d49/d54/d79 0 2024-04-24T22:45:23.802 INFO:tasks.workunit.client.1.smithi192.stdout:3/282: truncate d2/d4/f16 3693706 0 2024-04-24T22:45:23.803 INFO:tasks.workunit.client.1.smithi192.stdout:6/246: creat d0/d8/d29/d58/f5b x:0 0 0 2024-04-24T22:45:23.804 INFO:tasks.workunit.client.1.smithi192.stdout:6/247: dread - d0/d8/d29/f53 zero size 2024-04-24T22:45:23.806 INFO:tasks.workunit.client.1.smithi192.stdout:3/283: dread d2/d4/d12/f49 [0,4194304] 0 2024-04-24T22:45:23.807 INFO:tasks.workunit.client.0.smithi063.stdout:3/337: unlink da/d39/f3c 0 2024-04-24T22:45:23.807 INFO:tasks.workunit.client.0.smithi063.stdout:3/338: chown da/c2b 9 1 2024-04-24T22:45:23.807 INFO:tasks.workunit.client.1.smithi192.stdout:5/213: creat d0/d13/f40 x:0 0 0 2024-04-24T22:45:23.808 INFO:tasks.workunit.client.0.smithi063.stdout:3/339: write da/f15 [1011,125493] 0 2024-04-24T22:45:23.809 INFO:tasks.workunit.client.1.smithi192.stdout:7/259: truncate d2/db/d36/f3d 1291600 0 2024-04-24T22:45:23.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/310: mknod d6/d64/d21/d25/d62/d63/c6c 0 2024-04-24T22:45:23.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/311: write f5 [657906,32060] 0 2024-04-24T22:45:23.817 INFO:tasks.workunit.client.0.smithi063.stdout:3/340: symlink da/d18/d47/l75 0 2024-04-24T22:45:23.820 INFO:tasks.workunit.client.0.smithi063.stdout:0/312: mknod d6/d64/d24/d42/c6d 0 2024-04-24T22:45:23.820 INFO:tasks.workunit.client.1.smithi192.stdout:8/249: dread db/dc/d29/f2c [0,4194304] 0 2024-04-24T22:45:23.821 INFO:tasks.workunit.client.0.smithi063.stdout:8/364: link d1/d11/l13 d1/d11/d32/l7a 0 2024-04-24T22:45:23.825 INFO:tasks.workunit.client.1.smithi192.stdout:2/378: mknod d4/d9/d10/c6e 0 2024-04-24T22:45:23.826 INFO:tasks.workunit.client.0.smithi063.stdout:3/341: symlink da/d18/d47/d4d/d62/l76 0 2024-04-24T22:45:23.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/342: chown da/f36 2141047 1 2024-04-24T22:45:23.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/365: mkdir d1/d11/d49/d54/d7b 0 2024-04-24T22:45:23.833 INFO:tasks.workunit.client.1.smithi192.stdout:8/250: dread f5 [0,4194304] 0 2024-04-24T22:45:23.833 INFO:tasks.workunit.client.1.smithi192.stdout:8/251: chown ca 707132 1 2024-04-24T22:45:23.835 INFO:tasks.workunit.client.1.smithi192.stdout:5/214: symlink d0/d1b/d1c/l41 0 2024-04-24T22:45:23.837 INFO:tasks.workunit.client.1.smithi192.stdout:7/260: truncate d2/db/d21/f22 426481 0 2024-04-24T22:45:23.839 INFO:tasks.workunit.client.0.smithi063.stdout:0/313: link d6/da/f14 d6/d64/d21/d25/d62/d63/f6e 0 2024-04-24T22:45:23.840 INFO:tasks.workunit.client.1.smithi192.stdout:9/244: mkdir d3/dd/d53 0 2024-04-24T22:45:23.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/366: creat d1/d11/d49/f7c x:0 0 0 2024-04-24T22:45:23.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/379: mkdir d4/d9/d10/d6c/d6f 0 2024-04-24T22:45:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:4/317: sync 2024-04-24T22:45:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/333: sync 2024-04-24T22:45:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:6/343: sync 2024-04-24T22:45:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:5/341: sync 2024-04-24T22:45:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:9/340: sync 2024-04-24T22:45:23.844 INFO:tasks.workunit.client.0.smithi063.stdout:3/343: mknod da/c77 0 2024-04-24T22:45:23.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/380: chown d4/d9/d37/c6a 18 1 2024-04-24T22:45:23.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/381: chown d4/c2f 36889641 1 2024-04-24T22:45:23.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/382: readlink d4/l30 0 2024-04-24T22:45:23.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/383: truncate d4/d9/d37/f62 1029143 0 2024-04-24T22:45:23.845 INFO:tasks.workunit.client.1.smithi192.stdout:2/384: stat d4/d29/c54 0 2024-04-24T22:45:23.845 INFO:tasks.workunit.client.1.smithi192.stdout:6/248: mknod d0/d8/d29/d4c/c5c 0 2024-04-24T22:45:23.846 INFO:tasks.workunit.client.0.smithi063.stdout:5/342: dread d5/f6 [0,4194304] 0 2024-04-24T22:45:23.847 INFO:tasks.workunit.client.1.smithi192.stdout:3/284: getdents d2/d4/d12/d39/d3a 0 2024-04-24T22:45:23.847 INFO:tasks.workunit.client.0.smithi063.stdout:5/343: chown d5/d11/d23/d2d 159603706 1 2024-04-24T22:45:23.847 INFO:tasks.workunit.client.1.smithi192.stdout:3/285: dread - d2/d4/d7/d4f/f52 zero size 2024-04-24T22:45:23.851 INFO:tasks.workunit.client.0.smithi063.stdout:2/334: creat d0/d2/d39/f65 x:0 0 0 2024-04-24T22:45:23.853 INFO:tasks.workunit.client.1.smithi192.stdout:9/245: creat d3/d2c/f54 x:0 0 0 2024-04-24T22:45:23.854 INFO:tasks.workunit.client.0.smithi063.stdout:3/344: creat da/d40/f78 x:0 0 0 2024-04-24T22:45:23.856 INFO:tasks.workunit.client.0.smithi063.stdout:3/345: chown da/d18/f19 0 1 2024-04-24T22:45:23.857 INFO:tasks.workunit.client.1.smithi192.stdout:6/249: dread d0/d8/fe [0,4194304] 0 2024-04-24T22:45:23.858 INFO:tasks.workunit.client.0.smithi063.stdout:6/344: dwrite db/d1d/d2d/d48/f63 [0,4194304] 0 2024-04-24T22:45:23.858 INFO:tasks.workunit.client.0.smithi063.stdout:0/314: dread d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:23.858 INFO:tasks.workunit.client.1.smithi192.stdout:6/250: read - d0/d8/d29/f53 zero size 2024-04-24T22:45:23.859 INFO:tasks.workunit.client.0.smithi063.stdout:4/318: rename d2/d8/d40/f55 to d2/d8/d36/d3a/d50/f5d 0 2024-04-24T22:45:23.861 INFO:tasks.workunit.client.0.smithi063.stdout:4/319: dread d2/d8/f31 [0,4194304] 0 2024-04-24T22:45:23.862 INFO:tasks.workunit.client.1.smithi192.stdout:8/252: mknod db/dc/d22/d31/d33/c4a 0 2024-04-24T22:45:23.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/215: write d0/d1b/f37 [3059786,24159] 0 2024-04-24T22:45:23.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/216: read - d0/d13/f3f zero size 2024-04-24T22:45:23.870 INFO:tasks.workunit.client.0.smithi063.stdout:6/345: dread db/d1d/d20/d3e/f4f [0,4194304] 0 2024-04-24T22:45:23.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/217: stat d0/d1b/d1c/d2c/f35 0 2024-04-24T22:45:23.872 INFO:tasks.workunit.client.0.smithi063.stdout:4/320: dread d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:23.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/246: creat d3/dd/de/f55 x:0 0 0 2024-04-24T22:45:23.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/367: truncate d1/d2/d3/f14 1797695 0 2024-04-24T22:45:23.875 INFO:tasks.workunit.client.1.smithi192.stdout:6/251: write d0/d8/d29/f53 [728593,45172] 0 2024-04-24T22:45:23.876 INFO:tasks.workunit.client.1.smithi192.stdout:8/253: write db/dc/f13 [4250398,5327] 0 2024-04-24T22:45:23.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/254: fdatasync db/dc/d29/f48 0 2024-04-24T22:45:23.879 INFO:tasks.workunit.client.0.smithi063.stdout:0/315: rename d6/d3a/f41 to d6/dd/f6f 0 2024-04-24T22:45:23.880 INFO:tasks.workunit.client.1.smithi192.stdout:7/261: link d2/db/lc d2/db/d21/l4b 0 2024-04-24T22:45:23.881 INFO:tasks.workunit.client.1.smithi192.stdout:5/218: rmdir d0/d13/d3b 39 2024-04-24T22:45:23.882 INFO:tasks.workunit.client.1.smithi192.stdout:8/255: dread db/dc/d29/f2b [0,4194304] 0 2024-04-24T22:45:23.883 INFO:tasks.workunit.client.1.smithi192.stdout:8/256: chown db/dc/f17 0 1 2024-04-24T22:45:23.883 INFO:tasks.workunit.client.1.smithi192.stdout:9/247: unlink d3/c4c 0 2024-04-24T22:45:23.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/248: write d3/dd/de/f55 [335010,10386] 0 2024-04-24T22:45:23.885 INFO:tasks.workunit.client.0.smithi063.stdout:5/344: creat d5/d11/d76/f7b x:0 0 0 2024-04-24T22:45:23.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/252: rename d0/l15 to d0/d8/d29/d4c/l5d 0 2024-04-24T22:45:23.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/341: creat d6/d8/f71 x:0 0 0 2024-04-24T22:45:23.890 INFO:tasks.workunit.client.1.smithi192.stdout:3/286: link d2/d4/d12/d1a/d1d/l4d d2/d4/d7/d4b/d54/l5b 0 2024-04-24T22:45:23.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/287: fdatasync d2/d4/d12/d1a/d1d/f25 0 2024-04-24T22:45:23.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/346: unlink c4 0 2024-04-24T22:45:23.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/347: write db/d1d/d2d/d48/f51 [1687903,97657] 0 2024-04-24T22:45:23.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/348: chown db/d1d/f4e 6170858 1 2024-04-24T22:45:23.900 INFO:tasks.workunit.client.1.smithi192.stdout:8/257: symlink db/dc/d29/l4b 0 2024-04-24T22:45:23.902 INFO:tasks.workunit.client.1.smithi192.stdout:7/262: dread d2/db/f15 [0,4194304] 0 2024-04-24T22:45:23.902 INFO:tasks.workunit.client.1.smithi192.stdout:9/249: rmdir d3/dd/de/d17/d33 39 2024-04-24T22:45:23.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/258: dread db/dc/f18 [0,4194304] 0 2024-04-24T22:45:23.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/253: write d0/d8/d22/f27 [3813274,97656] 0 2024-04-24T22:45:23.907 INFO:tasks.workunit.client.0.smithi063.stdout:2/335: truncate d0/d2/d54/d7/f13 5456666 0 2024-04-24T22:45:23.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/288: mknod d2/d4/d12/d39/c5c 0 2024-04-24T22:45:23.910 INFO:tasks.workunit.client.1.smithi192.stdout:9/250: dread d3/fa [4194304,4194304] 0 2024-04-24T22:45:23.911 INFO:tasks.workunit.client.0.smithi063.stdout:3/346: truncate da/d18/f19 257262 0 2024-04-24T22:45:23.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/251: read - d3/d24/f52 zero size 2024-04-24T22:45:23.911 INFO:tasks.workunit.client.1.smithi192.stdout:3/289: fdatasync d2/d4/d12/d1a/d1d/f3f 0 2024-04-24T22:45:23.914 INFO:tasks.workunit.client.0.smithi063.stdout:5/345: mknod d5/d11/d18/c7c 0 2024-04-24T22:45:23.916 INFO:tasks.workunit.client.1.smithi192.stdout:7/263: mknod d2/db/d25/c4c 0 2024-04-24T22:45:23.917 INFO:tasks.workunit.client.0.smithi063.stdout:8/368: dwrite d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:23.918 INFO:tasks.workunit.client.0.smithi063.stdout:2/336: symlink d0/dd/l66 0 2024-04-24T22:45:23.918 INFO:tasks.workunit.client.0.smithi063.stdout:2/337: stat d0/dd/f11 0 2024-04-24T22:45:23.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/254: symlink d0/d8/d29/d4c/l5e 0 2024-04-24T22:45:23.923 INFO:tasks.workunit.client.1.smithi192.stdout:6/255: dread - d0/d8/d3d/f49 zero size 2024-04-24T22:45:23.928 INFO:tasks.workunit.client.1.smithi192.stdout:9/252: unlink d3/dd/f3b 0 2024-04-24T22:45:23.930 INFO:tasks.workunit.client.1.smithi192.stdout:3/290: rmdir d2/d4/d12/d1a/d1d/d55 39 2024-04-24T22:45:23.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/346: rename d5/d11/d33 to d5/d1d/d7d 0 2024-04-24T22:45:23.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/347: readlink d5/d11/d23/d2d/l38 0 2024-04-24T22:45:23.933 INFO:tasks.workunit.client.0.smithi063.stdout:1/383: sync 2024-04-24T22:45:23.933 INFO:tasks.workunit.client.0.smithi063.stdout:7/283: sync 2024-04-24T22:45:23.935 INFO:tasks.workunit.client.1.smithi192.stdout:9/253: dwrite d3/fa [8388608,4194304] 0 2024-04-24T22:45:23.935 INFO:tasks.workunit.client.0.smithi063.stdout:8/369: unlink d1/d2/d17/l18 0 2024-04-24T22:45:23.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/264: creat d2/db/d21/d42/f4d x:0 0 0 2024-04-24T22:45:23.940 INFO:tasks.workunit.client.0.smithi063.stdout:1/384: dwrite d3/da/dc/d1f/d21/d42/f73 [0,4194304] 0 2024-04-24T22:45:23.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/338: dread d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:23.941 INFO:tasks.workunit.client.0.smithi063.stdout:2/339: chown d0/d2/d54/d7/c2e 20 1 2024-04-24T22:45:23.942 INFO:tasks.workunit.client.0.smithi063.stdout:3/347: symlink da/l79 0 2024-04-24T22:45:23.945 INFO:tasks.workunit.client.0.smithi063.stdout:2/340: write d0/d2/d54/d7/f5c [205516,92016] 0 2024-04-24T22:45:23.947 INFO:tasks.workunit.client.1.smithi192.stdout:6/256: readlink d0/d8/d18/d2e/l37 0 2024-04-24T22:45:23.950 INFO:tasks.workunit.client.0.smithi063.stdout:6/349: rmdir db/d1d/d20/d29/d5d/d6f 0 2024-04-24T22:45:23.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/348: mknod d5/d24/d2c/c7e 0 2024-04-24T22:45:23.953 INFO:tasks.workunit.client.1.smithi192.stdout:3/291: rename d2/d4/d12/d39/f44 to d2/d4/d12/d1a/f5d 0 2024-04-24T22:45:23.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/284: symlink d0/d26/d2e/l65 0 2024-04-24T22:45:23.955 INFO:tasks.workunit.client.1.smithi192.stdout:5/219: getdents d0/d13/d19 0 2024-04-24T22:45:23.957 INFO:tasks.workunit.client.0.smithi063.stdout:8/370: rmdir d1/d11/d3c 39 2024-04-24T22:45:23.957 INFO:tasks.workunit.client.0.smithi063.stdout:8/371: dread - d1/d21/f61 zero size 2024-04-24T22:45:23.958 INFO:tasks.workunit.client.1.smithi192.stdout:9/254: unlink d3/fa 0 2024-04-24T22:45:23.961 INFO:tasks.workunit.client.1.smithi192.stdout:3/292: dwrite d2/d4/d12/f53 [0,4194304] 0 2024-04-24T22:45:23.963 INFO:tasks.workunit.client.1.smithi192.stdout:6/257: creat d0/d8/d3d/f5f x:0 0 0 2024-04-24T22:45:23.963 INFO:tasks.workunit.client.1.smithi192.stdout:7/265: dread d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:23.964 INFO:tasks.workunit.client.1.smithi192.stdout:5/220: dread d0/d1b/d1c/f20 [0,4194304] 0 2024-04-24T22:45:23.965 INFO:tasks.workunit.client.1.smithi192.stdout:5/221: truncate d0/d13/d19/f38 266383 0 2024-04-24T22:45:23.966 INFO:tasks.workunit.client.0.smithi063.stdout:3/348: rename da/d48/f56 to da/d39/f7a 0 2024-04-24T22:45:23.966 INFO:tasks.workunit.client.0.smithi063.stdout:6/350: rename db/d1d to db/d1d/d20/d3e/d47/d75 22 2024-04-24T22:45:23.968 INFO:tasks.workunit.client.0.smithi063.stdout:0/316: link d6/d64/d21/d25/c32 d6/d64/c70 0 2024-04-24T22:45:23.970 INFO:tasks.workunit.client.0.smithi063.stdout:2/341: creat d0/d2/d54/d7/f67 x:0 0 0 2024-04-24T22:45:23.972 INFO:tasks.workunit.client.1.smithi192.stdout:8/259: getdents db/dc 0 2024-04-24T22:45:23.972 INFO:tasks.workunit.client.0.smithi063.stdout:2/342: chown d0/dd/l1a 253682 1 2024-04-24T22:45:23.974 INFO:tasks.workunit.client.0.smithi063.stdout:5/349: dread d5/d24/d2c/f50 [0,4194304] 0 2024-04-24T22:45:23.974 INFO:tasks.workunit.client.0.smithi063.stdout:6/351: dwrite db/d1d/f49 [0,4194304] 0 2024-04-24T22:45:23.975 INFO:tasks.workunit.client.1.smithi192.stdout:3/293: chown d2/d4/d7/d4b/c59 523454 1 2024-04-24T22:45:23.982 INFO:tasks.workunit.client.1.smithi192.stdout:7/266: mknod d2/db/c4e 0 2024-04-24T22:45:23.988 INFO:tasks.workunit.client.0.smithi063.stdout:8/372: mknod d1/d22/c7d 0 2024-04-24T22:45:23.990 INFO:tasks.workunit.client.1.smithi192.stdout:9/255: creat d3/dd/d53/f56 x:0 0 0 2024-04-24T22:45:23.990 INFO:tasks.workunit.client.1.smithi192.stdout:9/256: read - d3/dd/f3a zero size 2024-04-24T22:45:23.991 INFO:tasks.workunit.client.0.smithi063.stdout:2/343: mkdir d0/d2/d2d/d31/d68 0 2024-04-24T22:45:23.991 INFO:tasks.workunit.client.1.smithi192.stdout:9/257: truncate d3/d2c/f42 222977 0 2024-04-24T22:45:23.991 INFO:tasks.workunit.client.0.smithi063.stdout:2/344: readlink d0/d2/d2d/l5a 0 2024-04-24T22:45:23.992 INFO:tasks.workunit.client.1.smithi192.stdout:9/258: truncate d3/d2c/f54 267578 0 2024-04-24T22:45:23.994 INFO:tasks.workunit.client.1.smithi192.stdout:7/267: symlink d2/db/d36/l4f 0 2024-04-24T22:45:23.996 INFO:tasks.workunit.client.0.smithi063.stdout:6/352: dread db/d1d/d2d/f36 [0,4194304] 0 2024-04-24T22:45:23.996 INFO:tasks.workunit.client.0.smithi063.stdout:1/385: creat d3/da/dc/d1f/d21/f7d x:0 0 0 2024-04-24T22:45:23.997 INFO:tasks.workunit.client.1.smithi192.stdout:7/268: dread f1 [0,4194304] 0 2024-04-24T22:45:23.998 INFO:tasks.workunit.client.0.smithi063.stdout:8/373: fsync d1/d11/d49/d54/f71 0 2024-04-24T22:45:23.999 INFO:tasks.workunit.client.1.smithi192.stdout:7/269: truncate d2/db/d21/d42/f4d 571423 0 2024-04-24T22:45:24.000 INFO:tasks.workunit.client.0.smithi063.stdout:2/345: unlink d0/dd/f11 0 2024-04-24T22:45:24.002 INFO:tasks.workunit.client.0.smithi063.stdout:1/386: dwrite d3/da/dc/d1f/d21/f7d [0,4194304] 0 2024-04-24T22:45:24.004 INFO:tasks.workunit.client.1.smithi192.stdout:6/258: getdents d0/d8/d29/d58 0 2024-04-24T22:45:24.004 INFO:tasks.workunit.client.1.smithi192.stdout:8/260: dwrite db/dc/d29/f2b [0,4194304] 0 2024-04-24T22:45:24.005 INFO:tasks.workunit.client.1.smithi192.stdout:1/238: sync 2024-04-24T22:45:24.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/362: sync 2024-04-24T22:45:24.005 INFO:tasks.workunit.client.1.smithi192.stdout:4/246: sync 2024-04-24T22:45:24.006 INFO:tasks.workunit.client.1.smithi192.stdout:8/261: chown f7 1 1 2024-04-24T22:45:24.007 INFO:tasks.workunit.client.0.smithi063.stdout:5/350: link d5/f5b d5/d11/d18/f7f 0 2024-04-24T22:45:24.010 INFO:tasks.workunit.client.1.smithi192.stdout:5/222: truncate d0/d1b/d1c/f20 2276222 0 2024-04-24T22:45:24.011 INFO:tasks.workunit.client.0.smithi063.stdout:3/349: getdents da/d18/d47/d4d 0 2024-04-24T22:45:24.011 INFO:tasks.workunit.client.1.smithi192.stdout:8/262: read f5 [4095099,1750] 0 2024-04-24T22:45:24.012 INFO:tasks.workunit.client.0.smithi063.stdout:3/350: fsync da/f29 0 2024-04-24T22:45:24.013 INFO:tasks.workunit.client.0.smithi063.stdout:3/351: fdatasync da/d18/d47/d4d/d62/f65 0 2024-04-24T22:45:24.013 INFO:tasks.workunit.client.0.smithi063.stdout:6/353: creat db/d1d/d2d/f76 x:0 0 0 2024-04-24T22:45:24.014 INFO:tasks.workunit.client.0.smithi063.stdout:5/351: dwrite d5/d11/d23/d2d/f70 [0,4194304] 0 2024-04-24T22:45:24.015 INFO:tasks.workunit.client.1.smithi192.stdout:7/270: dwrite d2/db/f20 [8388608,4194304] 0 2024-04-24T22:45:24.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/354: fdatasync db/d1c/f58 0 2024-04-24T22:45:24.016 INFO:tasks.workunit.client.0.smithi063.stdout:3/352: write da/f28 [434146,121357] 0 2024-04-24T22:45:24.016 INFO:tasks.workunit.client.1.smithi192.stdout:4/247: chown d2/d9/d25/d2e/d31/f4a 13692335 1 2024-04-24T22:45:24.016 INFO:tasks.workunit.client.1.smithi192.stdout:3/294: dwrite d2/f13 [0,4194304] 0 2024-04-24T22:45:24.016 INFO:tasks.workunit.client.1.smithi192.stdout:1/239: readlink d9/dd/le 0 2024-04-24T22:45:24.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/374: truncate d1/d11/d3c/f69 438401 0 2024-04-24T22:45:24.017 INFO:tasks.workunit.client.0.smithi063.stdout:6/355: chown db/d1d/d20/d3e/d47/f54 27459074 1 2024-04-24T22:45:24.019 INFO:tasks.workunit.client.0.smithi063.stdout:6/356: write db/d1d/d20/f2b [642093,90172] 0 2024-04-24T22:45:24.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/263: write db/dc/d29/f48 [880628,109787] 0 2024-04-24T22:45:24.020 INFO:tasks.workunit.client.1.smithi192.stdout:6/259: rmdir d0/d8/d3d/d3e/d40 39 2024-04-24T22:45:24.020 INFO:tasks.workunit.client.0.smithi063.stdout:6/357: chown db/f2a 51 1 2024-04-24T22:45:24.020 INFO:tasks.workunit.client.1.smithi192.stdout:1/240: read - d9/d15/d2d/d31/d2f/f4c zero size 2024-04-24T22:45:24.021 INFO:tasks.workunit.client.0.smithi063.stdout:7/285: link d0/d4/de/l1b d0/d26/d2e/l66 0 2024-04-24T22:45:24.022 INFO:tasks.workunit.client.1.smithi192.stdout:4/248: truncate d2/db/f3d 974690 0 2024-04-24T22:45:24.023 INFO:tasks.workunit.client.0.smithi063.stdout:1/387: mknod d3/da/dc/d1f/d21/d42/c7e 0 2024-04-24T22:45:24.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/260: write d0/f19 [1767162,103316] 0 2024-04-24T22:45:24.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/259: creat d3/dd/f57 x:0 0 0 2024-04-24T22:45:24.025 INFO:tasks.workunit.client.1.smithi192.stdout:8/264: fdatasync db/dc/f3c 0 2024-04-24T22:45:24.026 INFO:tasks.workunit.client.1.smithi192.stdout:0/363: dread d3/f2c [0,4194304] 0 2024-04-24T22:45:24.028 INFO:tasks.workunit.client.1.smithi192.stdout:5/223: rename d0 to d0/d13/d3b/d2f/d42 22 2024-04-24T22:45:24.033 INFO:tasks.workunit.client.0.smithi063.stdout:1/388: dwrite d3/da/dc/d1f/d21/d42/f58 [0,4194304] 0 2024-04-24T22:45:24.034 INFO:tasks.workunit.client.1.smithi192.stdout:6/261: dwrite d0/d8/d18/d2e/f51 [0,4194304] 0 2024-04-24T22:45:24.035 INFO:tasks.workunit.client.1.smithi192.stdout:6/262: chown d0 28 1 2024-04-24T22:45:24.038 INFO:tasks.workunit.client.0.smithi063.stdout:4/321: sync 2024-04-24T22:45:24.038 INFO:tasks.workunit.client.0.smithi063.stdout:9/342: sync 2024-04-24T22:45:24.040 INFO:tasks.workunit.client.0.smithi063.stdout:4/322: chown d2/d8/d36/l46 65380606 1 2024-04-24T22:45:24.042 INFO:tasks.workunit.client.1.smithi192.stdout:7/271: rmdir d2/db 39 2024-04-24T22:45:24.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/272: fdatasync d2/fa 0 2024-04-24T22:45:24.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/263: dwrite d0/d8/d3d/d3e/f4b [0,4194304] 0 2024-04-24T22:45:24.050 INFO:tasks.workunit.client.0.smithi063.stdout:3/353: mkdir da/d18/d47/d7b 0 2024-04-24T22:45:24.058 INFO:tasks.workunit.client.1.smithi192.stdout:0/364: unlink d3/f5a 0 2024-04-24T22:45:24.059 INFO:tasks.workunit.client.0.smithi063.stdout:6/358: mknod db/d1d/d20/d3e/d47/c77 0 2024-04-24T22:45:24.060 INFO:tasks.workunit.client.0.smithi063.stdout:7/286: rmdir d0/d4/de/d17/d35 39 2024-04-24T22:45:24.062 INFO:tasks.workunit.client.1.smithi192.stdout:3/295: symlink d2/l5e 0 2024-04-24T22:45:24.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/389: creat d3/da/dc/d1f/f7f x:0 0 0 2024-04-24T22:45:24.063 INFO:tasks.workunit.client.1.smithi192.stdout:2/385: sync 2024-04-24T22:45:24.064 INFO:tasks.workunit.client.0.smithi063.stdout:1/390: chown d3/da/dc/d1f/d21/d42 33541 1 2024-04-24T22:45:24.064 INFO:tasks.workunit.client.1.smithi192.stdout:3/296: write d2/d4/d12/d1a/d1d/f25 [1086716,32980] 0 2024-04-24T22:45:24.065 INFO:tasks.workunit.client.1.smithi192.stdout:3/297: fdatasync d2/d4/d12/f1c 0 2024-04-24T22:45:24.066 INFO:tasks.workunit.client.0.smithi063.stdout:3/354: mknod da/d48/c7c 0 2024-04-24T22:45:24.070 INFO:tasks.workunit.client.0.smithi063.stdout:2/346: rename d0/d2/d54/d7/f67 to d0/f69 0 2024-04-24T22:45:24.071 INFO:tasks.workunit.client.1.smithi192.stdout:1/241: fdatasync d9/dd/f2b 0 2024-04-24T22:45:24.072 INFO:tasks.workunit.client.0.smithi063.stdout:6/359: mknod db/d1d/d20/d29/d5d/c78 0 2024-04-24T22:45:24.074 INFO:tasks.workunit.client.0.smithi063.stdout:6/360: chown db/d1d/d20/d3e/f4f 0 1 2024-04-24T22:45:24.075 INFO:tasks.workunit.client.1.smithi192.stdout:8/265: truncate f3 4831360 0 2024-04-24T22:45:24.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/242: dread d9/dd/f2b [4194304,4194304] 0 2024-04-24T22:45:24.079 INFO:tasks.workunit.client.1.smithi192.stdout:8/266: dread db/f3d [0,4194304] 0 2024-04-24T22:45:24.082 INFO:tasks.workunit.client.1.smithi192.stdout:0/365: unlink d3/ld 0 2024-04-24T22:45:24.088 INFO:tasks.workunit.client.1.smithi192.stdout:7/273: creat d2/db/f50 x:0 0 0 2024-04-24T22:45:24.091 INFO:tasks.workunit.client.1.smithi192.stdout:6/264: link d0/d8/d22/f32 d0/d8/d29/d5a/f60 0 2024-04-24T22:45:24.092 INFO:tasks.workunit.client.1.smithi192.stdout:9/260: creat d3/f58 x:0 0 0 2024-04-24T22:45:24.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/361: write db/d1d/d2d/d59/d68/d70/f67 [931346,45514] 0 2024-04-24T22:45:24.093 INFO:tasks.workunit.client.1.smithi192.stdout:7/274: dread d2/db/d2a/f2b [0,4194304] 0 2024-04-24T22:45:24.094 INFO:tasks.workunit.client.0.smithi063.stdout:5/352: dwrite d5/f6 [0,4194304] 0 2024-04-24T22:45:24.095 INFO:tasks.workunit.client.1.smithi192.stdout:7/275: read - d2/db/d21/f39 zero size 2024-04-24T22:45:24.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/347: creat d0/d2/f6a x:0 0 0 2024-04-24T22:45:24.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/323: dread d2/f26 [0,4194304] 0 2024-04-24T22:45:24.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/265: dwrite d0/d8/d29/d58/f5b [0,4194304] 0 2024-04-24T22:45:24.098 INFO:tasks.workunit.client.0.smithi063.stdout:7/287: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:24.100 INFO:tasks.workunit.client.0.smithi063.stdout:1/391: mknod d3/da/dc/d1f/d21/d42/d6e/c80 0 2024-04-24T22:45:24.100 INFO:tasks.workunit.client.1.smithi192.stdout:8/267: fdatasync f5 0 2024-04-24T22:45:24.101 INFO:tasks.workunit.client.0.smithi063.stdout:2/348: chown d0/d2/d54/d7/f5c 1239 1 2024-04-24T22:45:24.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/362: rmdir db/d1d/d2d/d69 39 2024-04-24T22:45:24.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/224: creat d0/d13/f43 x:0 0 0 2024-04-24T22:45:24.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/343: rename d6/d8/db/d2b/f6a to d6/d2f/f72 0 2024-04-24T22:45:24.106 INFO:tasks.workunit.client.1.smithi192.stdout:3/298: mkdir d2/d4/d12/d1a/d1d/d55/d5f 0 2024-04-24T22:45:24.107 INFO:tasks.workunit.client.0.smithi063.stdout:8/375: dwrite d1/d2/d3/f14 [0,4194304] 0 2024-04-24T22:45:24.109 INFO:tasks.workunit.client.1.smithi192.stdout:3/299: write d2/d27/f4a [246805,46168] 0 2024-04-24T22:45:24.110 INFO:tasks.workunit.client.0.smithi063.stdout:7/288: symlink d0/d26/d5c/d63/l67 0 2024-04-24T22:45:24.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/225: dwrite d0/d13/f3f [0,4194304] 0 2024-04-24T22:45:24.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/226: write d0/d1b/f24 [840563,39038] 0 2024-04-24T22:45:24.113 INFO:tasks.workunit.client.1.smithi192.stdout:1/243: mknod d9/d15/c50 0 2024-04-24T22:45:24.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/363: symlink db/d1d/d2d/d59/d68/d70/l79 0 2024-04-24T22:45:24.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/392: dwrite d3/da/dc/d1f/f71 [0,4194304] 0 2024-04-24T22:45:24.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/289: fdatasync d0/d4/f19 0 2024-04-24T22:45:24.116 INFO:tasks.workunit.client.1.smithi192.stdout:7/276: creat d2/db/d2a/f51 x:0 0 0 2024-04-24T22:45:24.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/324: rename d2/d19/d33/c3f to d2/d8/d36/d3a/d50/c5e 0 2024-04-24T22:45:24.117 INFO:tasks.workunit.client.0.smithi063.stdout:2/349: creat d0/d2/d2d/d31/d68/f6b x:0 0 0 2024-04-24T22:45:24.118 INFO:tasks.workunit.client.1.smithi192.stdout:4/249: getdents d2/d9/d25 0 2024-04-24T22:45:24.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/250: write d2/db/f4e [63463,21454] 0 2024-04-24T22:45:24.123 INFO:tasks.workunit.client.1.smithi192.stdout:8/268: creat db/dc/d22/d31/f4c x:0 0 0 2024-04-24T22:45:24.124 INFO:tasks.workunit.client.0.smithi063.stdout:5/353: getdents d5/d11/d18 0 2024-04-24T22:45:24.124 INFO:tasks.workunit.client.1.smithi192.stdout:8/269: chown db/dc/c21 1986479 1 2024-04-24T22:45:24.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/386: creat d4/d9/f70 x:0 0 0 2024-04-24T22:45:24.126 INFO:tasks.workunit.client.1.smithi192.stdout:2/387: fdatasync d4/d9/d37/f3e 0 2024-04-24T22:45:24.126 INFO:tasks.workunit.client.0.smithi063.stdout:2/350: chown d0/d2/f1b 89 1 2024-04-24T22:45:24.127 INFO:tasks.workunit.client.0.smithi063.stdout:1/393: dwrite d3/da/dc/d1f/d2b/d79/f52 [0,4194304] 0 2024-04-24T22:45:24.134 INFO:tasks.workunit.client.1.smithi192.stdout:9/261: mkdir d3/d59 0 2024-04-24T22:45:24.139 INFO:tasks.workunit.client.1.smithi192.stdout:5/227: rename d0/d1b/d1c/d2c/f35 to d0/d1b/d1c/d2c/f44 0 2024-04-24T22:45:24.140 INFO:tasks.workunit.client.1.smithi192.stdout:5/228: readlink d0/d13/d19/l9 0 2024-04-24T22:45:24.140 INFO:tasks.workunit.client.1.smithi192.stdout:5/229: readlink d0/d1b/l28 0 2024-04-24T22:45:24.141 INFO:tasks.workunit.client.0.smithi063.stdout:2/351: mkdir d0/d2/d6c 0 2024-04-24T22:45:24.142 INFO:tasks.workunit.client.1.smithi192.stdout:7/277: mknod d2/db/d21/c52 0 2024-04-24T22:45:24.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/266: symlink d0/d8/l61 0 2024-04-24T22:45:24.147 INFO:tasks.workunit.client.1.smithi192.stdout:8/270: rename db/dc/d22/f39 to db/dc/d22/d31/f4d 0 2024-04-24T22:45:24.148 INFO:tasks.workunit.client.0.smithi063.stdout:2/352: readlink d0/d2/d27/d3a/l3b 0 2024-04-24T22:45:24.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/394: symlink d3/da/dc/d1f/d21/l81 0 2024-04-24T22:45:24.150 INFO:tasks.workunit.client.1.smithi192.stdout:1/244: mkdir d9/d51 0 2024-04-24T22:45:24.150 INFO:tasks.workunit.client.0.smithi063.stdout:2/353: write d0/d2/d54/d1c/d3f/f42 [1935649,18562] 0 2024-04-24T22:45:24.152 INFO:tasks.workunit.client.0.smithi063.stdout:5/354: creat d5/f80 x:0 0 0 2024-04-24T22:45:24.153 INFO:tasks.workunit.client.1.smithi192.stdout:5/230: truncate d0/d1b/d1c/f27 2738499 0 2024-04-24T22:45:24.153 INFO:tasks.workunit.client.0.smithi063.stdout:4/325: getdents d2/d19/d33 0 2024-04-24T22:45:24.155 INFO:tasks.workunit.client.1.smithi192.stdout:7/278: mknod d2/db/d36/c53 0 2024-04-24T22:45:24.156 INFO:tasks.workunit.client.0.smithi063.stdout:2/354: dwrite d0/d2/d27/f29 [0,4194304] 0 2024-04-24T22:45:24.156 INFO:tasks.workunit.client.1.smithi192.stdout:7/279: write d2/db/d2a/f51 [844552,22856] 0 2024-04-24T22:45:24.157 INFO:tasks.workunit.client.0.smithi063.stdout:1/395: rename d3/da/dc/d1f/d21/d49/f4f to d3/da/f82 0 2024-04-24T22:45:24.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/366: getdents d3/d27/d28/d2e/d4a/d5c 0 2024-04-24T22:45:24.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/355: mkdir d5/d11/d23/d2d/d62/d81 0 2024-04-24T22:45:24.160 INFO:tasks.workunit.client.1.smithi192.stdout:0/367: truncate d3/f55 993487 0 2024-04-24T22:45:24.162 INFO:tasks.workunit.client.1.smithi192.stdout:6/267: creat d0/d8/d3d/f62 x:0 0 0 2024-04-24T22:45:24.163 INFO:tasks.workunit.client.0.smithi063.stdout:1/396: dread d3/da/dc/f3a [0,4194304] 0 2024-04-24T22:45:24.163 INFO:tasks.workunit.client.0.smithi063.stdout:1/397: read d3/f7 [727431,90438] 0 2024-04-24T22:45:24.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/280: dwrite d2/db/d2a/f2e [0,4194304] 0 2024-04-24T22:45:24.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/262: creat d3/d59/f5a x:0 0 0 2024-04-24T22:45:24.165 INFO:tasks.workunit.client.0.smithi063.stdout:9/344: truncate d6/d8/f17 1629591 0 2024-04-24T22:45:24.165 INFO:tasks.workunit.client.1.smithi192.stdout:6/268: write d0/d8/d29/d58/f5b [1663676,101850] 0 2024-04-24T22:45:24.166 INFO:tasks.workunit.client.0.smithi063.stdout:2/355: chown d0/d2/d54/d7/f13 3 1 2024-04-24T22:45:24.168 INFO:tasks.workunit.client.1.smithi192.stdout:2/388: rename d4/c2f to d4/d29/c71 0 2024-04-24T22:45:24.170 INFO:tasks.workunit.client.1.smithi192.stdout:8/271: fdatasync db/dc/d29/f2c 0 2024-04-24T22:45:24.172 INFO:tasks.workunit.client.0.smithi063.stdout:5/356: creat d5/d1d/f82 x:0 0 0 2024-04-24T22:45:24.173 INFO:tasks.workunit.client.1.smithi192.stdout:3/300: creat d2/d4/d7/f60 x:0 0 0 2024-04-24T22:45:24.175 INFO:tasks.workunit.client.0.smithi063.stdout:4/326: link d2/f1c d2/d8/d36/d3a/d50/f5f 0 2024-04-24T22:45:24.176 INFO:tasks.workunit.client.0.smithi063.stdout:4/327: truncate d2/f35 997907 0 2024-04-24T22:45:24.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/245: write d9/d15/f47 [593174,79736] 0 2024-04-24T22:45:24.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/246: chown d9/c17 16033913 1 2024-04-24T22:45:24.178 INFO:tasks.workunit.client.0.smithi063.stdout:7/290: truncate d0/d4/de/f1a 5934896 0 2024-04-24T22:45:24.178 INFO:tasks.workunit.client.0.smithi063.stdout:7/291: rename d0/d26 to d0/d26/d2e/d68 22 2024-04-24T22:45:24.178 INFO:tasks.workunit.client.1.smithi192.stdout:5/231: creat d0/d1b/f45 x:0 0 0 2024-04-24T22:45:24.181 INFO:tasks.workunit.client.0.smithi063.stdout:1/398: creat d3/da/d34/f83 x:0 0 0 2024-04-24T22:45:24.183 INFO:tasks.workunit.client.0.smithi063.stdout:9/345: chown d6/d8/d1f/d20/d22 763873 1 2024-04-24T22:45:24.183 INFO:tasks.workunit.client.0.smithi063.stdout:9/346: read - d6/d2f/d42/f6f zero size 2024-04-24T22:45:24.185 INFO:tasks.workunit.client.0.smithi063.stdout:2/356: creat d0/d2/d2f/f6d x:0 0 0 2024-04-24T22:45:24.189 INFO:tasks.workunit.client.1.smithi192.stdout:9/263: rmdir d3/dd 39 2024-04-24T22:45:24.194 INFO:tasks.workunit.client.1.smithi192.stdout:6/269: write d0/d8/f2b [4603107,61928] 0 2024-04-24T22:45:24.194 INFO:tasks.workunit.client.0.smithi063.stdout:4/328: symlink d2/d19/l60 0 2024-04-24T22:45:24.195 INFO:tasks.workunit.client.0.smithi063.stdout:4/329: write d2/f35 [351925,40873] 0 2024-04-24T22:45:24.196 INFO:tasks.workunit.client.0.smithi063.stdout:4/330: dread - d2/d8/f5b zero size 2024-04-24T22:45:24.201 INFO:tasks.workunit.client.0.smithi063.stdout:2/357: write d0/f69 [334864,38266] 0 2024-04-24T22:45:24.203 INFO:tasks.workunit.client.1.smithi192.stdout:3/301: creat d2/d4/d12/d39/d3a/f61 x:0 0 0 2024-04-24T22:45:24.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/232: creat d0/d13/d3b/d2f/f46 x:0 0 0 2024-04-24T22:45:24.208 INFO:tasks.workunit.client.0.smithi063.stdout:5/357: mkdir d5/d11/d23/d83 0 2024-04-24T22:45:24.209 INFO:tasks.workunit.client.1.smithi192.stdout:0/368: mkdir d3/d27/d28/d2e/d4a/d5c/d80/d83 0 2024-04-24T22:45:24.211 INFO:tasks.workunit.client.1.smithi192.stdout:9/264: creat d3/d59/f5b x:0 0 0 2024-04-24T22:45:24.212 INFO:tasks.workunit.client.0.smithi063.stdout:4/331: mknod d2/d19/c61 0 2024-04-24T22:45:24.213 INFO:tasks.workunit.client.0.smithi063.stdout:5/358: dwrite d5/d1d/f64 [0,4194304] 0 2024-04-24T22:45:24.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/292: mkdir d0/d4/de/d17/d35/d69 0 2024-04-24T22:45:24.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/389: symlink d4/d9/l72 0 2024-04-24T22:45:24.219 INFO:tasks.workunit.client.1.smithi192.stdout:2/390: truncate d4/f5e 339575 0 2024-04-24T22:45:24.222 INFO:tasks.workunit.client.0.smithi063.stdout:4/332: creat d2/d8/d3c/f62 x:0 0 0 2024-04-24T22:45:24.222 INFO:tasks.workunit.client.0.smithi063.stdout:4/333: dread - d2/d8/d3c/f62 zero size 2024-04-24T22:45:24.225 INFO:tasks.workunit.client.1.smithi192.stdout:8/272: creat db/d43/f4e x:0 0 0 2024-04-24T22:45:24.227 INFO:tasks.workunit.client.1.smithi192.stdout:2/391: dwrite d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:24.229 INFO:tasks.workunit.client.1.smithi192.stdout:1/247: mknod d9/d15/d2d/c52 0 2024-04-24T22:45:24.232 INFO:tasks.workunit.client.1.smithi192.stdout:5/233: mknod d0/d1b/d1c/c47 0 2024-04-24T22:45:24.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/281: write d2/db/d21/f22 [1082954,93291] 0 2024-04-24T22:45:24.234 INFO:tasks.workunit.client.1.smithi192.stdout:7/282: dread - d2/db/d3a/f48 zero size 2024-04-24T22:45:24.236 INFO:tasks.workunit.client.0.smithi063.stdout:2/358: creat d0/dd/d62/f6e x:0 0 0 2024-04-24T22:45:24.237 INFO:tasks.workunit.client.0.smithi063.stdout:2/359: read d0/d2/d54/d1c/f2a [2483683,98520] 0 2024-04-24T22:45:24.238 INFO:tasks.workunit.client.0.smithi063.stdout:4/334: symlink d2/d8/d36/d3a/d50/l63 0 2024-04-24T22:45:24.239 INFO:tasks.workunit.client.1.smithi192.stdout:6/270: mkdir d0/d8/d29/d5a/d63 0 2024-04-24T22:45:24.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/399: link d3/da/dc/d1f/d21/l25 d3/da/dc/d1f/d21/l84 0 2024-04-24T22:45:24.241 INFO:tasks.workunit.client.1.smithi192.stdout:8/273: creat db/dc/d29/f4f x:0 0 0 2024-04-24T22:45:24.242 INFO:tasks.workunit.client.1.smithi192.stdout:8/274: readlink db/l16 0 2024-04-24T22:45:24.242 INFO:tasks.workunit.client.0.smithi063.stdout:2/360: chown d0/d2/d54/d7/f64 26247925 1 2024-04-24T22:45:24.243 INFO:tasks.workunit.client.1.smithi192.stdout:3/302: mkdir d2/d62 0 2024-04-24T22:45:24.243 INFO:tasks.workunit.client.1.smithi192.stdout:3/303: stat d2/d4/d12/f42 0 2024-04-24T22:45:24.244 INFO:tasks.workunit.client.0.smithi063.stdout:5/359: rename d5/d1d/d7d/f3f to d5/d24/d49/f84 0 2024-04-24T22:45:24.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/234: creat d0/d13/d19/f48 x:0 0 0 2024-04-24T22:45:24.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/361: truncate d0/d2/f1b 657257 0 2024-04-24T22:45:24.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/283: symlink d2/db/d25/l54 0 2024-04-24T22:45:24.253 INFO:tasks.workunit.client.1.smithi192.stdout:0/369: truncate d3/d27/f43 1999641 0 2024-04-24T22:45:24.254 INFO:tasks.workunit.client.1.smithi192.stdout:0/370: write d3/d27/d28/d2e/d4a/f4d [1275873,103177] 0 2024-04-24T22:45:24.254 INFO:tasks.workunit.client.0.smithi063.stdout:5/360: unlink d5/d1d/d7d/c68 0 2024-04-24T22:45:24.256 INFO:tasks.workunit.client.0.smithi063.stdout:5/361: write d5/d11/d23/d2d/d62/d73/f75 [2409112,85265] 0 2024-04-24T22:45:24.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/335: creat d2/d8/d40/d51/f64 x:0 0 0 2024-04-24T22:45:24.258 INFO:tasks.workunit.client.1.smithi192.stdout:8/275: creat db/dc/d22/d31/f50 x:0 0 0 2024-04-24T22:45:24.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/293: getdents d0/d26/d2e 0 2024-04-24T22:45:24.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/294: chown d0/d2f/f5a 6449 1 2024-04-24T22:45:24.260 INFO:tasks.workunit.client.1.smithi192.stdout:3/304: creat d2/d27/d28/f63 x:0 0 0 2024-04-24T22:45:24.262 INFO:tasks.workunit.client.1.smithi192.stdout:3/305: truncate d2/f29 130031 0 2024-04-24T22:45:24.262 INFO:tasks.workunit.client.1.smithi192.stdout:2/392: mknod d4/c73 0 2024-04-24T22:45:24.264 INFO:tasks.workunit.client.1.smithi192.stdout:8/276: dwrite db/d43/f4e [0,4194304] 0 2024-04-24T22:45:24.264 INFO:tasks.workunit.client.0.smithi063.stdout:0/317: sync 2024-04-24T22:45:24.265 INFO:tasks.workunit.client.1.smithi192.stdout:1/248: rename d9/d15/c33 to d9/d15/d2d/c53 0 2024-04-24T22:45:24.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/362: mknod d5/d11/d23/d2d/d62/d73/c85 0 2024-04-24T22:45:24.271 INFO:tasks.workunit.client.1.smithi192.stdout:7/284: truncate d2/db/d2a/f2b 1025177 0 2024-04-24T22:45:24.272 INFO:tasks.workunit.client.1.smithi192.stdout:7/285: write f0 [915178,97481] 0 2024-04-24T22:45:24.273 INFO:tasks.workunit.client.1.smithi192.stdout:7/286: chown d2/db/d36/c53 0 1 2024-04-24T22:45:24.275 INFO:tasks.workunit.client.0.smithi063.stdout:2/362: write d0/f57 [4128471,92374] 0 2024-04-24T22:45:24.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/318: creat d6/da/dc/f71 x:0 0 0 2024-04-24T22:45:24.277 INFO:tasks.workunit.client.1.smithi192.stdout:6/271: symlink d0/d8/d3d/d3e/d40/l64 0 2024-04-24T22:45:24.283 INFO:tasks.workunit.client.1.smithi192.stdout:3/306: symlink d2/d27/d28/d38/l64 0 2024-04-24T22:45:24.285 INFO:tasks.workunit.client.1.smithi192.stdout:8/277: symlink db/dc/d29/l51 0 2024-04-24T22:45:24.286 INFO:tasks.workunit.client.1.smithi192.stdout:0/371: write d3/f47 [2792289,114408] 0 2024-04-24T22:45:24.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/265: rename d3/dd/f57 to d3/dd/d53/f5c 0 2024-04-24T22:45:24.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/266: read - d3/f45 zero size 2024-04-24T22:45:24.292 INFO:tasks.workunit.client.1.smithi192.stdout:5/235: mknod d0/c49 0 2024-04-24T22:45:24.295 INFO:tasks.workunit.client.1.smithi192.stdout:2/393: mknod d4/d9/c74 0 2024-04-24T22:45:24.298 INFO:tasks.workunit.client.1.smithi192.stdout:0/372: rename d3/f2c to d3/d27/d3c/d64/f84 0 2024-04-24T22:45:24.300 INFO:tasks.workunit.client.1.smithi192.stdout:6/272: truncate d0/d8/f33 854733 0 2024-04-24T22:45:24.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/307: getdents d2/d62 0 2024-04-24T22:45:24.301 INFO:tasks.workunit.client.1.smithi192.stdout:9/267: mkdir d3/d5d 0 2024-04-24T22:45:24.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/394: unlink d4/d9/l68 0 2024-04-24T22:45:24.306 INFO:tasks.workunit.client.1.smithi192.stdout:3/308: dread d2/f33 [0,4194304] 0 2024-04-24T22:45:24.308 INFO:tasks.workunit.client.1.smithi192.stdout:9/268: chown d3/f21 90227 1 2024-04-24T22:45:24.308 INFO:tasks.workunit.client.1.smithi192.stdout:1/249: link d9/d15/d2d/f37 d9/d51/f54 0 2024-04-24T22:45:24.309 INFO:tasks.workunit.client.1.smithi192.stdout:9/269: chown d3/dd/c25 72 1 2024-04-24T22:45:24.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/236: creat d0/f4a x:0 0 0 2024-04-24T22:45:24.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/363: creat d0/d2/d54/d48/f6f x:0 0 0 2024-04-24T22:45:24.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/250: chown d9/d15/d2d/d31/d2f/f3a 19 1 2024-04-24T22:45:24.310 INFO:tasks.workunit.client.1.smithi192.stdout:0/373: link d3/f48 d3/d27/d28/d31/f85 0 2024-04-24T22:45:24.311 INFO:tasks.workunit.client.1.smithi192.stdout:7/287: link d2/db/d3a/l3e d2/db/d21/l55 0 2024-04-24T22:45:24.311 INFO:tasks.workunit.client.0.smithi063.stdout:4/336: dwrite d2/d8/d36/d3a/d50/f5f [0,4194304] 0 2024-04-24T22:45:24.312 INFO:tasks.workunit.client.1.smithi192.stdout:2/395: dread d4/d9/d37/f4e [0,4194304] 0 2024-04-24T22:45:24.312 INFO:tasks.workunit.client.0.smithi063.stdout:5/363: dread d5/d24/d49/d3b/f48 [0,4194304] 0 2024-04-24T22:45:24.315 INFO:tasks.workunit.client.1.smithi192.stdout:6/273: rename d0/d8/l12 to d0/d8/d3d/d3e/d40/l65 0 2024-04-24T22:45:24.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/270: chown d3/dd/de/d17/d33/f50 1605440 1 2024-04-24T22:45:24.317 INFO:tasks.workunit.client.0.smithi063.stdout:5/364: creat d5/d24/f86 x:0 0 0 2024-04-24T22:45:24.318 INFO:tasks.workunit.client.0.smithi063.stdout:0/319: dread d6/da/f29 [0,4194304] 0 2024-04-24T22:45:24.318 INFO:tasks.workunit.client.0.smithi063.stdout:7/295: dwrite d0/d4/de/f10 [0,4194304] 0 2024-04-24T22:45:24.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/288: symlink d2/db/l56 0 2024-04-24T22:45:24.324 INFO:tasks.workunit.client.1.smithi192.stdout:9/271: dwrite d3/dd/f49 [0,4194304] 0 2024-04-24T22:45:24.325 INFO:tasks.workunit.client.0.smithi063.stdout:0/320: unlink d6/d64/d21/l47 0 2024-04-24T22:45:24.327 INFO:tasks.workunit.client.0.smithi063.stdout:0/321: readlink d6/da/dc/l1c 0 2024-04-24T22:45:24.327 INFO:tasks.workunit.client.0.smithi063.stdout:5/365: dwrite d5/d11/d76/f7b [0,4194304] 0 2024-04-24T22:45:24.333 INFO:tasks.workunit.client.0.smithi063.stdout:7/296: creat d0/d4/de/d17/d28/d31/f6a x:0 0 0 2024-04-24T22:45:24.334 INFO:tasks.workunit.client.0.smithi063.stdout:7/297: chown d0/d26/d2e/l65 41667 1 2024-04-24T22:45:24.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/309: mkdir d2/d4/d12/d65 0 2024-04-24T22:45:24.340 INFO:tasks.workunit.client.0.smithi063.stdout:0/322: creat d6/d64/d24/f72 x:0 0 0 2024-04-24T22:45:24.340 INFO:tasks.workunit.client.1.smithi192.stdout:6/274: mknod d0/d8/d29/d5a/c66 0 2024-04-24T22:45:24.341 INFO:tasks.workunit.client.1.smithi192.stdout:6/275: truncate d0/d8/d29/f3b 1017037 0 2024-04-24T22:45:24.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/251: dread d9/f1a [0,4194304] 0 2024-04-24T22:45:24.345 INFO:tasks.workunit.client.1.smithi192.stdout:1/252: truncate d9/d15/d2d/d31/f32 4198374 0 2024-04-24T22:45:24.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/337: getdents d2/d8/d40 0 2024-04-24T22:45:24.346 INFO:tasks.workunit.client.1.smithi192.stdout:1/253: chown d9/f4f 54 1 2024-04-24T22:45:24.347 INFO:tasks.workunit.client.1.smithi192.stdout:1/254: chown d9/d15/d2d/d31/d2f/f3a 117 1 2024-04-24T22:45:24.347 INFO:tasks.workunit.client.0.smithi063.stdout:3/355: sync 2024-04-24T22:45:24.348 INFO:tasks.workunit.client.1.smithi192.stdout:7/289: write d2/f1a [4818296,30798] 0 2024-04-24T22:45:24.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/255: chown d9/f4f 9796756 1 2024-04-24T22:45:24.351 INFO:tasks.workunit.client.1.smithi192.stdout:7/290: truncate d2/db/d21/f28 4759207 0 2024-04-24T22:45:24.353 INFO:tasks.workunit.client.1.smithi192.stdout:7/291: fdatasync d2/db/f20 0 2024-04-24T22:45:24.353 INFO:tasks.workunit.client.1.smithi192.stdout:2/396: dwrite d4/f69 [0,4194304] 0 2024-04-24T22:45:24.354 INFO:tasks.workunit.client.0.smithi063.stdout:4/338: rename d2/d8/d36/d3a/d50/l63 to d2/d19/l65 0 2024-04-24T22:45:24.354 INFO:tasks.workunit.client.0.smithi063.stdout:0/323: dread d6/f7 [4194304,4194304] 0 2024-04-24T22:45:24.356 INFO:tasks.workunit.client.0.smithi063.stdout:3/356: symlink da/d18/d3a/d63/l7d 0 2024-04-24T22:45:24.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/374: creat d3/d27/d3c/d5d/f86 x:0 0 0 2024-04-24T22:45:24.359 INFO:tasks.workunit.client.0.smithi063.stdout:7/298: truncate d0/d4/de/d17/f42 557080 0 2024-04-24T22:45:24.359 INFO:tasks.workunit.client.1.smithi192.stdout:0/375: read d3/d27/d28/f20 [3067372,7158] 0 2024-04-24T22:45:24.360 INFO:tasks.workunit.client.0.smithi063.stdout:4/339: creat d2/d8/d40/d51/f66 x:0 0 0 2024-04-24T22:45:24.360 INFO:tasks.workunit.client.0.smithi063.stdout:4/340: dread - d2/d8/d40/d51/f66 zero size 2024-04-24T22:45:24.361 INFO:tasks.workunit.client.1.smithi192.stdout:3/310: chown d2/d27/c3d 2 1 2024-04-24T22:45:24.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/376: read d3/d27/f30 [214976,16660] 0 2024-04-24T22:45:24.362 INFO:tasks.workunit.client.0.smithi063.stdout:0/324: rmdir d6/da 39 2024-04-24T22:45:24.362 INFO:tasks.workunit.client.1.smithi192.stdout:0/377: dread - d3/d27/d28/d2e/d4a/f72 zero size 2024-04-24T22:45:24.363 INFO:tasks.workunit.client.0.smithi063.stdout:0/325: stat d6/dd/f3f 0 2024-04-24T22:45:24.363 INFO:tasks.workunit.client.1.smithi192.stdout:3/311: read d2/d4/f14 [478214,109022] 0 2024-04-24T22:45:24.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/276: symlink d0/d8/d3d/l67 0 2024-04-24T22:45:24.364 INFO:tasks.workunit.client.1.smithi192.stdout:6/277: fsync d0/d8/d3d/d3e/d40/f47 0 2024-04-24T22:45:24.365 INFO:tasks.workunit.client.0.smithi063.stdout:3/357: rename da/d18/d47/d4d/c55 to da/d39/c7e 0 2024-04-24T22:45:24.365 INFO:tasks.workunit.client.0.smithi063.stdout:3/358: fsync da/d48/f4a 0 2024-04-24T22:45:24.366 INFO:tasks.workunit.client.0.smithi063.stdout:3/359: chown da/d18/d47/d4d/f72 126057 1 2024-04-24T22:45:24.368 INFO:tasks.workunit.client.1.smithi192.stdout:1/256: unlink d9/d15/d2d/d31/d2f/f3e 0 2024-04-24T22:45:24.368 INFO:tasks.workunit.client.0.smithi063.stdout:4/341: dread - d2/f29 zero size 2024-04-24T22:45:24.369 INFO:tasks.workunit.client.1.smithi192.stdout:9/272: truncate d3/fb 912559 0 2024-04-24T22:45:24.370 INFO:tasks.workunit.client.1.smithi192.stdout:3/312: dwrite d2/d4/d7/d4f/f52 [0,4194304] 0 2024-04-24T22:45:24.371 INFO:tasks.workunit.client.1.smithi192.stdout:3/313: read d2/d4/f14 [514069,81142] 0 2024-04-24T22:45:24.372 INFO:tasks.workunit.client.1.smithi192.stdout:3/314: write d2/d27/d28/f63 [594436,91068] 0 2024-04-24T22:45:24.374 INFO:tasks.workunit.client.1.smithi192.stdout:3/315: dread - d2/d4/d7/d4f/f5a zero size 2024-04-24T22:45:24.375 INFO:tasks.workunit.client.1.smithi192.stdout:7/292: readlink d2/l3f 0 2024-04-24T22:45:24.375 INFO:tasks.workunit.client.1.smithi192.stdout:3/316: readlink d2/d4/d12/d1a/d1d/l58 0 2024-04-24T22:45:24.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/299: creat d0/d4/de/d17/d35/d69/f6b x:0 0 0 2024-04-24T22:45:24.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/397: stat d4/d9/f5a 0 2024-04-24T22:45:24.378 INFO:tasks.workunit.client.0.smithi063.stdout:0/326: dwrite d6/da/dc/f71 [0,4194304] 0 2024-04-24T22:45:24.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/360: mknod da/d39/c7f 0 2024-04-24T22:45:24.380 INFO:tasks.workunit.client.0.smithi063.stdout:7/300: write d0/d4/de/d17/d28/f64 [259828,48093] 0 2024-04-24T22:45:24.382 INFO:tasks.workunit.client.1.smithi192.stdout:7/293: dwrite d2/db/d2a/f3c [4194304,4194304] 0 2024-04-24T22:45:24.389 INFO:tasks.workunit.client.1.smithi192.stdout:0/378: creat d3/d27/d6a/f87 x:0 0 0 2024-04-24T22:45:24.391 INFO:tasks.workunit.client.0.smithi063.stdout:0/327: unlink l3 0 2024-04-24T22:45:24.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/237: getdents d0/d1b/d1c/d2c 0 2024-04-24T22:45:24.394 INFO:tasks.workunit.client.0.smithi063.stdout:3/361: creat da/d18/d47/d4d/d62/f80 x:0 0 0 2024-04-24T22:45:24.395 INFO:tasks.workunit.client.1.smithi192.stdout:6/278: creat d0/d8/d3d/d3e/d40/f68 x:0 0 0 2024-04-24T22:45:24.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/301: mkdir d0/d4/de/d17/d35/d6c 0 2024-04-24T22:45:24.398 INFO:tasks.workunit.client.0.smithi063.stdout:4/342: creat d2/d8/d5c/f67 x:0 0 0 2024-04-24T22:45:24.399 INFO:tasks.workunit.client.0.smithi063.stdout:4/343: read d2/f1c [1953771,7349] 0 2024-04-24T22:45:24.400 INFO:tasks.workunit.client.0.smithi063.stdout:0/328: creat d6/d64/d24/f73 x:0 0 0 2024-04-24T22:45:24.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/273: mkdir d3/dd/de/d39/d5e 0 2024-04-24T22:45:24.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/274: stat d3/d2c/f54 0 2024-04-24T22:45:24.402 INFO:tasks.workunit.client.0.smithi063.stdout:3/362: creat da/d40/f81 x:0 0 0 2024-04-24T22:45:24.403 INFO:tasks.workunit.client.0.smithi063.stdout:7/302: stat d0/d4/c1d 0 2024-04-24T22:45:24.403 INFO:tasks.workunit.client.0.smithi063.stdout:3/363: read - da/d18/f58 zero size 2024-04-24T22:45:24.404 INFO:tasks.workunit.client.1.smithi192.stdout:3/317: rmdir d2/d4/d7/d4f 39 2024-04-24T22:45:24.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/364: write da/d18/d3a/f69 [372612,8759] 0 2024-04-24T22:45:24.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/257: dwrite d9/dd/ff [0,4194304] 0 2024-04-24T22:45:24.406 INFO:tasks.workunit.client.0.smithi063.stdout:4/344: dwrite d2/d8/d5c/f67 [0,4194304] 0 2024-04-24T22:45:24.411 INFO:tasks.workunit.client.1.smithi192.stdout:7/294: unlink d2/c26 0 2024-04-24T22:45:24.413 INFO:tasks.workunit.client.0.smithi063.stdout:3/365: dwrite da/d40/f78 [0,4194304] 0 2024-04-24T22:45:24.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/329: fsync d6/d64/d21/f36 0 2024-04-24T22:45:24.417 INFO:tasks.workunit.client.1.smithi192.stdout:0/379: chown d3/d27/c49 9638 1 2024-04-24T22:45:24.418 INFO:tasks.workunit.client.1.smithi192.stdout:2/398: dwrite d4/d29/f26 [4194304,4194304] 0 2024-04-24T22:45:24.418 INFO:tasks.workunit.client.0.smithi063.stdout:4/345: creat d2/d8/d40/f68 x:0 0 0 2024-04-24T22:45:24.424 INFO:tasks.workunit.client.1.smithi192.stdout:5/238: dread d0/d1b/f37 [0,4194304] 0 2024-04-24T22:45:24.424 INFO:tasks.workunit.client.1.smithi192.stdout:6/279: dread d0/d8/f33 [0,4194304] 0 2024-04-24T22:45:24.425 INFO:tasks.workunit.client.1.smithi192.stdout:6/280: chown d0/cd 44214 1 2024-04-24T22:45:24.426 INFO:tasks.workunit.client.1.smithi192.stdout:6/281: chown d0/d8/d29/f30 6525 1 2024-04-24T22:45:24.429 INFO:tasks.workunit.client.0.smithi063.stdout:6/364: sync 2024-04-24T22:45:24.429 INFO:tasks.workunit.client.0.smithi063.stdout:8/376: sync 2024-04-24T22:45:24.429 INFO:tasks.workunit.client.0.smithi063.stdout:9/347: sync 2024-04-24T22:45:24.431 INFO:tasks.workunit.client.0.smithi063.stdout:8/377: truncate d1/d21/f61 583460 0 2024-04-24T22:45:24.431 INFO:tasks.workunit.client.0.smithi063.stdout:9/348: stat d6/d8/db/d2b/d36 0 2024-04-24T22:45:24.436 INFO:tasks.workunit.client.0.smithi063.stdout:8/378: dwrite d1/d11/d32/d38/f42 [0,4194304] 0 2024-04-24T22:45:24.471 INFO:tasks.workunit.client.0.smithi063.stdout:1/400: sync 2024-04-24T22:45:24.471 INFO:tasks.workunit.client.0.smithi063.stdout:2/364: sync 2024-04-24T22:45:24.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/366: sync 2024-04-24T22:45:24.475 INFO:tasks.workunit.client.0.smithi063.stdout:1/401: dwrite d3/f19 [0,4194304] 0 2024-04-24T22:45:24.520 INFO:tasks.workunit.client.0.smithi063.stdout:7/303: symlink d0/d4/de/l6d 0 2024-04-24T22:45:24.523 INFO:tasks.workunit.client.1.smithi192.stdout:1/258: chown d9/dd/c45 1820 1 2024-04-24T22:45:24.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/295: mkdir d2/db/d2a/d57 0 2024-04-24T22:45:24.525 INFO:tasks.workunit.client.0.smithi063.stdout:4/346: creat d2/d8/d40/d51/f69 x:0 0 0 2024-04-24T22:45:24.527 INFO:tasks.workunit.client.0.smithi063.stdout:6/365: write db/d1d/d20/f71 [767896,104192] 0 2024-04-24T22:45:24.527 INFO:tasks.workunit.client.0.smithi063.stdout:9/349: chown d6/d8/f17 247694643 1 2024-04-24T22:45:24.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/379: mknod d1/d2/d3/c7e 0 2024-04-24T22:45:24.530 INFO:tasks.workunit.client.1.smithi192.stdout:2/399: mkdir d4/d29/d60/d75 0 2024-04-24T22:45:24.533 INFO:tasks.workunit.client.1.smithi192.stdout:5/239: creat d0/d13/d19/f4b x:0 0 0 2024-04-24T22:45:24.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/365: symlink d0/dd/d62/l70 0 2024-04-24T22:45:24.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/366: readlink d0/d2/d39/l4c 0 2024-04-24T22:45:24.535 INFO:tasks.workunit.client.1.smithi192.stdout:3/318: creat d2/d4/d12/d65/f66 x:0 0 0 2024-04-24T22:45:24.537 INFO:tasks.workunit.client.1.smithi192.stdout:1/259: chown d9/dd/c1d 0 1 2024-04-24T22:45:24.538 INFO:tasks.workunit.client.1.smithi192.stdout:7/296: mknod d2/db/d3a/c58 0 2024-04-24T22:45:24.539 INFO:tasks.workunit.client.0.smithi063.stdout:0/330: mkdir d6/d74 0 2024-04-24T22:45:24.540 INFO:tasks.workunit.client.0.smithi063.stdout:0/331: write d6/da/f10 [2553627,108703] 0 2024-04-24T22:45:24.540 INFO:tasks.workunit.client.0.smithi063.stdout:4/347: chown d2/d8/l3d 198 1 2024-04-24T22:45:24.542 INFO:tasks.workunit.client.0.smithi063.stdout:6/366: rename db/d1d/d2d/d59 to db/d1d/d20/d3f/d5f/d7a 0 2024-04-24T22:45:24.543 INFO:tasks.workunit.client.0.smithi063.stdout:0/332: dread d6/da/dc/f71 [0,4194304] 0 2024-04-24T22:45:24.544 INFO:tasks.workunit.client.1.smithi192.stdout:0/380: creat d3/d27/d28/d81/f88 x:0 0 0 2024-04-24T22:45:24.545 INFO:tasks.workunit.client.0.smithi063.stdout:0/333: dread - d6/d64/d24/f73 zero size 2024-04-24T22:45:24.545 INFO:tasks.workunit.client.0.smithi063.stdout:9/350: rename d6/d2f/l39 to d6/d2f/l73 0 2024-04-24T22:45:24.546 INFO:tasks.workunit.client.1.smithi192.stdout:7/297: truncate d2/db/d21/f39 67550 0 2024-04-24T22:45:24.547 INFO:tasks.workunit.client.0.smithi063.stdout:7/304: dread d0/d4/de/d17/f42 [0,4194304] 0 2024-04-24T22:45:24.549 INFO:tasks.workunit.client.0.smithi063.stdout:7/305: read - d0/d4/de/f60 zero size 2024-04-24T22:45:24.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/240: getdents d0/d13/d3b/d2f/d34 0 2024-04-24T22:45:24.550 INFO:tasks.workunit.client.0.smithi063.stdout:7/306: read d0/fa [2442895,37414] 0 2024-04-24T22:45:24.552 INFO:tasks.workunit.client.1.smithi192.stdout:5/241: readlink d0/d13/d19/l9 0 2024-04-24T22:45:24.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/275: getdents d3/dd/d53 0 2024-04-24T22:45:24.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/366: dwrite da/d39/f41 [0,4194304] 0 2024-04-24T22:45:24.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/381: dwrite d3/d27/d28/d2e/d4a/f72 [0,4194304] 0 2024-04-24T22:45:24.554 INFO:tasks.workunit.client.0.smithi063.stdout:3/367: write da/d18/d47/d4d/f5a [2763416,7051] 0 2024-04-24T22:45:24.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/298: symlink d2/db/d25/l59 0 2024-04-24T22:45:24.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/260: symlink d9/d15/d2d/d48/l55 0 2024-04-24T22:45:24.565 INFO:tasks.workunit.client.0.smithi063.stdout:1/402: truncate d3/da/dc/d1f/d2b/d79/f2e 2857592 0 2024-04-24T22:45:24.567 INFO:tasks.workunit.client.1.smithi192.stdout:3/319: dread d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:24.568 INFO:tasks.workunit.client.1.smithi192.stdout:9/276: symlink d3/d2c/l5f 0 2024-04-24T22:45:24.570 INFO:tasks.workunit.client.1.smithi192.stdout:0/382: read d3/f1c [156805,4468] 0 2024-04-24T22:45:24.571 INFO:tasks.workunit.client.1.smithi192.stdout:0/383: truncate d3/d27/d28/f17 4955671 0 2024-04-24T22:45:24.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/334: rename d6/d5b to d6/d64/d21/d25/d62/d75 0 2024-04-24T22:45:24.572 INFO:tasks.workunit.client.0.smithi063.stdout:4/348: dwrite d2/d8/f3e [0,4194304] 0 2024-04-24T22:45:24.573 INFO:tasks.workunit.client.1.smithi192.stdout:1/261: symlink d9/d15/d2d/d31/l56 0 2024-04-24T22:45:24.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/349: write d2/d8/d5c/f67 [2707694,65406] 0 2024-04-24T22:45:24.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/367: creat d5/f87 x:0 0 0 2024-04-24T22:45:24.575 INFO:tasks.workunit.client.0.smithi063.stdout:4/350: read d2/d8/d5c/f67 [1242672,114801] 0 2024-04-24T22:45:24.576 INFO:tasks.workunit.client.1.smithi192.stdout:2/400: rename d4/d9/c6b to d4/d9/c76 0 2024-04-24T22:45:24.579 INFO:tasks.workunit.client.1.smithi192.stdout:9/277: mkdir d3/dd/d53/d60 0 2024-04-24T22:45:24.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/367: creat d0/d2/d63/f71 x:0 0 0 2024-04-24T22:45:24.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/368: dread - d0/d2/d2f/f58 zero size 2024-04-24T22:45:24.621 INFO:tasks.workunit.client.0.smithi063.stdout:7/307: mknod d0/d26/d2e/c6e 0 2024-04-24T22:45:24.624 INFO:tasks.workunit.client.1.smithi192.stdout:7/299: symlink d2/l5a 0 2024-04-24T22:45:24.625 INFO:tasks.workunit.client.1.smithi192.stdout:7/300: write d2/db/f20 [9965749,73347] 0 2024-04-24T22:45:24.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/380: write d1/d22/d33/f57 [680949,33713] 0 2024-04-24T22:45:24.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/403: symlink d3/da/l85 0 2024-04-24T22:45:24.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/404: chown d3/f63 243014 1 2024-04-24T22:45:24.634 INFO:tasks.workunit.client.1.smithi192.stdout:3/320: mknod d2/d4/d12/d1a/d1d/c67 0 2024-04-24T22:45:24.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/301: mkdir d2/db/d21/d42/d5b 0 2024-04-24T22:45:24.645 INFO:tasks.workunit.client.1.smithi192.stdout:2/401: mknod d4/d9/c77 0 2024-04-24T22:45:24.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/351: unlink d2/d19/c2f 0 2024-04-24T22:45:24.656 INFO:tasks.workunit.client.1.smithi192.stdout:3/321: creat d2/d4/d12/d1a/d1d/d55/d5f/f68 x:0 0 0 2024-04-24T22:45:24.657 INFO:tasks.workunit.client.0.smithi063.stdout:2/369: mkdir d0/dd/d72 0 2024-04-24T22:45:24.658 INFO:tasks.workunit.client.1.smithi192.stdout:3/322: chown d2/d4/d12/d1a/d1d/d55/f32 953805 1 2024-04-24T22:45:24.658 INFO:tasks.workunit.client.0.smithi063.stdout:2/370: write d0/d2/d27/f29 [1860354,17027] 0 2024-04-24T22:45:24.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/384: getdents d3/d6d/d79 0 2024-04-24T22:45:24.661 INFO:tasks.workunit.client.1.smithi192.stdout:0/385: chown d3/d27/d28/d31/f85 2071271346 1 2024-04-24T22:45:24.665 INFO:tasks.workunit.client.0.smithi063.stdout:1/405: chown d3/da/l4b 1207 1 2024-04-24T22:45:24.665 INFO:tasks.workunit.client.1.smithi192.stdout:9/278: link d3/d24/f4a d3/d5d/f61 0 2024-04-24T22:45:24.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/323: rmdir d2/d4/d12/d1a/d1d/d55 39 2024-04-24T22:45:24.669 INFO:tasks.workunit.client.1.smithi192.stdout:0/386: mknod d3/d27/d3c/d5d/c89 0 2024-04-24T22:45:24.672 INFO:tasks.workunit.client.1.smithi192.stdout:7/302: link d2/db/c4e d2/db/d49/c5c 0 2024-04-24T22:45:24.672 INFO:tasks.workunit.client.0.smithi063.stdout:1/406: dwrite d3/da/dc/d1f/f7f [0,4194304] 0 2024-04-24T22:45:24.675 INFO:tasks.workunit.client.1.smithi192.stdout:2/402: creat d4/f78 x:0 0 0 2024-04-24T22:45:24.679 INFO:tasks.workunit.client.0.smithi063.stdout:1/407: dwrite d3/da/dc/d1f/d21/f7d [4194304,4194304] 0 2024-04-24T22:45:24.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/403: dread d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:24.683 INFO:tasks.workunit.client.0.smithi063.stdout:2/371: rmdir d0/d2/d2d/d31 39 2024-04-24T22:45:24.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/324: dwrite d2/d4/d7/fd [0,4194304] 0 2024-04-24T22:45:24.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/381: creat d1/d11/d49/d54/d7b/f7f x:0 0 0 2024-04-24T22:45:24.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/382: dread - d1/d11/d49/f7c zero size 2024-04-24T22:45:24.700 INFO:tasks.workunit.client.0.smithi063.stdout:6/367: link db/cc db/d1d/d20/c7b 0 2024-04-24T22:45:24.701 INFO:tasks.workunit.client.0.smithi063.stdout:6/368: chown db/d1d/f4e 63418580 1 2024-04-24T22:45:24.703 INFO:tasks.workunit.client.0.smithi063.stdout:6/369: write db/d1d/d20/f40 [531463,19431] 0 2024-04-24T22:45:24.704 INFO:tasks.workunit.client.1.smithi192.stdout:7/303: link f0 d2/db/d21/d42/d5b/f5d 0 2024-04-24T22:45:24.705 INFO:tasks.workunit.client.0.smithi063.stdout:6/370: fdatasync db/d1d/d20/d3e/f5e 0 2024-04-24T22:45:24.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/408: mknod d3/da/dc/d1f/d2b/d79/c86 0 2024-04-24T22:45:24.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/371: chown db/d1d/d20/d3e/c73 84 1 2024-04-24T22:45:24.709 INFO:tasks.workunit.client.0.smithi063.stdout:3/368: getdents da/d18/d47/d4d/d62 0 2024-04-24T22:45:24.710 INFO:tasks.workunit.client.1.smithi192.stdout:2/404: mknod d4/d9/d10/d6c/d6f/c79 0 2024-04-24T22:45:24.713 INFO:tasks.workunit.client.0.smithi063.stdout:7/308: getdents d0/d4 0 2024-04-24T22:45:24.714 INFO:tasks.workunit.client.1.smithi192.stdout:2/405: mknod d4/d9/d10/d6c/d6f/c7a 0 2024-04-24T22:45:24.718 INFO:tasks.workunit.client.0.smithi063.stdout:2/372: mknod d0/d2/d2d/d31/c73 0 2024-04-24T22:45:24.721 INFO:tasks.workunit.client.1.smithi192.stdout:2/406: creat d4/d29/d60/d75/f7b x:0 0 0 2024-04-24T22:45:24.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/383: dread d1/d2/fd [0,4194304] 0 2024-04-24T22:45:24.722 INFO:tasks.workunit.client.1.smithi192.stdout:1/262: symlink d9/d15/d2d/d31/l57 0 2024-04-24T22:45:24.724 INFO:tasks.workunit.client.0.smithi063.stdout:1/409: rename d3/da/d34/d68/c75 to d3/da/d34/d68/c87 0 2024-04-24T22:45:24.727 INFO:tasks.workunit.client.0.smithi063.stdout:2/373: creat d0/d2/d2d/d31/d68/f74 x:0 0 0 2024-04-24T22:45:24.729 INFO:tasks.workunit.client.0.smithi063.stdout:8/384: dwrite d1/d11/d3c/f69 [0,4194304] 0 2024-04-24T22:45:24.730 INFO:tasks.workunit.client.1.smithi192.stdout:4/251: sync 2024-04-24T22:45:24.731 INFO:tasks.workunit.client.1.smithi192.stdout:4/252: fsync d2/d9/d25/d2e/d31/d36/d5b/f54 0 2024-04-24T22:45:24.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/410: creat d3/da/dc/d11/d5e/f88 x:0 0 0 2024-04-24T22:45:24.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/253: mkdir d2/db/d5d 0 2024-04-24T22:45:24.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/351: dwrite d6/d8/db/d1c/f69 [0,4194304] 0 2024-04-24T22:45:24.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/374: creat d0/dd/f75 x:0 0 0 2024-04-24T22:45:24.742 INFO:tasks.workunit.client.0.smithi063.stdout:2/375: dread - d0/d2/d51/f61 zero size 2024-04-24T22:45:24.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/385: mknod d1/d22/d33/c80 0 2024-04-24T22:45:24.745 INFO:tasks.workunit.client.0.smithi063.stdout:7/309: rmdir d0/d26/d5c/d5e 0 2024-04-24T22:45:24.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/352: dwrite d6/d8/db/d6b/d4f/f5a [0,4194304] 0 2024-04-24T22:45:24.749 INFO:tasks.workunit.client.0.smithi063.stdout:1/411: creat d3/da/dc/d11/f89 x:0 0 0 2024-04-24T22:45:24.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/353: dread - d6/f6c zero size 2024-04-24T22:45:24.750 INFO:tasks.workunit.client.0.smithi063.stdout:2/376: creat d0/d2/d54/d1c/d3f/f76 x:0 0 0 2024-04-24T22:45:24.753 INFO:tasks.workunit.client.1.smithi192.stdout:3/325: creat d2/d4/f69 x:0 0 0 2024-04-24T22:45:24.755 INFO:tasks.workunit.client.0.smithi063.stdout:9/354: read d6/d2f/f72 [2682022,123137] 0 2024-04-24T22:45:24.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/412: mkdir d3/da/d34/d68/d8a 0 2024-04-24T22:45:24.757 INFO:tasks.workunit.client.0.smithi063.stdout:1/413: chown d3/da/dc/d1f/d21/c2a 157 1 2024-04-24T22:45:24.761 INFO:tasks.workunit.client.1.smithi192.stdout:3/326: creat d2/d4/d7/d4b/d54/f6a x:0 0 0 2024-04-24T22:45:24.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/327: chown d2/d4/l21 0 1 2024-04-24T22:45:24.763 INFO:tasks.workunit.client.0.smithi063.stdout:2/377: mkdir d0/dd/d72/d77 0 2024-04-24T22:45:24.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/355: symlink d6/d2f/d42/l74 0 2024-04-24T22:45:24.765 INFO:tasks.workunit.client.1.smithi192.stdout:3/328: mknod d2/d4/d7/d4b/c6b 0 2024-04-24T22:45:24.769 INFO:tasks.workunit.client.1.smithi192.stdout:3/329: dread d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:24.770 INFO:tasks.workunit.client.1.smithi192.stdout:3/330: write d2/d4/d12/d65/f66 [988747,30494] 0 2024-04-24T22:45:24.772 INFO:tasks.workunit.client.0.smithi063.stdout:9/356: mknod d6/d8/db/d2b/c75 0 2024-04-24T22:45:24.773 INFO:tasks.workunit.client.0.smithi063.stdout:2/378: creat d0/dd/d72/d77/f78 x:0 0 0 2024-04-24T22:45:24.774 INFO:tasks.workunit.client.1.smithi192.stdout:3/331: truncate d2/d4/d7/d4f/f5a 624883 0 2024-04-24T22:45:24.777 INFO:tasks.workunit.client.0.smithi063.stdout:4/352: dwrite d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:24.777 INFO:tasks.workunit.client.0.smithi063.stdout:4/353: fsync d2/d19/f37 0 2024-04-24T22:45:24.779 INFO:tasks.workunit.client.0.smithi063.stdout:2/379: unlink d0/d2/d39/c3c 0 2024-04-24T22:45:24.780 INFO:tasks.workunit.client.1.smithi192.stdout:0/387: unlink d3/f5e 0 2024-04-24T22:45:24.780 INFO:tasks.workunit.client.1.smithi192.stdout:0/388: fsync d3/d27/d28/f24 0 2024-04-24T22:45:24.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/332: unlink d2/d4/d7/c10 0 2024-04-24T22:45:24.783 INFO:tasks.workunit.client.1.smithi192.stdout:3/333: chown d2/d4/d7/d41 250470278 1 2024-04-24T22:45:24.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/414: link d3/f7 d3/da/dc/f8b 0 2024-04-24T22:45:24.785 INFO:tasks.workunit.client.0.smithi063.stdout:2/380: dwrite d0/d2/d63/f71 [0,4194304] 0 2024-04-24T22:45:24.790 INFO:tasks.workunit.client.0.smithi063.stdout:6/372: mkdir db/d1d/d7c 0 2024-04-24T22:45:24.792 INFO:tasks.workunit.client.0.smithi063.stdout:1/415: fsync d3/da/f6a 0 2024-04-24T22:45:24.793 INFO:tasks.workunit.client.0.smithi063.stdout:1/416: write d3/da/dc/d1f/f71 [2633588,5218] 0 2024-04-24T22:45:24.794 INFO:tasks.workunit.client.0.smithi063.stdout:2/381: unlink d0/f60 0 2024-04-24T22:45:24.796 INFO:tasks.workunit.client.1.smithi192.stdout:3/334: dwrite d2/d4/d12/d39/d3a/f61 [0,4194304] 0 2024-04-24T22:45:24.800 INFO:tasks.workunit.client.0.smithi063.stdout:2/382: dwrite d0/f57 [0,4194304] 0 2024-04-24T22:45:24.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/386: unlink d1/d11/l2b 0 2024-04-24T22:45:24.804 INFO:tasks.workunit.client.0.smithi063.stdout:5/368: symlink d5/d24/l88 0 2024-04-24T22:45:24.807 INFO:tasks.workunit.client.1.smithi192.stdout:7/304: rename d2/db/c4e to d2/db/d25/c5e 0 2024-04-24T22:45:24.813 INFO:tasks.workunit.client.0.smithi063.stdout:2/383: dread d0/d2/f1b [0,4194304] 0 2024-04-24T22:45:24.816 INFO:tasks.workunit.client.1.smithi192.stdout:2/407: rename d4/d9/d10/l32 to d4/d29/d60/d75/l7c 0 2024-04-24T22:45:24.818 INFO:tasks.workunit.client.0.smithi063.stdout:5/369: creat d5/d24/d2c/f89 x:0 0 0 2024-04-24T22:45:24.819 INFO:tasks.workunit.client.0.smithi063.stdout:5/370: chown d5/d24/d36/c5e 54 1 2024-04-24T22:45:24.820 INFO:tasks.workunit.client.1.smithi192.stdout:2/408: symlink d4/d9/d37/l7d 0 2024-04-24T22:45:24.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/371: write d5/d24/d36/f5a [614181,84053] 0 2024-04-24T22:45:24.821 INFO:tasks.workunit.client.0.smithi063.stdout:5/372: truncate d5/d24/d44/f5c 1039646 0 2024-04-24T22:45:24.822 INFO:tasks.workunit.client.0.smithi063.stdout:0/335: rename d6/f7 to d6/da/d2b/d60/f76 0 2024-04-24T22:45:24.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/336: chown d6/dd/c30 1 1 2024-04-24T22:45:24.823 INFO:tasks.workunit.client.0.smithi063.stdout:8/387: unlink d1/d22/l6e 0 2024-04-24T22:45:24.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/337: chown d6/dd/c30 86 1 2024-04-24T22:45:24.824 INFO:tasks.workunit.client.1.smithi192.stdout:2/409: symlink d4/d9/d10/d6c/d6f/l7e 0 2024-04-24T22:45:24.825 INFO:tasks.workunit.client.0.smithi063.stdout:0/338: write d6/d64/d21/d25/d31/f35 [2616327,67464] 0 2024-04-24T22:45:24.825 INFO:tasks.workunit.client.0.smithi063.stdout:2/384: truncate d0/d2/d54/d7/f64 2329093 0 2024-04-24T22:45:24.827 INFO:tasks.workunit.client.0.smithi063.stdout:1/417: creat d3/da/dc/f8c x:0 0 0 2024-04-24T22:45:24.827 INFO:tasks.workunit.client.0.smithi063.stdout:2/385: write d0/d2/d2f/f6d [332170,62383] 0 2024-04-24T22:45:24.828 INFO:tasks.workunit.client.0.smithi063.stdout:6/373: link db/d1d/d20/d3e/d47/c77 db/d1d/d20/c7d 0 2024-04-24T22:45:24.830 INFO:tasks.workunit.client.0.smithi063.stdout:8/388: dwrite d1/d22/f77 [0,4194304] 0 2024-04-24T22:45:24.830 INFO:tasks.workunit.client.0.smithi063.stdout:1/418: write d3/da/dc/d39/f66 [405744,130081] 0 2024-04-24T22:45:24.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/373: creat d5/d24/d49/d3b/f8a x:0 0 0 2024-04-24T22:45:24.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/279: mkdir d3/dd/d62 0 2024-04-24T22:45:24.834 INFO:tasks.workunit.client.0.smithi063.stdout:3/369: rename da/d18/d47/c5f to da/d18/d47/d7b/c82 0 2024-04-24T22:45:24.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/410: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:24.839 INFO:tasks.workunit.client.0.smithi063.stdout:0/339: symlink d6/da/d2b/l77 0 2024-04-24T22:45:24.842 INFO:tasks.workunit.client.1.smithi192.stdout:9/280: dwrite d3/d2c/f42 [0,4194304] 0 2024-04-24T22:45:24.844 INFO:tasks.workunit.client.1.smithi192.stdout:9/281: chown d3/dd/de/f51 276228826 1 2024-04-24T22:45:24.844 INFO:tasks.workunit.client.1.smithi192.stdout:4/254: write d2/f3 [825481,30006] 0 2024-04-24T22:45:24.852 INFO:tasks.workunit.client.0.smithi063.stdout:5/374: unlink d5/f10 0 2024-04-24T22:45:24.852 INFO:tasks.workunit.client.0.smithi063.stdout:6/374: creat db/d1d/d20/d3f/d5f/d7a/d68/f7e x:0 0 0 2024-04-24T22:45:24.853 INFO:tasks.workunit.client.1.smithi192.stdout:9/282: dwrite d3/f43 [0,4194304] 0 2024-04-24T22:45:24.853 INFO:tasks.workunit.client.1.smithi192.stdout:2/411: rename d4/d29/d60/f57 to d4/d9/d10/f7f 0 2024-04-24T22:45:24.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/412: truncate d4/d29/d60/d75/f7b 689614 0 2024-04-24T22:45:24.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/283: dread d3/dd/f49 [0,4194304] 0 2024-04-24T22:45:24.860 INFO:tasks.workunit.client.1.smithi192.stdout:9/284: dread d3/dd/de/f2f [0,4194304] 0 2024-04-24T22:45:24.860 INFO:tasks.workunit.client.0.smithi063.stdout:3/370: mknod da/d18/d3a/d63/c83 0 2024-04-24T22:45:24.861 INFO:tasks.workunit.client.1.smithi192.stdout:9/285: chown d3/dd/de/d17/f27 16498 1 2024-04-24T22:45:24.862 INFO:tasks.workunit.client.0.smithi063.stdout:3/371: read da/f31 [412786,53108] 0 2024-04-24T22:45:24.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/340: mknod d6/d3a/c78 0 2024-04-24T22:45:24.864 INFO:tasks.workunit.client.0.smithi063.stdout:3/372: write da/d48/f5c [376582,50692] 0 2024-04-24T22:45:24.864 INFO:tasks.workunit.client.0.smithi063.stdout:1/419: symlink d3/l8d 0 2024-04-24T22:45:24.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/341: dread d6/da/f10 [0,4194304] 0 2024-04-24T22:45:24.876 INFO:tasks.workunit.client.1.smithi192.stdout:0/389: write d3/d27/f43 [1389409,95494] 0 2024-04-24T22:45:24.877 INFO:tasks.workunit.client.1.smithi192.stdout:9/286: dwrite d3/dd/de/d17/f27 [0,4194304] 0 2024-04-24T22:45:24.879 INFO:tasks.workunit.client.1.smithi192.stdout:9/287: fdatasync d3/dd/de/f16 0 2024-04-24T22:45:24.879 INFO:tasks.workunit.client.1.smithi192.stdout:9/288: rename d3/d2c to d3/d2c/d63 22 2024-04-24T22:45:24.884 INFO:tasks.workunit.client.0.smithi063.stdout:3/373: mknod da/d18/d3a/c84 0 2024-04-24T22:45:24.886 INFO:tasks.workunit.client.1.smithi192.stdout:0/390: unlink d3/d27/d28/d81/f88 0 2024-04-24T22:45:24.888 INFO:tasks.workunit.client.1.smithi192.stdout:9/289: readlink d3/dd/de/l1b 0 2024-04-24T22:45:24.892 INFO:tasks.workunit.client.1.smithi192.stdout:3/335: write d2/d4/f3e [4347252,52604] 0 2024-04-24T22:45:24.892 INFO:tasks.workunit.client.1.smithi192.stdout:9/290: dread d3/d2c/f42 [0,4194304] 0 2024-04-24T22:45:24.894 INFO:tasks.workunit.client.1.smithi192.stdout:3/336: dread d2/d4/d7/d4f/f5a [0,4194304] 0 2024-04-24T22:45:24.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/337: truncate d2/f1e 754935 0 2024-04-24T22:45:24.896 INFO:tasks.workunit.client.0.smithi063.stdout:7/310: rename d0/d26 to d0/d4/de/d6f 0 2024-04-24T22:45:24.903 INFO:tasks.workunit.client.0.smithi063.stdout:4/354: mknod d2/d8/d40/c6a 0 2024-04-24T22:45:24.904 INFO:tasks.workunit.client.0.smithi063.stdout:3/374: dread da/f28 [0,4194304] 0 2024-04-24T22:45:24.904 INFO:tasks.workunit.client.0.smithi063.stdout:3/375: fdatasync da/d18/d47/d4d/f5a 0 2024-04-24T22:45:24.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/357: rename d6/f14 to d6/d8/db/d6b/d4f/d59/f76 0 2024-04-24T22:45:24.907 INFO:tasks.workunit.client.0.smithi063.stdout:9/358: readlink d6/d8/db/d6b/l5f 0 2024-04-24T22:45:24.908 INFO:tasks.workunit.client.0.smithi063.stdout:9/359: write d6/d8/db/d2b/d36/f40 [3228295,92822] 0 2024-04-24T22:45:24.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/360: mkdir d6/d2f/d42/d77 0 2024-04-24T22:45:24.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/355: creat d2/d8/d36/f6b x:0 0 0 2024-04-24T22:45:24.917 INFO:tasks.workunit.client.0.smithi063.stdout:2/386: dwrite d0/d2/f1b [0,4194304] 0 2024-04-24T22:45:24.917 INFO:tasks.workunit.client.0.smithi063.stdout:2/387: chown d0/l4 7412 1 2024-04-24T22:45:24.918 INFO:tasks.workunit.client.0.smithi063.stdout:0/342: rename d6/d3a/c78 to d6/d64/d21/d25/c79 0 2024-04-24T22:45:24.919 INFO:tasks.workunit.client.0.smithi063.stdout:9/361: mkdir d6/d8/db/d1c/d78 0 2024-04-24T22:45:24.920 INFO:tasks.workunit.client.0.smithi063.stdout:4/356: creat d2/d8/d40/d51/f6c x:0 0 0 2024-04-24T22:45:24.921 INFO:tasks.workunit.client.0.smithi063.stdout:2/388: mknod d0/d2/d35/c79 0 2024-04-24T22:45:24.922 INFO:tasks.workunit.client.0.smithi063.stdout:2/389: stat d0/d2/d54/f25 0 2024-04-24T22:45:24.924 INFO:tasks.workunit.client.0.smithi063.stdout:2/390: chown d0/d2/d54/l38 13355196 1 2024-04-24T22:45:24.925 INFO:tasks.workunit.client.0.smithi063.stdout:9/362: creat d6/d8/db/d6b/d4f/f79 x:0 0 0 2024-04-24T22:45:24.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/363: truncate d6/d8/db/d6b/d4f/f79 354954 0 2024-04-24T22:45:24.928 INFO:tasks.workunit.client.0.smithi063.stdout:9/364: fsync d6/d8/db/d2b/f34 0 2024-04-24T22:45:24.930 INFO:tasks.workunit.client.0.smithi063.stdout:7/311: rename d0/l4c to d0/d4/de/d6f/l70 0 2024-04-24T22:45:24.934 INFO:tasks.workunit.client.0.smithi063.stdout:6/375: dwrite db/d1d/d20/d3e/f4f [0,4194304] 0 2024-04-24T22:45:24.936 INFO:tasks.workunit.client.0.smithi063.stdout:9/365: creat d6/d8/d53/f7a x:0 0 0 2024-04-24T22:45:24.937 INFO:tasks.workunit.client.0.smithi063.stdout:3/376: rename da/d48/f4a to da/d18/d47/f85 0 2024-04-24T22:45:24.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/376: chown db/d1c/c5a 11 1 2024-04-24T22:45:24.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/343: dread d6/da/dc/d26/d33/f51 [0,4194304] 0 2024-04-24T22:45:24.944 INFO:tasks.workunit.client.1.smithi192.stdout:8/278: sync 2024-04-24T22:45:24.945 INFO:tasks.workunit.client.0.smithi063.stdout:9/366: dwrite d6/d8/db/d2b/f34 [0,4194304] 0 2024-04-24T22:45:24.946 INFO:tasks.workunit.client.0.smithi063.stdout:1/420: truncate d3/da/dc/d1f/f7f 1934474 0 2024-04-24T22:45:24.947 INFO:tasks.workunit.client.0.smithi063.stdout:3/377: getdents da/d18/d3a 0 2024-04-24T22:45:24.952 INFO:tasks.workunit.client.0.smithi063.stdout:9/367: creat d6/d2f/d42/f7b x:0 0 0 2024-04-24T22:45:24.953 INFO:tasks.workunit.client.0.smithi063.stdout:9/368: dread - d6/d8/d53/f7a zero size 2024-04-24T22:45:24.954 INFO:tasks.workunit.client.1.smithi192.stdout:3/338: chown d2/d27/d28/c3c 437 1 2024-04-24T22:45:24.954 INFO:tasks.workunit.client.1.smithi192.stdout:9/291: dread d3/dd/de/d17/f23 [0,4194304] 0 2024-04-24T22:45:24.955 INFO:tasks.workunit.client.1.smithi192.stdout:9/292: stat d3/d59/f5b 0 2024-04-24T22:45:24.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/413: getdents d4/d9/d10/d6c/d6f 0 2024-04-24T22:45:24.958 INFO:tasks.workunit.client.0.smithi063.stdout:7/312: dread d0/d4/de/f1a [0,4194304] 0 2024-04-24T22:45:24.958 INFO:tasks.workunit.client.0.smithi063.stdout:2/391: write d0/d2/d54/d7/f53 [4897591,127546] 0 2024-04-24T22:45:24.958 INFO:tasks.workunit.client.0.smithi063.stdout:3/378: dwrite da/f3b [4194304,4194304] 0 2024-04-24T22:45:24.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/421: mknod d3/da/d34/d6d/c8e 0 2024-04-24T22:45:24.964 INFO:tasks.workunit.client.1.smithi192.stdout:3/339: dwrite d2/f13 [0,4194304] 0 2024-04-24T22:45:24.968 INFO:tasks.workunit.client.0.smithi063.stdout:7/313: dwrite d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:24.971 INFO:tasks.workunit.client.1.smithi192.stdout:8/279: dwrite db/dc/f18 [0,4194304] 0 2024-04-24T22:45:24.972 INFO:tasks.workunit.client.1.smithi192.stdout:8/280: write db/dc/d29/f4f [297353,96337] 0 2024-04-24T22:45:24.976 INFO:tasks.workunit.client.1.smithi192.stdout:2/414: dread d4/d9/d37/f4e [0,4194304] 0 2024-04-24T22:45:24.977 INFO:tasks.workunit.client.1.smithi192.stdout:2/415: read d4/d29/f34 [30104,119473] 0 2024-04-24T22:45:24.981 INFO:tasks.workunit.client.1.smithi192.stdout:9/293: rename d3/dd/de/d17/d33/f50 to d3/dd/de/f64 0 2024-04-24T22:45:24.982 INFO:tasks.workunit.client.1.smithi192.stdout:3/340: fdatasync d2/d4/d12/d1a/f5d 0 2024-04-24T22:45:24.982 INFO:tasks.workunit.client.1.smithi192.stdout:9/294: fdatasync d3/f4 0 2024-04-24T22:45:24.986 INFO:tasks.workunit.client.1.smithi192.stdout:2/416: dwrite d4/d9/d37/f44 [0,4194304] 0 2024-04-24T22:45:24.987 INFO:tasks.workunit.client.1.smithi192.stdout:8/281: mknod db/c52 0 2024-04-24T22:45:24.988 INFO:tasks.workunit.client.1.smithi192.stdout:8/282: chown f7 4909 1 2024-04-24T22:45:24.988 INFO:tasks.workunit.client.1.smithi192.stdout:9/295: creat d3/d24/f65 x:0 0 0 2024-04-24T22:45:24.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/341: symlink d2/d62/l6c 0 2024-04-24T22:45:24.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/342: rmdir d2/d4/d12/d39/d3a 39 2024-04-24T22:45:24.993 INFO:tasks.workunit.client.0.smithi063.stdout:3/379: dread da/d18/d3a/d63/f6e [0,4194304] 0 2024-04-24T22:45:24.995 INFO:tasks.workunit.client.0.smithi063.stdout:2/392: mkdir d0/d2/d6c/d7a 0 2024-04-24T22:45:24.995 INFO:tasks.workunit.client.1.smithi192.stdout:9/296: truncate d3/fb 452348 0 2024-04-24T22:45:24.995 INFO:tasks.workunit.client.0.smithi063.stdout:7/314: rename d0/d4/c18 to d0/d4/de/d17/d35/c71 0 2024-04-24T22:45:24.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/380: write da/d48/f5c [9214803,63025] 0 2024-04-24T22:45:25.000 INFO:tasks.workunit.client.1.smithi192.stdout:2/417: rename d4/d9/d10/c15 to d4/d9/d10/c80 0 2024-04-24T22:45:25.002 INFO:tasks.workunit.client.1.smithi192.stdout:9/297: creat d3/d2c/f66 x:0 0 0 2024-04-24T22:45:25.003 INFO:tasks.workunit.client.0.smithi063.stdout:7/315: truncate d0/d4/f19 1217957 0 2024-04-24T22:45:25.004 INFO:tasks.workunit.client.1.smithi192.stdout:8/283: getdents db/dc/d22/d31 0 2024-04-24T22:45:25.006 INFO:tasks.workunit.client.1.smithi192.stdout:8/284: rename db/dc/f3c to db/dc/f53 0 2024-04-24T22:45:25.007 INFO:tasks.workunit.client.0.smithi063.stdout:3/381: dwrite da/f29 [0,4194304] 0 2024-04-24T22:45:25.007 INFO:tasks.workunit.client.1.smithi192.stdout:9/298: symlink d3/dd/de/d17/l67 0 2024-04-24T22:45:25.009 INFO:tasks.workunit.client.0.smithi063.stdout:6/377: dread db/f10 [0,4194304] 0 2024-04-24T22:45:25.012 INFO:tasks.workunit.client.0.smithi063.stdout:3/382: write da/d18/f61 [465250,8357] 0 2024-04-24T22:45:25.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/378: rename db/d1d/d20/d3f/d5f/d7a to db/d1d/d20/d3f/d7f 0 2024-04-24T22:45:25.017 INFO:tasks.workunit.client.1.smithi192.stdout:8/285: rename db/dc/d29/l2a to db/dc/d22/l54 0 2024-04-24T22:45:25.018 INFO:tasks.workunit.client.0.smithi063.stdout:3/383: mknod da/d18/d47/c86 0 2024-04-24T22:45:25.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/344: dwrite d6/d64/d21/f36 [0,4194304] 0 2024-04-24T22:45:25.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/286: symlink db/d43/l55 0 2024-04-24T22:45:25.023 INFO:tasks.workunit.client.0.smithi063.stdout:6/379: unlink db/d1d/d20/l3b 0 2024-04-24T22:45:25.025 INFO:tasks.workunit.client.0.smithi063.stdout:1/422: dread d3/da/dc/d1f/d21/f69 [0,4194304] 0 2024-04-24T22:45:25.025 INFO:tasks.workunit.client.0.smithi063.stdout:1/423: fdatasync d3/da/dc/f8c 0 2024-04-24T22:45:25.026 INFO:tasks.workunit.client.0.smithi063.stdout:3/384: dread da/d18/f68 [0,4194304] 0 2024-04-24T22:45:25.027 INFO:tasks.workunit.client.0.smithi063.stdout:6/380: mknod db/d1d/d2d/d48/c80 0 2024-04-24T22:45:25.030 INFO:tasks.workunit.client.1.smithi192.stdout:8/287: getdents db/dc/d22/d31/d33 0 2024-04-24T22:45:25.030 INFO:tasks.workunit.client.0.smithi063.stdout:1/424: creat d3/da/dc/d39/d5a/f8f x:0 0 0 2024-04-24T22:45:25.032 INFO:tasks.workunit.client.1.smithi192.stdout:8/288: creat db/f56 x:0 0 0 2024-04-24T22:45:25.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/289: symlink db/dc/d22/l57 0 2024-04-24T22:45:25.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/290: dread - db/dc/d29/f47 zero size 2024-04-24T22:45:25.036 INFO:tasks.workunit.client.0.smithi063.stdout:9/369: dwrite d6/d8/db/d6b/f23 [0,4194304] 0 2024-04-24T22:45:25.037 INFO:tasks.workunit.client.0.smithi063.stdout:3/385: dwrite da/d18/d47/d4d/d62/f80 [0,4194304] 0 2024-04-24T22:45:25.038 INFO:tasks.workunit.client.1.smithi192.stdout:8/291: creat db/dc/d22/d31/d33/f58 x:0 0 0 2024-04-24T22:45:25.039 INFO:tasks.workunit.client.1.smithi192.stdout:8/292: dread - db/dc/d22/f23 zero size 2024-04-24T22:45:25.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/425: rename d3/da/dc/d1f/f30 to d3/da/dc/d11/d5e/d6f/f90 0 2024-04-24T22:45:25.040 INFO:tasks.workunit.client.0.smithi063.stdout:6/381: link db/d1d/d20/d3e/c73 db/d1d/d20/d3f/d7f/c81 0 2024-04-24T22:45:25.041 INFO:tasks.workunit.client.0.smithi063.stdout:9/370: symlink d6/d8/db/d6b/d4f/d59/l7c 0 2024-04-24T22:45:25.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/371: write d6/d8/db/d2b/f34 [1778691,12438] 0 2024-04-24T22:45:25.044 INFO:tasks.workunit.client.0.smithi063.stdout:3/386: chown da/d39/l46 458943 1 2024-04-24T22:45:25.044 INFO:tasks.workunit.client.0.smithi063.stdout:9/372: chown d6/d8/db/d1c/l43 10 1 2024-04-24T22:45:25.044 INFO:tasks.workunit.client.0.smithi063.stdout:3/387: fdatasync da/d18/d3a/f69 0 2024-04-24T22:45:25.045 INFO:tasks.workunit.client.0.smithi063.stdout:9/373: fdatasync d6/d2f/d42/f7b 0 2024-04-24T22:45:25.052 INFO:tasks.workunit.client.0.smithi063.stdout:9/374: dwrite d6/d8/db/d1c/f57 [0,4194304] 0 2024-04-24T22:45:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:24 smithi063 ceph-mon[26912]: pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 8.9 MiB/s rd, 126 MiB/s wr, 238 op/s 2024-04-24T22:45:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:24 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:25.054 INFO:tasks.workunit.client.0.smithi063.stdout:9/375: write d6/d8/d1f/d20/f21 [2588584,48903] 0 2024-04-24T22:45:25.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/388: read da/fc [3763995,55830] 0 2024-04-24T22:45:25.058 INFO:tasks.workunit.client.0.smithi063.stdout:2/393: write d0/d2/d54/d7/fc [847784,101269] 0 2024-04-24T22:45:25.060 INFO:tasks.workunit.client.0.smithi063.stdout:2/394: write d0/d2/d2f/f6d [478584,94163] 0 2024-04-24T22:45:25.061 INFO:tasks.workunit.client.0.smithi063.stdout:9/376: mkdir d6/d8/db/d2b/d36/d7d 0 2024-04-24T22:45:25.063 INFO:tasks.workunit.client.0.smithi063.stdout:6/382: rmdir db/d1d/d7c 0 2024-04-24T22:45:25.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/389: dwrite f7 [0,4194304] 0 2024-04-24T22:45:25.065 INFO:tasks.workunit.client.0.smithi063.stdout:1/426: creat d3/da/dc/d1f/d21/f91 x:0 0 0 2024-04-24T22:45:25.068 INFO:tasks.workunit.client.0.smithi063.stdout:3/390: dread da/d39/f41 [0,4194304] 0 2024-04-24T22:45:25.068 INFO:tasks.workunit.client.0.smithi063.stdout:2/395: rmdir d0/d2/d51 39 2024-04-24T22:45:25.069 INFO:tasks.workunit.client.0.smithi063.stdout:7/316: truncate d0/d4/de/d17/d35/f54 831641 0 2024-04-24T22:45:25.070 INFO:tasks.workunit.client.0.smithi063.stdout:9/377: rmdir d6/d8/db/d6b 39 2024-04-24T22:45:25.070 INFO:tasks.workunit.client.0.smithi063.stdout:9/378: dread - d6/f6c zero size 2024-04-24T22:45:25.071 INFO:tasks.workunit.client.0.smithi063.stdout:6/383: stat db/d1d/d2d/d69 0 2024-04-24T22:45:25.072 INFO:tasks.workunit.client.0.smithi063.stdout:6/384: chown db/d1d/d20/d29/d5d 46873013 1 2024-04-24T22:45:25.074 INFO:tasks.workunit.client.0.smithi063.stdout:1/427: write d3/f6c [1887077,6402] 0 2024-04-24T22:45:25.075 INFO:tasks.workunit.client.0.smithi063.stdout:3/391: rmdir da/d39 39 2024-04-24T22:45:25.075 INFO:tasks.workunit.client.0.smithi063.stdout:1/428: chown d3/da/dc/f8c 5409 1 2024-04-24T22:45:25.076 INFO:tasks.workunit.client.0.smithi063.stdout:3/392: fsync da/d18/d47/f6a 0 2024-04-24T22:45:25.078 INFO:tasks.workunit.client.0.smithi063.stdout:1/429: fsync d3/da/dc/d1f/d21/f53 0 2024-04-24T22:45:25.081 INFO:tasks.workunit.client.0.smithi063.stdout:6/385: mkdir db/d1d/d2d/d69/d82 0 2024-04-24T22:45:25.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/345: dwrite d6/d64/d21/d25/d62/d63/f6e [0,4194304] 0 2024-04-24T22:45:25.081 INFO:tasks.workunit.client.0.smithi063.stdout:1/430: chown d3/da/dc/d1f/d2b/d79/f52 137352927 1 2024-04-24T22:45:25.082 INFO:tasks.workunit.client.0.smithi063.stdout:3/393: mknod da/d18/d47/c87 0 2024-04-24T22:45:25.084 INFO:tasks.workunit.client.0.smithi063.stdout:7/317: chown d0/d4/f11 743097 1 2024-04-24T22:45:25.084 INFO:tasks.workunit.client.1.smithi192.stdout:8/293: rmdir db/dc/d22/d31/d33 39 2024-04-24T22:45:25.085 INFO:tasks.workunit.client.0.smithi063.stdout:7/318: fsync d0/d4/de/f60 0 2024-04-24T22:45:25.085 INFO:tasks.workunit.client.0.smithi063.stdout:3/394: truncate da/d18/d47/d4d/f72 320059 0 2024-04-24T22:45:25.086 INFO:tasks.workunit.client.0.smithi063.stdout:9/379: rmdir d6/d2f/d42/d77 0 2024-04-24T22:45:25.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/395: write da/f60 [639411,3456] 0 2024-04-24T22:45:25.087 INFO:tasks.workunit.client.0.smithi063.stdout:7/319: read d0/f3d [1889942,7524] 0 2024-04-24T22:45:25.087 INFO:tasks.workunit.client.0.smithi063.stdout:0/346: dwrite d6/d64/d21/f36 [0,4194304] 0 2024-04-24T22:45:25.089 INFO:tasks.workunit.client.0.smithi063.stdout:0/347: dread - d6/d64/d21/d25/d62/d75/f61 zero size 2024-04-24T22:45:25.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/396: read da/d18/d3a/f69 [257545,21939] 0 2024-04-24T22:45:25.093 INFO:tasks.workunit.client.1.smithi192.stdout:8/294: truncate f3 530631 0 2024-04-24T22:45:25.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/380: dread d6/d8/db/d1c/f57 [0,4194304] 0 2024-04-24T22:45:25.093 INFO:tasks.workunit.client.0.smithi063.stdout:1/431: dread d3/da/dc/f3a [0,4194304] 0 2024-04-24T22:45:25.097 INFO:tasks.workunit.client.0.smithi063.stdout:0/348: dwrite d6/da/f14 [4194304,4194304] 0 2024-04-24T22:45:25.099 INFO:tasks.workunit.client.1.smithi192.stdout:9/299: dwrite d3/d5d/f61 [0,4194304] 0 2024-04-24T22:45:25.103 INFO:tasks.workunit.client.0.smithi063.stdout:1/432: dwrite d3/da/dc/d1f/d21/d42/f73 [0,4194304] 0 2024-04-24T22:45:25.104 INFO:tasks.workunit.client.0.smithi063.stdout:1/433: chown d3/da/dc/d1f/l62 20428174 1 2024-04-24T22:45:25.105 INFO:tasks.workunit.client.0.smithi063.stdout:1/434: stat d3/da/f5c 0 2024-04-24T22:45:25.105 INFO:tasks.workunit.client.0.smithi063.stdout:3/397: chown da/d18/d47/d4d/c6b 30060 1 2024-04-24T22:45:25.107 INFO:tasks.workunit.client.0.smithi063.stdout:6/386: link db/d1d/c3d db/d1c/c83 0 2024-04-24T22:45:25.108 INFO:tasks.workunit.client.0.smithi063.stdout:6/387: fdatasync db/d1d/d20/f40 0 2024-04-24T22:45:25.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/300: truncate d3/f4 3884775 0 2024-04-24T22:45:25.115 INFO:tasks.workunit.client.0.smithi063.stdout:0/349: creat d6/d3a/f7a x:0 0 0 2024-04-24T22:45:25.120 INFO:tasks.workunit.client.1.smithi192.stdout:9/301: creat d3/dd/de/d17/f68 x:0 0 0 2024-04-24T22:45:25.122 INFO:tasks.workunit.client.0.smithi063.stdout:7/320: creat d0/f72 x:0 0 0 2024-04-24T22:45:25.123 INFO:tasks.workunit.client.1.smithi192.stdout:9/302: rename d3/dd/de/d39/c4b to d3/dd/d53/d60/c69 0 2024-04-24T22:45:25.128 INFO:tasks.workunit.client.1.smithi192.stdout:9/303: rename d3/d2c to d3/dd/de/d17/d6a 0 2024-04-24T22:45:25.140 INFO:tasks.workunit.client.1.smithi192.stdout:8/295: read f3 [335679,90615] 0 2024-04-24T22:45:25.143 INFO:tasks.workunit.client.1.smithi192.stdout:8/296: mknod db/dc/d22/c59 0 2024-04-24T22:45:25.144 INFO:tasks.workunit.client.1.smithi192.stdout:8/297: write db/dc/d22/d31/f50 [29062,14347] 0 2024-04-24T22:45:25.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/398: mknod da/c88 0 2024-04-24T22:45:25.145 INFO:tasks.workunit.client.1.smithi192.stdout:8/298: chown f3 0 1 2024-04-24T22:45:25.145 INFO:tasks.workunit.client.0.smithi063.stdout:6/388: mkdir db/d1d/d20/d84 0 2024-04-24T22:45:25.148 INFO:tasks.workunit.client.0.smithi063.stdout:9/381: truncate d6/d2f/f35 1735017 0 2024-04-24T22:45:25.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/435: write d3/da/dc/d1f/d2b/d79/f2e [3380228,19080] 0 2024-04-24T22:45:25.150 INFO:tasks.workunit.client.0.smithi063.stdout:7/321: truncate d0/d4/de/d17/d28/d31/f6a 943761 0 2024-04-24T22:45:25.152 INFO:tasks.workunit.client.1.smithi192.stdout:8/299: rename db/dc/f53 to db/f5a 0 2024-04-24T22:45:25.152 INFO:tasks.workunit.client.1.smithi192.stdout:9/304: dwrite d3/dd/de/f2f [0,4194304] 0 2024-04-24T22:45:25.153 INFO:tasks.workunit.client.1.smithi192.stdout:9/305: stat d3/dd/de/d17/f27 0 2024-04-24T22:45:25.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/396: dwrite d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:25.154 INFO:tasks.workunit.client.0.smithi063.stdout:7/322: read d0/d4/f19 [995287,44184] 0 2024-04-24T22:45:25.154 INFO:tasks.workunit.client.0.smithi063.stdout:0/350: creat d6/d64/d21/d25/f7b x:0 0 0 2024-04-24T22:45:25.155 INFO:tasks.workunit.client.1.smithi192.stdout:8/300: write db/dc/f44 [541453,125619] 0 2024-04-24T22:45:25.156 INFO:tasks.workunit.client.0.smithi063.stdout:3/399: dread da/d18/f19 [0,4194304] 0 2024-04-24T22:45:25.157 INFO:tasks.workunit.client.1.smithi192.stdout:8/301: fdatasync db/dc/d29/f48 0 2024-04-24T22:45:25.159 INFO:tasks.workunit.client.0.smithi063.stdout:6/389: dwrite db/d1d/d20/d3f/d7f/d68/f7e [0,4194304] 0 2024-04-24T22:45:25.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/306: dwrite d3/dd/de/f64 [0,4194304] 0 2024-04-24T22:45:25.164 INFO:tasks.workunit.client.0.smithi063.stdout:2/397: truncate d0/d2/d51/f61 1010128 0 2024-04-24T22:45:25.169 INFO:tasks.workunit.client.0.smithi063.stdout:9/382: unlink d6/d8/db/d2b/c3c 0 2024-04-24T22:45:25.170 INFO:tasks.workunit.client.1.smithi192.stdout:8/302: creat db/dc/f5b x:0 0 0 2024-04-24T22:45:25.171 INFO:tasks.workunit.client.1.smithi192.stdout:6/282: sync 2024-04-24T22:45:25.171 INFO:tasks.workunit.client.1.smithi192.stdout:5/242: sync 2024-04-24T22:45:25.174 INFO:tasks.workunit.client.1.smithi192.stdout:9/307: creat d3/dd/d62/f6b x:0 0 0 2024-04-24T22:45:25.174 INFO:tasks.workunit.client.1.smithi192.stdout:9/308: fdatasync d3/d24/f44 0 2024-04-24T22:45:25.176 INFO:tasks.workunit.client.1.smithi192.stdout:5/243: symlink d0/d13/d19/l4c 0 2024-04-24T22:45:25.177 INFO:tasks.workunit.client.0.smithi063.stdout:1/436: link d3/da/d34/f83 d3/da/d34/d6d/f92 0 2024-04-24T22:45:25.179 INFO:tasks.workunit.client.1.smithi192.stdout:6/283: symlink d0/d8/d29/d5a/l69 0 2024-04-24T22:45:25.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/390: mkdir db/d85 0 2024-04-24T22:45:25.181 INFO:tasks.workunit.client.1.smithi192.stdout:6/284: chown d0/d8/d22/f27 711057345 1 2024-04-24T22:45:25.184 INFO:tasks.workunit.client.0.smithi063.stdout:2/398: symlink d0/dd/l7b 0 2024-04-24T22:45:25.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/400: mknod da/d39/c89 0 2024-04-24T22:45:25.185 INFO:tasks.workunit.client.1.smithi192.stdout:5/244: dwrite d0/d1b/d1c/f33 [0,4194304] 0 2024-04-24T22:45:25.187 INFO:tasks.workunit.client.1.smithi192.stdout:6/285: unlink d0/l21 0 2024-04-24T22:45:25.192 INFO:tasks.workunit.client.0.smithi063.stdout:9/383: dread d6/d8/db/d1c/f57 [0,4194304] 0 2024-04-24T22:45:25.192 INFO:tasks.workunit.client.1.smithi192.stdout:9/309: rename d3/d24/f4a to d3/dd/f6c 0 2024-04-24T22:45:25.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/384: read - d6/d8/d1f/d20/d22/d31/f68 zero size 2024-04-24T22:45:25.196 INFO:tasks.workunit.client.0.smithi063.stdout:9/385: truncate d6/f30 2676702 0 2024-04-24T22:45:25.196 INFO:tasks.workunit.client.1.smithi192.stdout:6/286: mkdir d0/d8/d29/d4c/d4d/d6a 0 2024-04-24T22:45:25.196 INFO:tasks.workunit.client.0.smithi063.stdout:9/386: stat d6/d8/d1f/d20/d2a/d4b/l58 0 2024-04-24T22:45:25.197 INFO:tasks.workunit.client.0.smithi063.stdout:6/391: mknod db/c86 0 2024-04-24T22:45:25.198 INFO:tasks.workunit.client.0.smithi063.stdout:7/323: creat d0/f73 x:0 0 0 2024-04-24T22:45:25.199 INFO:tasks.workunit.client.1.smithi192.stdout:9/310: mkdir d3/dd/d62/d6d 0 2024-04-24T22:45:25.199 INFO:tasks.workunit.client.0.smithi063.stdout:7/324: chown d0/d4/de/f61 1473 1 2024-04-24T22:45:25.199 INFO:tasks.workunit.client.1.smithi192.stdout:9/311: write d3/d24/f4e [902786,88456] 0 2024-04-24T22:45:25.203 INFO:tasks.workunit.client.1.smithi192.stdout:9/312: dread d3/dd/de/d17/f27 [0,4194304] 0 2024-04-24T22:45:25.204 INFO:tasks.workunit.client.1.smithi192.stdout:9/313: fsync d3/f43 0 2024-04-24T22:45:25.208 INFO:tasks.workunit.client.1.smithi192.stdout:8/303: dread f6 [4194304,4194304] 0 2024-04-24T22:45:25.209 INFO:tasks.workunit.client.1.smithi192.stdout:8/304: chown db/f3d 241607 1 2024-04-24T22:45:25.210 INFO:tasks.workunit.client.1.smithi192.stdout:8/305: write db/d43/f4e [2333044,80166] 0 2024-04-24T22:45:25.260 INFO:tasks.workunit.client.0.smithi063.stdout:8/389: sync 2024-04-24T22:45:25.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/375: sync 2024-04-24T22:45:25.267 INFO:tasks.workunit.client.0.smithi063.stdout:8/390: dwrite d1/d11/d49/f7c [0,4194304] 0 2024-04-24T22:45:25.274 INFO:tasks.workunit.client.0.smithi063.stdout:8/391: dwrite d1/f62 [0,4194304] 0 2024-04-24T22:45:25.318 INFO:tasks.workunit.client.0.smithi063.stdout:2/399: rename d0/d2/d51 to d0/dd/d62/d7c 0 2024-04-24T22:45:25.318 INFO:tasks.workunit.client.0.smithi063.stdout:7/325: rename d0/d4 to d0/d4/de/d6f/d2e/d74 22 2024-04-24T22:45:25.320 INFO:tasks.workunit.client.0.smithi063.stdout:3/401: mknod da/d18/d47/d4d/c8a 0 2024-04-24T22:45:25.322 INFO:tasks.workunit.client.1.smithi192.stdout:6/287: fdatasync d0/d8/d22/f32 0 2024-04-24T22:45:25.323 INFO:tasks.workunit.client.1.smithi192.stdout:6/288: write d0/f19 [2047614,48122] 0 2024-04-24T22:45:25.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/314: symlink d3/dd/de/d17/d6a/l6e 0 2024-04-24T22:45:25.331 INFO:tasks.workunit.client.1.smithi192.stdout:8/306: unlink db/dc/d29/f48 0 2024-04-24T22:45:25.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/245: truncate d0/d1b/f37 763319 0 2024-04-24T22:45:25.338 INFO:tasks.workunit.client.0.smithi063.stdout:6/392: creat db/d1d/d20/d3e/f87 x:0 0 0 2024-04-24T22:45:25.339 INFO:tasks.workunit.client.1.smithi192.stdout:9/315: mkdir d3/dd/de/d17/d6a/d6f 0 2024-04-24T22:45:25.340 INFO:tasks.workunit.client.1.smithi192.stdout:9/316: chown d3/dd/f38 1068508910 1 2024-04-24T22:45:25.340 INFO:tasks.workunit.client.0.smithi063.stdout:4/357: sync 2024-04-24T22:45:25.340 INFO:tasks.workunit.client.0.smithi063.stdout:4/358: fdatasync d2/f35 0 2024-04-24T22:45:25.341 INFO:tasks.workunit.client.0.smithi063.stdout:5/376: creat d5/d11/d23/d2d/d62/f8b x:0 0 0 2024-04-24T22:45:25.342 INFO:tasks.workunit.client.0.smithi063.stdout:5/377: stat d5/f6d 0 2024-04-24T22:45:25.343 INFO:tasks.workunit.client.1.smithi192.stdout:8/307: readlink db/dc/l12 0 2024-04-24T22:45:25.345 INFO:tasks.workunit.client.1.smithi192.stdout:6/289: fsync d0/d8/f31 0 2024-04-24T22:45:25.345 INFO:tasks.workunit.client.0.smithi063.stdout:6/393: dwrite db/d1d/d20/f71 [0,4194304] 0 2024-04-24T22:45:25.349 INFO:tasks.workunit.client.0.smithi063.stdout:2/400: creat d0/d2/d54/d1c/d3f/f7d x:0 0 0 2024-04-24T22:45:25.349 INFO:tasks.workunit.client.0.smithi063.stdout:6/394: fsync db/d1d/d20/d29/f2e 0 2024-04-24T22:45:25.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/308: fsync f5 0 2024-04-24T22:45:25.353 INFO:tasks.workunit.client.0.smithi063.stdout:3/402: mknod da/d18/d47/d7b/c8b 0 2024-04-24T22:45:25.353 INFO:tasks.workunit.client.0.smithi063.stdout:4/359: dwrite d2/fb [4194304,4194304] 0 2024-04-24T22:45:25.354 INFO:tasks.workunit.client.0.smithi063.stdout:2/401: read d0/d2/d27/f37 [433425,117221] 0 2024-04-24T22:45:25.357 INFO:tasks.workunit.client.0.smithi063.stdout:5/378: dwrite d5/d24/d2c/f55 [0,4194304] 0 2024-04-24T22:45:25.358 INFO:tasks.workunit.client.0.smithi063.stdout:0/351: getdents d6/d64/d21/d25/d62/d75 0 2024-04-24T22:45:25.360 INFO:tasks.workunit.client.0.smithi063.stdout:1/437: rmdir d3/da/d34/d68/d8a 0 2024-04-24T22:45:25.361 INFO:tasks.workunit.client.0.smithi063.stdout:7/326: dwrite d0/d4/de/d17/f22 [0,4194304] 0 2024-04-24T22:45:25.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/246: fsync d0/fc 0 2024-04-24T22:45:25.363 INFO:tasks.workunit.client.0.smithi063.stdout:9/387: mknod d6/d8/db/d2b/d48/c7e 0 2024-04-24T22:45:25.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/247: truncate d0/d1b/f45 236293 0 2024-04-24T22:45:25.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/248: rename d0/d1b/d1c/c47 to d0/d1b/d1c/d2c/c4d 0 2024-04-24T22:45:25.376 INFO:tasks.workunit.client.1.smithi192.stdout:9/317: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:25.377 INFO:tasks.workunit.client.0.smithi063.stdout:8/392: mknod d1/d11/c81 0 2024-04-24T22:45:25.378 INFO:tasks.workunit.client.0.smithi063.stdout:8/393: dread - d1/d21/f3a zero size 2024-04-24T22:45:25.378 INFO:tasks.workunit.client.1.smithi192.stdout:5/249: chown d0/d13/d19/f1a 2458 1 2024-04-24T22:45:25.378 INFO:tasks.workunit.client.1.smithi192.stdout:9/318: dread - d3/dd/d53/f56 zero size 2024-04-24T22:45:25.378 INFO:tasks.workunit.client.1.smithi192.stdout:8/309: mknod db/dc/d22/d31/c5c 0 2024-04-24T22:45:25.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/263: sync 2024-04-24T22:45:25.381 INFO:tasks.workunit.client.0.smithi063.stdout:3/403: creat da/d18/d47/d4d/f8c x:0 0 0 2024-04-24T22:45:25.381 INFO:tasks.workunit.client.1.smithi192.stdout:8/310: unlink f7 0 2024-04-24T22:45:25.383 INFO:tasks.workunit.client.0.smithi063.stdout:8/394: dwrite d1/d2/d3/f59 [0,4194304] 0 2024-04-24T22:45:25.383 INFO:tasks.workunit.client.0.smithi063.stdout:4/360: creat d2/d8/d36/d3a/f6d x:0 0 0 2024-04-24T22:45:25.384 INFO:tasks.workunit.client.1.smithi192.stdout:9/319: creat d3/dd/de/d39/f70 x:0 0 0 2024-04-24T22:45:25.385 INFO:tasks.workunit.client.1.smithi192.stdout:9/320: dread - d3/f31 zero size 2024-04-24T22:45:25.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/264: fsync d9/f13 0 2024-04-24T22:45:25.387 INFO:tasks.workunit.client.1.smithi192.stdout:6/290: dread d0/d8/f17 [0,4194304] 0 2024-04-24T22:45:25.387 INFO:tasks.workunit.client.0.smithi063.stdout:5/379: creat d5/d24/d36/f8c x:0 0 0 2024-04-24T22:45:25.388 INFO:tasks.workunit.client.0.smithi063.stdout:0/352: mkdir d6/d64/d21/d25/d62/d63/d7c 0 2024-04-24T22:45:25.389 INFO:tasks.workunit.client.1.smithi192.stdout:6/291: truncate d0/d8/d3d/d3e/d40/f68 80660 0 2024-04-24T22:45:25.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/402: rename d0/d2/d54/d48/f4f to d0/d2/d2f/f7e 0 2024-04-24T22:45:25.393 INFO:tasks.workunit.client.1.smithi192.stdout:8/311: mkdir db/d5d 0 2024-04-24T22:45:25.394 INFO:tasks.workunit.client.0.smithi063.stdout:2/403: dread - d0/d2/d2d/d31/d68/f74 zero size 2024-04-24T22:45:25.395 INFO:tasks.workunit.client.0.smithi063.stdout:5/380: dwrite d5/d24/d2c/f89 [0,4194304] 0 2024-04-24T22:45:25.397 INFO:tasks.workunit.client.0.smithi063.stdout:5/381: readlink d5/d11/d18/l2e 0 2024-04-24T22:45:25.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/395: dread d1/d11/f29 [0,4194304] 0 2024-04-24T22:45:25.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:24 smithi192 ceph-mon[33991]: pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 8.9 MiB/s rd, 126 MiB/s wr, 238 op/s 2024-04-24T22:45:25.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:24 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:25.441 INFO:tasks.workunit.client.0.smithi063.stdout:9/388: stat d6/d8/db/d6b/f23 0 2024-04-24T22:45:25.445 INFO:tasks.workunit.client.1.smithi192.stdout:5/250: rmdir d0/d1b/d1c 39 2024-04-24T22:45:25.445 INFO:tasks.workunit.client.1.smithi192.stdout:5/251: readlink d0/d13/d19/l9 0 2024-04-24T22:45:25.449 INFO:tasks.workunit.client.1.smithi192.stdout:9/321: rename d3/dd/de/f64 to d3/f71 0 2024-04-24T22:45:25.450 INFO:tasks.workunit.client.1.smithi192.stdout:9/322: chown d3/dd/de/f51 2378 1 2024-04-24T22:45:25.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/361: readlink d2/d8/l3d 0 2024-04-24T22:45:25.451 INFO:tasks.workunit.client.0.smithi063.stdout:4/362: read d2/d19/f25 [923746,39438] 0 2024-04-24T22:45:25.452 INFO:tasks.workunit.client.0.smithi063.stdout:4/363: dread - d2/f29 zero size 2024-04-24T22:45:25.452 INFO:tasks.workunit.client.0.smithi063.stdout:6/395: link db/d1d/d2d/d48/f51 db/d1d/d20/d3f/d7f/f88 0 2024-04-24T22:45:25.455 INFO:tasks.workunit.client.0.smithi063.stdout:0/353: symlink d6/d64/d21/d25/d62/d75/l7d 0 2024-04-24T22:45:25.456 INFO:tasks.workunit.client.1.smithi192.stdout:5/252: mknod d0/c4e 0 2024-04-24T22:45:25.457 INFO:tasks.workunit.client.0.smithi063.stdout:7/327: fsync d0/d4/f11 0 2024-04-24T22:45:25.459 INFO:tasks.workunit.client.1.smithi192.stdout:5/253: fdatasync d0/d13/d3b/d2f/f2a 0 2024-04-24T22:45:25.462 INFO:tasks.workunit.client.0.smithi063.stdout:2/404: rename d0/d2/d2f/f7e to d0/d2/d27/f7f 0 2024-04-24T22:45:25.463 INFO:tasks.workunit.client.0.smithi063.stdout:7/328: dwrite d0/d4/de/d6f/f53 [0,4194304] 0 2024-04-24T22:45:25.464 INFO:tasks.workunit.client.0.smithi063.stdout:5/382: symlink d5/d24/d36/l8d 0 2024-04-24T22:45:25.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/405: dread d0/f69 [0,4194304] 0 2024-04-24T22:45:25.467 INFO:tasks.workunit.client.0.smithi063.stdout:9/389: symlink d6/d8/db/d6b/d4f/d59/l7f 0 2024-04-24T22:45:25.467 INFO:tasks.workunit.client.1.smithi192.stdout:1/265: dwrite d9/dd/f19 [0,4194304] 0 2024-04-24T22:45:25.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/254: mkdir d0/d13/d3b/d2f/d34/d4f 0 2024-04-24T22:45:25.471 INFO:tasks.workunit.client.1.smithi192.stdout:8/312: dwrite db/f34 [0,4194304] 0 2024-04-24T22:45:25.473 INFO:tasks.workunit.client.0.smithi063.stdout:9/390: write d6/d8/d1f/d20/d22/d31/f68 [399114,63139] 0 2024-04-24T22:45:25.474 INFO:tasks.workunit.client.0.smithi063.stdout:3/404: dwrite da/f14 [0,4194304] 0 2024-04-24T22:45:25.474 INFO:tasks.workunit.client.1.smithi192.stdout:1/266: fdatasync d9/dd/f29 0 2024-04-24T22:45:25.477 INFO:tasks.workunit.client.0.smithi063.stdout:5/383: dwrite d5/d24/f86 [0,4194304] 0 2024-04-24T22:45:25.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/438: truncate d3/da/dc/d1f/d21/f69 2710390 0 2024-04-24T22:45:25.483 INFO:tasks.workunit.client.1.smithi192.stdout:8/313: stat db/dc/c25 0 2024-04-24T22:45:25.487 INFO:tasks.workunit.client.1.smithi192.stdout:1/267: rmdir d9/d15/d2d/d31 39 2024-04-24T22:45:25.491 INFO:tasks.workunit.client.1.smithi192.stdout:7/305: sync 2024-04-24T22:45:25.491 INFO:tasks.workunit.client.1.smithi192.stdout:4/255: sync 2024-04-24T22:45:25.492 INFO:tasks.workunit.client.1.smithi192.stdout:5/255: mkdir d0/d1b/d50 0 2024-04-24T22:45:25.492 INFO:tasks.workunit.client.1.smithi192.stdout:4/256: stat d2/db/l13 0 2024-04-24T22:45:25.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/329: rmdir d0/d4 39 2024-04-24T22:45:25.498 INFO:tasks.workunit.client.0.smithi063.stdout:6/396: write db/d1d/d2d/d48/f51 [1367646,51731] 0 2024-04-24T22:45:25.500 INFO:tasks.workunit.client.1.smithi192.stdout:1/268: symlink d9/d15/d2d/d4a/l58 0 2024-04-24T22:45:25.501 INFO:tasks.workunit.client.0.smithi063.stdout:5/384: mknod d5/d24/c8e 0 2024-04-24T22:45:25.504 INFO:tasks.workunit.client.0.smithi063.stdout:0/354: link d6/dd/f54 d6/d74/f7e 0 2024-04-24T22:45:25.505 INFO:tasks.workunit.client.1.smithi192.stdout:1/269: rmdir d9/d15/d2d/d31/d2f 39 2024-04-24T22:45:25.505 INFO:tasks.workunit.client.1.smithi192.stdout:1/270: readlink l1 0 2024-04-24T22:45:25.507 INFO:tasks.workunit.client.0.smithi063.stdout:1/439: creat d3/da/d34/f93 x:0 0 0 2024-04-24T22:45:25.512 INFO:tasks.workunit.client.1.smithi192.stdout:1/271: symlink d9/d15/d2d/l59 0 2024-04-24T22:45:25.513 INFO:tasks.workunit.client.1.smithi192.stdout:8/314: write f3 [1144414,117016] 0 2024-04-24T22:45:25.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/440: dwrite d3/da/dc/d11/f35 [8388608,4194304] 0 2024-04-24T22:45:25.514 INFO:tasks.workunit.client.1.smithi192.stdout:7/306: write f0 [705056,98315] 0 2024-04-24T22:45:25.515 INFO:tasks.workunit.client.0.smithi063.stdout:0/355: truncate d6/da/f10 640919 0 2024-04-24T22:45:25.517 INFO:tasks.workunit.client.1.smithi192.stdout:7/307: truncate d2/db/d25/f47 186645 0 2024-04-24T22:45:25.521 INFO:tasks.workunit.client.1.smithi192.stdout:4/257: dread d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:25.522 INFO:tasks.workunit.client.1.smithi192.stdout:4/258: write d2/db/f4e [95027,115453] 0 2024-04-24T22:45:25.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/308: rename d2/db/d21/l2f to d2/db/d2a/d57/l5f 0 2024-04-24T22:45:25.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/406: link d0/dd/f75 d0/d2/d54/f80 0 2024-04-24T22:45:25.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/407: chown d0/d2/d54/c50 1306269768 1 2024-04-24T22:45:25.528 INFO:tasks.workunit.client.0.smithi063.stdout:6/397: dread db/d1d/d20/d3e/f5e [0,4194304] 0 2024-04-24T22:45:25.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/309: read d2/db/d21/d42/f4d [243661,104328] 0 2024-04-24T22:45:25.529 INFO:tasks.workunit.client.0.smithi063.stdout:4/364: rename d2/d8/l9 to d2/d8/l6e 0 2024-04-24T22:45:25.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/385: mkdir d5/d11/d23/d2d/d62/d81/d8f 0 2024-04-24T22:45:25.534 INFO:tasks.workunit.client.0.smithi063.stdout:1/441: symlink d3/da/l94 0 2024-04-24T22:45:25.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/386: write d5/d11/d23/d2d/d62/d73/f75 [1459464,32942] 0 2024-04-24T22:45:25.535 INFO:tasks.workunit.client.0.smithi063.stdout:5/387: chown d5/d24/f3d 13652079 1 2024-04-24T22:45:25.537 INFO:tasks.workunit.client.0.smithi063.stdout:9/391: rename d6/d8/l45 to d6/l80 0 2024-04-24T22:45:25.538 INFO:tasks.workunit.client.1.smithi192.stdout:7/310: dread d2/db/d21/d42/d5b/f5d [0,4194304] 0 2024-04-24T22:45:25.539 INFO:tasks.workunit.client.0.smithi063.stdout:6/398: creat db/d1c/f89 x:0 0 0 2024-04-24T22:45:25.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/388: dread d5/d24/d2c/f89 [0,4194304] 0 2024-04-24T22:45:25.541 INFO:tasks.workunit.client.1.smithi192.stdout:4/259: mknod d2/c5e 0 2024-04-24T22:45:25.545 INFO:tasks.workunit.client.0.smithi063.stdout:3/405: getdents da/d18/d47/d7b 0 2024-04-24T22:45:25.547 INFO:tasks.workunit.client.1.smithi192.stdout:1/272: dwrite d9/d15/f1f [0,4194304] 0 2024-04-24T22:45:25.547 INFO:tasks.workunit.client.0.smithi063.stdout:3/406: dread f7 [0,4194304] 0 2024-04-24T22:45:25.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/356: mknod d6/d64/d5d/c7f 0 2024-04-24T22:45:25.549 INFO:tasks.workunit.client.1.smithi192.stdout:7/311: chown d2/c1e 230269232 1 2024-04-24T22:45:25.549 INFO:tasks.workunit.client.1.smithi192.stdout:1/273: readlink d9/d15/d2d/d48/l55 0 2024-04-24T22:45:25.552 INFO:tasks.workunit.client.1.smithi192.stdout:8/315: dwrite db/dc/f13 [0,4194304] 0 2024-04-24T22:45:25.554 INFO:tasks.workunit.client.1.smithi192.stdout:4/260: mkdir d2/d5c/d5f 0 2024-04-24T22:45:25.560 INFO:tasks.workunit.client.0.smithi063.stdout:9/392: mkdir d6/d2f/d42/d81 0 2024-04-24T22:45:25.561 INFO:tasks.workunit.client.0.smithi063.stdout:6/399: symlink db/d1d/d20/d29/l8a 0 2024-04-24T22:45:25.562 INFO:tasks.workunit.client.1.smithi192.stdout:8/316: creat db/dc/d22/f5e x:0 0 0 2024-04-24T22:45:25.569 INFO:tasks.workunit.client.0.smithi063.stdout:2/408: creat d0/d2/d2d/f81 x:0 0 0 2024-04-24T22:45:25.574 INFO:tasks.workunit.client.0.smithi063.stdout:9/393: creat d6/d8/d53/f82 x:0 0 0 2024-04-24T22:45:25.576 INFO:tasks.workunit.client.1.smithi192.stdout:1/274: rename d9/d15/l46 to d9/dd/l5a 0 2024-04-24T22:45:25.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/312: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:25.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/365: truncate d2/d19/f25 4020306 0 2024-04-24T22:45:25.580 INFO:tasks.workunit.client.1.smithi192.stdout:4/261: getdents d2/d9/d25/d2e/d31/d36/d5b/d28/d2f 0 2024-04-24T22:45:25.581 INFO:tasks.workunit.client.0.smithi063.stdout:0/357: dread d6/da/dc/f71 [0,4194304] 0 2024-04-24T22:45:25.582 INFO:tasks.workunit.client.1.smithi192.stdout:1/275: creat d9/d15/d2d/d4a/f5b x:0 0 0 2024-04-24T22:45:25.582 INFO:tasks.workunit.client.0.smithi063.stdout:0/358: dread - d6/d64/d21/d25/f7b zero size 2024-04-24T22:45:25.584 INFO:tasks.workunit.client.0.smithi063.stdout:3/407: dread da/d18/f2a [0,4194304] 0 2024-04-24T22:45:25.585 INFO:tasks.workunit.client.1.smithi192.stdout:8/317: getdents db/dc/d22/d31/d33 0 2024-04-24T22:45:25.589 INFO:tasks.workunit.client.1.smithi192.stdout:7/313: dwrite d2/db/f10 [0,4194304] 0 2024-04-24T22:45:25.591 INFO:tasks.workunit.client.1.smithi192.stdout:8/318: symlink db/dc/d29/l5f 0 2024-04-24T22:45:25.596 INFO:tasks.workunit.client.1.smithi192.stdout:7/314: creat d2/db/f60 x:0 0 0 2024-04-24T22:45:25.598 INFO:tasks.workunit.client.1.smithi192.stdout:8/319: symlink db/l60 0 2024-04-24T22:45:25.599 INFO:tasks.workunit.client.1.smithi192.stdout:4/262: write d2/d9/d25/d2e/d31/d36/d5b/f37 [1048502,94870] 0 2024-04-24T22:45:25.603 INFO:tasks.workunit.client.1.smithi192.stdout:8/320: rename db/dc/d29/c40 to db/dc/d29/c61 0 2024-04-24T22:45:25.605 INFO:tasks.workunit.client.1.smithi192.stdout:7/315: dwrite d2/db/d3a/f48 [0,4194304] 0 2024-04-24T22:45:25.605 INFO:tasks.workunit.client.1.smithi192.stdout:1/276: dread d9/f2c [4194304,4194304] 0 2024-04-24T22:45:25.606 INFO:tasks.workunit.client.1.smithi192.stdout:1/277: fsync d9/dd/ff 0 2024-04-24T22:45:25.610 INFO:tasks.workunit.client.1.smithi192.stdout:4/263: getdents d2/d9/d25/d2e/d31/d36/d5b 0 2024-04-24T22:45:25.610 INFO:tasks.workunit.client.1.smithi192.stdout:3/343: sync 2024-04-24T22:45:25.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/418: sync 2024-04-24T22:45:25.610 INFO:tasks.workunit.client.1.smithi192.stdout:0/391: sync 2024-04-24T22:45:25.611 INFO:tasks.workunit.client.0.smithi063.stdout:5/389: link d5/d11/d23/d2d/l38 d5/d24/d36/l90 0 2024-04-24T22:45:25.611 INFO:tasks.workunit.client.0.smithi063.stdout:1/442: dwrite d3/da/dc/f8b [0,4194304] 0 2024-04-24T22:45:25.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/408: fsync da/d18/d3a/d63/f64 0 2024-04-24T22:45:25.615 INFO:tasks.workunit.client.0.smithi063.stdout:4/366: mkdir d2/d8/d3c/d6f 0 2024-04-24T22:45:25.616 INFO:tasks.workunit.client.1.smithi192.stdout:8/321: dwrite db/dc/f1b [0,4194304] 0 2024-04-24T22:45:25.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/316: dwrite d2/f17 [4194304,4194304] 0 2024-04-24T22:45:25.622 INFO:tasks.workunit.client.0.smithi063.stdout:6/400: dwrite db/ff [0,4194304] 0 2024-04-24T22:45:25.622 INFO:tasks.workunit.client.1.smithi192.stdout:3/344: symlink d2/d4/d7/d4b/d54/l6d 0 2024-04-24T22:45:25.623 INFO:tasks.workunit.client.1.smithi192.stdout:2/419: dread d4/d9/d37/f44 [0,4194304] 0 2024-04-24T22:45:25.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/409: getdents d0/dd/d72/d77 0 2024-04-24T22:45:25.627 INFO:tasks.workunit.client.1.smithi192.stdout:8/322: mkdir db/dc/d22/d31/d33/d62 0 2024-04-24T22:45:25.629 INFO:tasks.workunit.client.1.smithi192.stdout:3/345: write d2/d4/d7/d4f/f52 [4709887,55066] 0 2024-04-24T22:45:25.630 INFO:tasks.workunit.client.0.smithi063.stdout:4/367: write d2/d8/f1d [145806,66318] 0 2024-04-24T22:45:25.632 INFO:tasks.workunit.client.1.smithi192.stdout:7/317: dwrite d2/db/d21/f22 [0,4194304] 0 2024-04-24T22:45:25.634 INFO:tasks.workunit.client.1.smithi192.stdout:2/420: rename d4/c8 to d4/d9/d10/d6c/d6f/c81 0 2024-04-24T22:45:25.637 INFO:tasks.workunit.client.1.smithi192.stdout:3/346: dread d2/d4/f3e [4194304,4194304] 0 2024-04-24T22:45:25.638 INFO:tasks.workunit.client.0.smithi063.stdout:2/410: creat d0/d2/d35/f82 x:0 0 0 2024-04-24T22:45:25.644 INFO:tasks.workunit.client.1.smithi192.stdout:7/318: dwrite d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:25.644 INFO:tasks.workunit.client.1.smithi192.stdout:4/264: link d2/d9/d25/d2e/d31/d36/d5b/d28/c49 d2/d9/c60 0 2024-04-24T22:45:25.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/421: symlink d4/d9/l82 0 2024-04-24T22:45:25.647 INFO:tasks.workunit.client.1.smithi192.stdout:2/422: chown d4/d9/c74 4592 1 2024-04-24T22:45:25.647 INFO:tasks.workunit.client.0.smithi063.stdout:2/411: mknod d0/d2/d27/c83 0 2024-04-24T22:45:25.647 INFO:tasks.workunit.client.1.smithi192.stdout:7/319: creat d2/db/d2a/f61 x:0 0 0 2024-04-24T22:45:25.649 INFO:tasks.workunit.client.0.smithi063.stdout:3/409: creat da/d18/f8d x:0 0 0 2024-04-24T22:45:25.650 INFO:tasks.workunit.client.0.smithi063.stdout:4/368: symlink d2/d8/d3c/d6f/l70 0 2024-04-24T22:45:25.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/443: getdents d3/da/d34/d6d 0 2024-04-24T22:45:25.653 INFO:tasks.workunit.client.1.smithi192.stdout:7/320: dread d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:25.654 INFO:tasks.workunit.client.1.smithi192.stdout:4/265: unlink d2/d9/d25/d2e/d31/d36/d5b/d28/l57 0 2024-04-24T22:45:25.655 INFO:tasks.workunit.client.1.smithi192.stdout:2/423: mknod d4/d9/c83 0 2024-04-24T22:45:25.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/347: read d2/d4/d7/f2c [2598678,106656] 0 2024-04-24T22:45:25.656 INFO:tasks.workunit.client.0.smithi063.stdout:3/410: dread da/d18/f2a [0,4194304] 0 2024-04-24T22:45:25.657 INFO:tasks.workunit.client.0.smithi063.stdout:1/444: mknod d3/da/dc/d39/c95 0 2024-04-24T22:45:25.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/321: rmdir d2/db/d36 39 2024-04-24T22:45:25.662 INFO:tasks.workunit.client.0.smithi063.stdout:3/411: symlink da/d18/d3a/l8e 0 2024-04-24T22:45:25.664 INFO:tasks.workunit.client.0.smithi063.stdout:2/412: link d0/d2/d54/f80 d0/d2/d27/d3a/f84 0 2024-04-24T22:45:25.665 INFO:tasks.workunit.client.0.smithi063.stdout:0/359: write d6/da/d2b/d60/f76 [8313437,33510] 0 2024-04-24T22:45:25.666 INFO:tasks.workunit.client.1.smithi192.stdout:2/424: mkdir d4/d9/d10/d6c/d6f/d84 0 2024-04-24T22:45:25.667 INFO:tasks.workunit.client.1.smithi192.stdout:1/278: dwrite d9/f13 [0,4194304] 0 2024-04-24T22:45:25.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/425: fsync d4/f78 0 2024-04-24T22:45:25.668 INFO:tasks.workunit.client.0.smithi063.stdout:1/445: mkdir d3/da/dc/d1f/d2b/d79/d96 0 2024-04-24T22:45:25.672 INFO:tasks.workunit.client.0.smithi063.stdout:4/369: creat d2/d8/f71 x:0 0 0 2024-04-24T22:45:25.673 INFO:tasks.workunit.client.1.smithi192.stdout:3/348: mknod d2/d4/d7/c6e 0 2024-04-24T22:45:25.678 INFO:tasks.workunit.client.0.smithi063.stdout:2/413: creat d0/d2/d27/f85 x:0 0 0 2024-04-24T22:45:25.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/330: sync 2024-04-24T22:45:25.679 INFO:tasks.workunit.client.0.smithi063.stdout:8/396: sync 2024-04-24T22:45:25.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/266: dread d2/f1f [0,4194304] 0 2024-04-24T22:45:25.679 INFO:tasks.workunit.client.0.smithi063.stdout:0/360: fsync d6/d64/d21/d25/f55 0 2024-04-24T22:45:25.680 INFO:tasks.workunit.client.0.smithi063.stdout:8/397: chown d1/d2/d17/c78 303811504 1 2024-04-24T22:45:25.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/426: write d4/d9/d10/f58 [47263,88492] 0 2024-04-24T22:45:25.681 INFO:tasks.workunit.client.0.smithi063.stdout:7/331: read d0/f4f [2115203,84781] 0 2024-04-24T22:45:25.681 INFO:tasks.workunit.client.0.smithi063.stdout:5/390: truncate d5/d24/d2c/f3e 4171946 0 2024-04-24T22:45:25.684 INFO:tasks.workunit.client.0.smithi063.stdout:4/370: creat d2/d8/d3c/f72 x:0 0 0 2024-04-24T22:45:25.684 INFO:tasks.workunit.client.1.smithi192.stdout:7/322: truncate d2/fa 3646101 0 2024-04-24T22:45:25.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/349: creat d2/d27/f6f x:0 0 0 2024-04-24T22:45:25.687 INFO:tasks.workunit.client.1.smithi192.stdout:2/427: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:25.700 INFO:tasks.workunit.client.0.smithi063.stdout:8/398: creat d1/d2/d3/f82 x:0 0 0 2024-04-24T22:45:25.702 INFO:tasks.workunit.client.0.smithi063.stdout:2/414: truncate d0/d2/d54/d7/f64 2792037 0 2024-04-24T22:45:25.706 INFO:tasks.workunit.client.0.smithi063.stdout:8/399: creat d1/d21/f83 x:0 0 0 2024-04-24T22:45:25.708 INFO:tasks.workunit.client.0.smithi063.stdout:4/371: mkdir d2/d8/d5c/d73 0 2024-04-24T22:45:25.709 INFO:tasks.workunit.client.0.smithi063.stdout:3/412: truncate da/d40/f78 3181239 0 2024-04-24T22:45:25.709 INFO:tasks.workunit.client.0.smithi063.stdout:2/415: mkdir d0/d2/d54/d7/d86 0 2024-04-24T22:45:25.711 INFO:tasks.workunit.client.0.smithi063.stdout:8/400: chown d1/d11/d49/d54/f60 0 1 2024-04-24T22:45:25.712 INFO:tasks.workunit.client.0.smithi063.stdout:4/372: stat d2/d19/c56 0 2024-04-24T22:45:25.713 INFO:tasks.workunit.client.0.smithi063.stdout:2/416: dread d0/d2/d54/d7/f19 [0,4194304] 0 2024-04-24T22:45:25.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/446: write d3/da/dc/d1f/f3e [120596,15420] 0 2024-04-24T22:45:25.714 INFO:tasks.workunit.client.0.smithi063.stdout:5/391: write d5/d11/d18/f20 [1910605,110267] 0 2024-04-24T22:45:25.715 INFO:tasks.workunit.client.0.smithi063.stdout:3/413: getdents da/d18/d3a/d63 0 2024-04-24T22:45:25.718 INFO:tasks.workunit.client.0.smithi063.stdout:0/361: write d6/da/dc/f13 [3167927,129546] 0 2024-04-24T22:45:25.719 INFO:tasks.workunit.client.0.smithi063.stdout:2/417: rmdir d0/d2/d2d/d31/d68 39 2024-04-24T22:45:25.722 INFO:tasks.workunit.client.0.smithi063.stdout:0/362: mkdir d6/d64/d21/d80 0 2024-04-24T22:45:25.723 INFO:tasks.workunit.client.0.smithi063.stdout:5/392: write d5/d11/f15 [5045618,57218] 0 2024-04-24T22:45:25.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/394: sync 2024-04-24T22:45:25.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/401: sync 2024-04-24T22:45:25.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/401: dwrite d1/d11/d32/d38/f67 [0,4194304] 0 2024-04-24T22:45:25.731 INFO:tasks.workunit.client.0.smithi063.stdout:5/393: rmdir d5/d24/d2c 39 2024-04-24T22:45:25.732 INFO:tasks.workunit.client.0.smithi063.stdout:2/418: write d0/d2/d54/d7/f5c [67293,72463] 0 2024-04-24T22:45:25.733 INFO:tasks.workunit.client.0.smithi063.stdout:3/414: dread da/d18/f2a [0,4194304] 0 2024-04-24T22:45:25.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/402: rmdir d1/d11/d49/d54 39 2024-04-24T22:45:25.735 INFO:tasks.workunit.client.0.smithi063.stdout:2/419: chown d0/d2/d54/d7/d86 0 1 2024-04-24T22:45:25.735 INFO:tasks.workunit.client.0.smithi063.stdout:4/373: dread d2/fd [0,4194304] 0 2024-04-24T22:45:25.735 INFO:tasks.workunit.client.0.smithi063.stdout:6/402: mknod db/d1d/d2d/d69/d82/c8b 0 2024-04-24T22:45:25.737 INFO:tasks.workunit.client.0.smithi063.stdout:9/395: rmdir d6/d8/d1f/d20/d2a 39 2024-04-24T22:45:25.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/415: unlink da/l43 0 2024-04-24T22:45:25.740 INFO:tasks.workunit.client.0.smithi063.stdout:2/420: mkdir d0/dd/d72/d77/d87 0 2024-04-24T22:45:25.742 INFO:tasks.workunit.client.1.smithi192.stdout:6/292: sync 2024-04-24T22:45:25.742 INFO:tasks.workunit.client.1.smithi192.stdout:9/323: sync 2024-04-24T22:45:25.742 INFO:tasks.workunit.client.1.smithi192.stdout:5/256: sync 2024-04-24T22:45:25.742 INFO:tasks.workunit.client.0.smithi063.stdout:6/403: rename db/c19 to db/d1d/d20/d3e/c8c 0 2024-04-24T22:45:25.743 INFO:tasks.workunit.client.0.smithi063.stdout:3/416: write da/f15 [146706,31850] 0 2024-04-24T22:45:25.744 INFO:tasks.workunit.client.0.smithi063.stdout:1/447: dread d3/da/dc/d1f/d2b/d79/f2e [0,4194304] 0 2024-04-24T22:45:25.744 INFO:tasks.workunit.client.0.smithi063.stdout:0/363: link d6/d64/d24/f72 d6/d64/d21/d25/d62/d75/f81 0 2024-04-24T22:45:25.748 INFO:tasks.workunit.client.0.smithi063.stdout:5/394: creat d5/d24/f91 x:0 0 0 2024-04-24T22:45:25.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/396: dwrite d6/d8/f71 [0,4194304] 0 2024-04-24T22:45:25.748 INFO:tasks.workunit.client.0.smithi063.stdout:2/421: mknod d0/d2/d6c/c88 0 2024-04-24T22:45:25.749 INFO:tasks.workunit.client.0.smithi063.stdout:6/404: write db/d1d/d2d/d48/f6e [778649,62159] 0 2024-04-24T22:45:25.750 INFO:tasks.workunit.client.0.smithi063.stdout:8/403: creat d1/d2/f84 x:0 0 0 2024-04-24T22:45:25.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/374: creat d2/f74 x:0 0 0 2024-04-24T22:45:25.751 INFO:tasks.workunit.client.1.smithi192.stdout:1/279: mknod d9/d51/c5c 0 2024-04-24T22:45:25.752 INFO:tasks.workunit.client.0.smithi063.stdout:1/448: chown d3/da/dc/d1f/f4d 20 1 2024-04-24T22:45:25.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/267: rename d2/d9/d25/d2e/d31/d36/d5b/f54 to d2/d9/d25/d2e/d31/d36/d5b/d28/f61 0 2024-04-24T22:45:25.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/417: symlink da/d18/d3a/l8f 0 2024-04-24T22:45:25.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/293: fsync d0/d8/d3d/d3e/d40/f68 0 2024-04-24T22:45:25.756 INFO:tasks.workunit.client.1.smithi192.stdout:1/280: write d9/d15/f1f [127998,23642] 0 2024-04-24T22:45:25.756 INFO:tasks.workunit.client.0.smithi063.stdout:9/397: dread - d6/d8/db/d2b/f49 zero size 2024-04-24T22:45:25.760 INFO:tasks.workunit.client.0.smithi063.stdout:9/398: dread - d6/d2f/d42/f6f zero size 2024-04-24T22:45:25.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/449: creat d3/da/dc/d39/d5a/f97 x:0 0 0 2024-04-24T22:45:25.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/268: dwrite d2/db/f52 [0,4194304] 0 2024-04-24T22:45:25.766 INFO:tasks.workunit.client.0.smithi063.stdout:6/405: mkdir db/d1d/d8d 0 2024-04-24T22:45:25.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/364: dwrite d6/d64/d21/d25/d62/d63/f6e [0,4194304] 0 2024-04-24T22:45:25.768 INFO:tasks.workunit.client.0.smithi063.stdout:5/395: dwrite d5/d24/d44/d71/f72 [0,4194304] 0 2024-04-24T22:45:25.770 INFO:tasks.workunit.client.0.smithi063.stdout:8/404: symlink d1/d11/d49/d54/d7b/l85 0 2024-04-24T22:45:25.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/365: truncate d6/d64/f3b 4331933 0 2024-04-24T22:45:25.770 INFO:tasks.workunit.client.0.smithi063.stdout:2/422: write d0/f69 [852198,930] 0 2024-04-24T22:45:25.772 INFO:tasks.workunit.client.0.smithi063.stdout:8/405: stat d1/d22/d33/c80 0 2024-04-24T22:45:25.773 INFO:tasks.workunit.client.0.smithi063.stdout:2/423: write d0/d2/f47 [1290055,64492] 0 2024-04-24T22:45:25.774 INFO:tasks.workunit.client.0.smithi063.stdout:3/418: rename da/d40/l42 to da/d18/d47/d4d/d62/l90 0 2024-04-24T22:45:25.775 INFO:tasks.workunit.client.0.smithi063.stdout:8/406: write d1/d11/d32/d38/f42 [1270166,76964] 0 2024-04-24T22:45:25.775 INFO:tasks.workunit.client.0.smithi063.stdout:5/396: read d5/d24/f3d [800974,5905] 0 2024-04-24T22:45:25.776 INFO:tasks.workunit.client.1.smithi192.stdout:9/324: dread d3/dd/de/d17/d6a/f42 [0,4194304] 0 2024-04-24T22:45:25.777 INFO:tasks.workunit.client.0.smithi063.stdout:2/424: write d0/d2/d63/f71 [3659684,13343] 0 2024-04-24T22:45:25.777 INFO:tasks.workunit.client.0.smithi063.stdout:5/397: write d5/d24/f51 [15512,56319] 0 2024-04-24T22:45:25.779 INFO:tasks.workunit.client.1.smithi192.stdout:7/323: truncate d2/f1a 7196565 0 2024-04-24T22:45:25.779 INFO:tasks.workunit.client.0.smithi063.stdout:2/425: fsync d0/d2/d35/f52 0 2024-04-24T22:45:25.779 INFO:tasks.workunit.client.1.smithi192.stdout:5/257: rmdir d0/d1b/d1c 39 2024-04-24T22:45:25.780 INFO:tasks.workunit.client.0.smithi063.stdout:4/375: link d2/d8/d36/d3a/f6d d2/d8/d3c/f75 0 2024-04-24T22:45:25.782 INFO:tasks.workunit.client.1.smithi192.stdout:6/294: creat d0/d8/d29/d5a/f6b x:0 0 0 2024-04-24T22:45:25.785 INFO:tasks.workunit.client.0.smithi063.stdout:6/406: mknod db/d1d/d20/d3f/d7f/d68/d70/c8e 0 2024-04-24T22:45:25.785 INFO:tasks.workunit.client.0.smithi063.stdout:7/332: sync 2024-04-24T22:45:25.786 INFO:tasks.workunit.client.1.smithi192.stdout:3/350: dread d2/d4/d12/d1a/d1d/d55/f32 [0,4194304] 0 2024-04-24T22:45:25.787 INFO:tasks.workunit.client.1.smithi192.stdout:2/428: link d4/d9/d37/f62 d4/d9/d10/d6c/f85 0 2024-04-24T22:45:25.788 INFO:tasks.workunit.client.1.smithi192.stdout:2/429: chown d4/d29/f51 450 1 2024-04-24T22:45:25.789 INFO:tasks.workunit.client.0.smithi063.stdout:8/407: mkdir d1/d2/d3/d86 0 2024-04-24T22:45:25.793 INFO:tasks.workunit.client.1.smithi192.stdout:5/258: dread - d0/d1b/d1c/d2c/f3d zero size 2024-04-24T22:45:25.794 INFO:tasks.workunit.client.0.smithi063.stdout:3/419: rename da/d18/f68 to da/d18/d47/d7b/f91 0 2024-04-24T22:45:25.795 INFO:tasks.workunit.client.0.smithi063.stdout:3/420: chown da/l79 110596 1 2024-04-24T22:45:25.795 INFO:tasks.workunit.client.0.smithi063.stdout:3/421: fdatasync da/d18/d3a/d63/f64 0 2024-04-24T22:45:25.796 INFO:tasks.workunit.client.0.smithi063.stdout:3/422: stat da/d48/f5c 0 2024-04-24T22:45:25.796 INFO:tasks.workunit.client.0.smithi063.stdout:3/423: chown da/d18/d47/l6d 10091 1 2024-04-24T22:45:25.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/426: fsync d0/d2/d2d/d31/d68/f74 0 2024-04-24T22:45:25.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/427: chown d0/d2/d2f 8 1 2024-04-24T22:45:25.803 INFO:tasks.workunit.client.1.smithi192.stdout:6/295: dwrite d0/d8/f34 [0,4194304] 0 2024-04-24T22:45:25.803 INFO:tasks.workunit.client.0.smithi063.stdout:4/376: truncate d2/f29 724489 0 2024-04-24T22:45:25.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/407: rmdir db/d1d/d20/d3f/d7f/d68 39 2024-04-24T22:45:25.807 INFO:tasks.workunit.client.0.smithi063.stdout:2/428: dwrite d0/d2/d27/f29 [0,4194304] 0 2024-04-24T22:45:25.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/325: symlink d3/dd/de/d17/l72 0 2024-04-24T22:45:25.808 INFO:tasks.workunit.client.0.smithi063.stdout:2/429: stat d0/l1 0 2024-04-24T22:45:25.810 INFO:tasks.workunit.client.1.smithi192.stdout:7/324: symlink d2/db/d36/l62 0 2024-04-24T22:45:25.811 INFO:tasks.workunit.client.1.smithi192.stdout:7/325: truncate d2/db/d2a/f61 488231 0 2024-04-24T22:45:25.819 INFO:tasks.workunit.client.0.smithi063.stdout:3/424: unlink da/d18/f19 0 2024-04-24T22:45:25.822 INFO:tasks.workunit.client.0.smithi063.stdout:9/399: dwrite d6/d2f/f35 [0,4194304] 0 2024-04-24T22:45:25.824 INFO:tasks.workunit.client.0.smithi063.stdout:9/400: stat d6/d8/d1f/c3a 0 2024-04-24T22:45:25.824 INFO:tasks.workunit.client.0.smithi063.stdout:9/401: readlink d6/l18 0 2024-04-24T22:45:25.829 INFO:tasks.workunit.client.0.smithi063.stdout:5/398: dwrite d5/d11/f3c [0,4194304] 0 2024-04-24T22:45:25.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/425: mknod da/d40/c92 0 2024-04-24T22:45:25.835 INFO:tasks.workunit.client.1.smithi192.stdout:4/269: truncate d2/d9/d25/d2e/d31/d36/d5b/f24 641620 0 2024-04-24T22:45:25.835 INFO:tasks.workunit.client.0.smithi063.stdout:3/426: dread - da/d18/d47/d4d/f8c zero size 2024-04-24T22:45:25.837 INFO:tasks.workunit.client.0.smithi063.stdout:6/408: dwrite db/d1d/d2d/d69/f6c [0,4194304] 0 2024-04-24T22:45:25.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/366: truncate d6/d64/f3b 3374110 0 2024-04-24T22:45:25.839 INFO:tasks.workunit.client.0.smithi063.stdout:6/409: fdatasync db/d1d/f49 0 2024-04-24T22:45:25.839 INFO:tasks.workunit.client.1.smithi192.stdout:6/296: fsync d0/d8/f31 0 2024-04-24T22:45:25.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/430: truncate d4/f41 4554242 0 2024-04-24T22:45:25.843 INFO:tasks.workunit.client.0.smithi063.stdout:0/367: truncate d6/d3a/f7a 487589 0 2024-04-24T22:45:25.843 INFO:tasks.workunit.client.1.smithi192.stdout:9/326: mknod d3/dd/de/c73 0 2024-04-24T22:45:25.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/410: chown db/d1d/d2d/f36 5 1 2024-04-24T22:45:25.846 INFO:tasks.workunit.client.0.smithi063.stdout:1/450: getdents d3/da/dc/d11/d5e/d6f 0 2024-04-24T22:45:25.847 INFO:tasks.workunit.client.0.smithi063.stdout:9/402: mknod d6/d8/db/d2b/d48/c83 0 2024-04-24T22:45:25.847 INFO:tasks.workunit.client.0.smithi063.stdout:5/399: mkdir d5/d11/d76/d92 0 2024-04-24T22:45:25.849 INFO:tasks.workunit.client.1.smithi192.stdout:1/281: link d9/dd/c45 d9/d15/c5d 0 2024-04-24T22:45:25.850 INFO:tasks.workunit.client.1.smithi192.stdout:1/282: chown c8 241 1 2024-04-24T22:45:25.851 INFO:tasks.workunit.client.0.smithi063.stdout:5/400: write d5/f6d [885585,8626] 0 2024-04-24T22:45:25.852 INFO:tasks.workunit.client.0.smithi063.stdout:1/451: write d3/da/d34/f93 [634177,59632] 0 2024-04-24T22:45:25.852 INFO:tasks.workunit.client.0.smithi063.stdout:8/408: dwrite d1/d21/f5d [0,4194304] 0 2024-04-24T22:45:25.852 INFO:tasks.workunit.client.0.smithi063.stdout:9/403: write d6/d8/db/d6b/f23 [3328895,44513] 0 2024-04-24T22:45:25.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/270: creat d2/d9/d25/d2e/d31/d36/f62 x:0 0 0 2024-04-24T22:45:25.852 INFO:tasks.workunit.client.0.smithi063.stdout:7/333: dread d0/d4/de/f21 [0,4194304] 0 2024-04-24T22:45:25.854 INFO:tasks.workunit.client.0.smithi063.stdout:9/404: readlink d6/d8/d1f/d20/d22/l2c 0 2024-04-24T22:45:25.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/452: read - d3/da/f5c zero size 2024-04-24T22:45:25.856 INFO:tasks.workunit.client.1.smithi192.stdout:6/297: symlink d0/d8/d22/l6c 0 2024-04-24T22:45:25.861 INFO:tasks.workunit.client.0.smithi063.stdout:9/405: write d6/d2f/d42/f6f [372472,33909] 0 2024-04-24T22:45:25.862 INFO:tasks.workunit.client.0.smithi063.stdout:1/453: fdatasync d3/da/dc/d39/d5a/f8f 0 2024-04-24T22:45:25.863 INFO:tasks.workunit.client.0.smithi063.stdout:6/411: unlink db/d1c/c5a 0 2024-04-24T22:45:25.864 INFO:tasks.workunit.client.1.smithi192.stdout:2/431: creat d4/d29/f86 x:0 0 0 2024-04-24T22:45:25.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/427: dwrite da/f28 [0,4194304] 0 2024-04-24T22:45:25.867 INFO:tasks.workunit.client.0.smithi063.stdout:9/406: readlink d6/d8/db/d6b/d4f/d59/l7f 0 2024-04-24T22:45:25.871 INFO:tasks.workunit.client.0.smithi063.stdout:2/430: rename d0/d2/d54 to d0/d2/d2d/d31/d89 0 2024-04-24T22:45:25.872 INFO:tasks.workunit.client.1.smithi192.stdout:7/326: mkdir d2/db/d21/d42/d5b/d63 0 2024-04-24T22:45:25.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/334: dwrite d0/f73 [0,4194304] 0 2024-04-24T22:45:25.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/431: chown d0/d2/d6c 1003 1 2024-04-24T22:45:25.874 INFO:tasks.workunit.client.1.smithi192.stdout:9/327: dwrite d3/fb [4194304,4194304] 0 2024-04-24T22:45:25.878 INFO:tasks.workunit.client.1.smithi192.stdout:4/271: dread d2/f3 [0,4194304] 0 2024-04-24T22:45:25.879 INFO:tasks.workunit.client.0.smithi063.stdout:5/401: dread d5/f9 [0,4194304] 0 2024-04-24T22:45:25.884 INFO:tasks.workunit.client.0.smithi063.stdout:4/377: write d2/d8/f1a [460100,116573] 0 2024-04-24T22:45:25.885 INFO:tasks.workunit.client.0.smithi063.stdout:0/368: dread f5 [0,4194304] 0 2024-04-24T22:45:25.886 INFO:tasks.workunit.client.1.smithi192.stdout:6/298: write d0/d8/f2a [4604621,54377] 0 2024-04-24T22:45:25.888 INFO:tasks.workunit.client.1.smithi192.stdout:3/351: link d2/d4/d12/d39/d3a/l45 d2/d4/d7/d41/l70 0 2024-04-24T22:45:25.890 INFO:tasks.workunit.client.0.smithi063.stdout:5/402: dwrite d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:25.892 INFO:tasks.workunit.client.0.smithi063.stdout:5/403: write d5/d11/f15 [1284026,13449] 0 2024-04-24T22:45:25.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/327: rename d2/db/d2a/f51 to d2/db/d36/f64 0 2024-04-24T22:45:25.898 INFO:tasks.workunit.client.1.smithi192.stdout:1/283: mkdir d9/d15/d5e 0 2024-04-24T22:45:25.899 INFO:tasks.workunit.client.1.smithi192.stdout:1/284: write d9/f13 [3901991,117402] 0 2024-04-24T22:45:25.900 INFO:tasks.workunit.client.0.smithi063.stdout:2/432: rmdir d0/dd 39 2024-04-24T22:45:25.903 INFO:tasks.workunit.client.1.smithi192.stdout:9/328: creat d3/dd/d53/d60/f74 x:0 0 0 2024-04-24T22:45:25.905 INFO:tasks.workunit.client.1.smithi192.stdout:9/329: stat d3/dd/de/c3d 0 2024-04-24T22:45:25.906 INFO:tasks.workunit.client.0.smithi063.stdout:3/428: symlink da/d18/d47/d4d/d62/l93 0 2024-04-24T22:45:25.907 INFO:tasks.workunit.client.1.smithi192.stdout:4/272: rmdir d2/db 39 2024-04-24T22:45:25.908 INFO:tasks.workunit.client.1.smithi192.stdout:1/285: dwrite d9/f42 [0,4194304] 0 2024-04-24T22:45:25.908 INFO:tasks.workunit.client.1.smithi192.stdout:4/273: readlink d2/d9/l4b 0 2024-04-24T22:45:25.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/274: dread - d2/f50 zero size 2024-04-24T22:45:25.910 INFO:tasks.workunit.client.0.smithi063.stdout:7/335: creat d0/d4/f75 x:0 0 0 2024-04-24T22:45:25.910 INFO:tasks.workunit.client.1.smithi192.stdout:4/275: write d2/f50 [684679,104684] 0 2024-04-24T22:45:25.915 INFO:tasks.workunit.client.1.smithi192.stdout:6/299: creat d0/d8/d29/d4c/f6d x:0 0 0 2024-04-24T22:45:25.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/336: dwrite d0/d4/de/f25 [0,4194304] 0 2024-04-24T22:45:25.919 INFO:tasks.workunit.client.1.smithi192.stdout:6/300: dread d0/d8/f34 [0,4194304] 0 2024-04-24T22:45:25.920 INFO:tasks.workunit.client.1.smithi192.stdout:6/301: readlink d0/d8/d3d/l67 0 2024-04-24T22:45:25.922 INFO:tasks.workunit.client.1.smithi192.stdout:6/302: write d0/d8/f2b [5409650,120364] 0 2024-04-24T22:45:25.922 INFO:tasks.workunit.client.1.smithi192.stdout:3/352: unlink d2/d4/d12/d1a/d1d/f25 0 2024-04-24T22:45:25.924 INFO:tasks.workunit.client.1.smithi192.stdout:6/303: read d0/f19 [570489,14764] 0 2024-04-24T22:45:25.924 INFO:tasks.workunit.client.0.smithi063.stdout:6/412: creat db/d1d/d20/d84/f8f x:0 0 0 2024-04-24T22:45:25.925 INFO:tasks.workunit.client.0.smithi063.stdout:4/378: dwrite d2/d8/f3e [0,4194304] 0 2024-04-24T22:45:25.926 INFO:tasks.workunit.client.1.smithi192.stdout:6/304: dread - d0/d8/d3d/f5f zero size 2024-04-24T22:45:25.927 INFO:tasks.workunit.client.1.smithi192.stdout:3/353: dread d2/f33 [0,4194304] 0 2024-04-24T22:45:25.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/454: dread d3/fd [0,4194304] 0 2024-04-24T22:45:25.935 INFO:tasks.workunit.client.0.smithi063.stdout:5/404: creat d5/d1d/f93 x:0 0 0 2024-04-24T22:45:25.935 INFO:tasks.workunit.client.0.smithi063.stdout:2/433: mknod d0/d2/d6c/c8a 0 2024-04-24T22:45:25.943 INFO:tasks.workunit.client.0.smithi063.stdout:2/434: dwrite d0/d2/d2d/d31/f5b [0,4194304] 0 2024-04-24T22:45:25.944 INFO:tasks.workunit.client.0.smithi063.stdout:2/435: dread - d0/d2/d2d/f81 zero size 2024-04-24T22:45:25.944 INFO:tasks.workunit.client.0.smithi063.stdout:8/409: link d1/d21/c2a d1/d11/d32/d6a/c87 0 2024-04-24T22:45:25.946 INFO:tasks.workunit.client.0.smithi063.stdout:8/410: dread - d1/d11/d32/f65 zero size 2024-04-24T22:45:25.946 INFO:tasks.workunit.client.0.smithi063.stdout:2/436: fdatasync d0/d2/d2d/d31/d89/d48/f5d 0 2024-04-24T22:45:25.947 INFO:tasks.workunit.client.1.smithi192.stdout:4/276: read d2/db/d10/f4f [361934,91899] 0 2024-04-24T22:45:25.950 INFO:tasks.workunit.client.0.smithi063.stdout:7/337: mkdir d0/d4/de/d6f/d2e/d76 0 2024-04-24T22:45:25.950 INFO:tasks.workunit.client.0.smithi063.stdout:7/338: fsync d0/f48 0 2024-04-24T22:45:25.952 INFO:tasks.workunit.client.0.smithi063.stdout:6/413: creat db/d1c/f90 x:0 0 0 2024-04-24T22:45:25.952 INFO:tasks.workunit.client.1.smithi192.stdout:0/392: sync 2024-04-24T22:45:25.953 INFO:tasks.workunit.client.0.smithi063.stdout:9/407: dread d6/d8/d1f/f52 [0,4194304] 0 2024-04-24T22:45:25.955 INFO:tasks.workunit.client.1.smithi192.stdout:0/393: chown d3/d27/f71 1615633425 1 2024-04-24T22:45:25.956 INFO:tasks.workunit.client.0.smithi063.stdout:7/339: dread d0/d4/de/f21 [0,4194304] 0 2024-04-24T22:45:25.956 INFO:tasks.workunit.client.1.smithi192.stdout:0/394: chown d3/d6d 314 1 2024-04-24T22:45:25.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 -- 172.21.15.63:0/981470374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 msgr2=0x7ff94c0fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:25.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 --2- 172.21.15.63:0/981470374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c0fc6a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff948004940 tx=0x7ff948009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:25.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 -- 172.21.15.63:0/981470374 shutdown_connections 2024-04-24T22:45:25.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 --2- 172.21.15.63:0/981470374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff94c0fcbe0 0x7ff94c0fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 --2- 172.21.15.63:0/981470374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c0fc6a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.953+0000 7ff9546cf700 1 -- 172.21.15.63:0/981470374 >> 172.21.15.63:0/981470374 conn(0x7ff94c0f6b80 msgr2=0x7ff94c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.954+0000 7ff9546cf700 1 -- 172.21.15.63:0/981470374 shutdown_connections 2024-04-24T22:45:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.954+0000 7ff9546cf700 1 -- 172.21.15.63:0/981470374 wait complete. 2024-04-24T22:45:25.957 INFO:tasks.workunit.client.0.smithi063.stdout:6/414: truncate db/d1d/d20/d3e/d47/f62 995939 0 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.954+0000 7ff9546cf700 1 Processor -- start 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.954+0000 7ff9546cf700 1 -- start start 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.955+0000 7ff9546cf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.955+0000 7ff9546cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff94c0fcbe0 0x7ff94c10bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.955+0000 7ff9546cf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c10c2e0 con 0x7ff94c0fc290 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.955+0000 7ff9546cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c10cb50 con 0x7ff94c0fcbe0 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47264/0 (socket says 172.21.15.63:47264) 2024-04-24T22:45:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 -- 172.21.15.63:0/1417208477 learned_addr learned my addr 172.21.15.63:0/1417208477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 -- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff94c0fcbe0 msgr2=0x7ff94c10bc90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff94c0fcbe0 0x7ff94c10bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.956+0000 7ff95246b700 1 -- 172.21.15.63:0/1417208477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948005040 con 0x7ff94c0fc290 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff95246b700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b970 tx=0x7ff948009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94800e040 con 0x7ff94c0fc290 2024-04-24T22:45:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff948004290 con 0x7ff94c0fc290 2024-04-24T22:45:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94801c630 con 0x7ff94c0fc290 2024-04-24T22:45:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c10cdd0 con 0x7ff94c0fc290 2024-04-24T22:45:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.957+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c10d390 con 0x7ff94c0fc290 2024-04-24T22:45:25.961 INFO:tasks.workunit.client.0.smithi063.stdout:6/415: write db/d1d/f4c [2449196,35429] 0 2024-04-24T22:45:25.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/408: write d6/d8/d53/f82 [56802,11915] 0 2024-04-24T22:45:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.959+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c1019e0 con 0x7ff94c0fc290 2024-04-24T22:45:25.962 INFO:tasks.workunit.client.0.smithi063.stdout:4/379: creat d2/d8/d3c/d6f/f76 x:0 0 0 2024-04-24T22:45:25.962 INFO:tasks.workunit.client.0.smithi063.stdout:1/455: unlink d3/da/dc/l76 0 2024-04-24T22:45:25.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.960+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7ff94800bbf0 con 0x7ff94c0fc290 2024-04-24T22:45:25.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.961+0000 7ff942ffd700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 0x7ff93806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:25.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.961+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff94808b7b0 con 0x7ff94c0fc290 2024-04-24T22:45:25.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.961+0000 7ff951c6a700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 0x7ff93806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:25.965 INFO:tasks.workunit.client.0.smithi063.stdout:1/456: truncate d3/da/dc/d39/f66 1257490 0 2024-04-24T22:45:25.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.963+0000 7ff951c6a700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 0x7ff93806dd10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009b80 tx=0x7ff93c009d60 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:25.967 INFO:tasks.workunit.client.1.smithi192.stdout:2/432: rmdir d4/d9/d10/d6c/d6f/d84 0 2024-04-24T22:45:25.967 INFO:tasks.workunit.client.0.smithi063.stdout:1/457: truncate d3/da/dc/d11/d5e/d6f/f90 8399644 0 2024-04-24T22:45:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:25.968+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff9480552f0 con 0x7ff94c0fc290 2024-04-24T22:45:25.972 INFO:tasks.workunit.client.1.smithi192.stdout:2/433: dread d4/d9/fb [0,4194304] 0 2024-04-24T22:45:25.973 INFO:tasks.workunit.client.1.smithi192.stdout:7/328: symlink d2/db/d49/l65 0 2024-04-24T22:45:25.974 INFO:tasks.workunit.client.0.smithi063.stdout:1/458: dwrite d3/da/dc/d11/f46 [0,4194304] 0 2024-04-24T22:45:25.977 INFO:tasks.workunit.client.1.smithi192.stdout:1/286: unlink d9/dd/l36 0 2024-04-24T22:45:25.979 INFO:tasks.workunit.client.1.smithi192.stdout:1/287: read d9/f3c [2105514,127187] 0 2024-04-24T22:45:25.983 INFO:tasks.workunit.client.0.smithi063.stdout:6/416: unlink db/d1d/f49 0 2024-04-24T22:45:25.984 INFO:tasks.workunit.client.0.smithi063.stdout:6/417: write db/d1d/d20/f2b [4501411,43060] 0 2024-04-24T22:45:25.986 INFO:tasks.workunit.client.0.smithi063.stdout:9/409: fsync d6/d2f/f72 0 2024-04-24T22:45:25.987 INFO:tasks.workunit.client.1.smithi192.stdout:8/323: sync 2024-04-24T22:45:25.987 INFO:tasks.workunit.client.1.smithi192.stdout:5/259: sync 2024-04-24T22:45:25.987 INFO:tasks.workunit.client.0.smithi063.stdout:9/410: chown d6/d2f/d42/f7b 74360 1 2024-04-24T22:45:25.988 INFO:tasks.workunit.client.1.smithi192.stdout:0/395: mkdir d3/d27/d28/d2e/d4a/d5c/d8a 0 2024-04-24T22:45:25.988 INFO:tasks.workunit.client.1.smithi192.stdout:5/260: chown d0 3624 1 2024-04-24T22:45:25.989 INFO:tasks.workunit.client.0.smithi063.stdout:5/405: symlink d5/d11/l94 0 2024-04-24T22:45:25.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/406: stat d5/d24/d36/f5a 0 2024-04-24T22:45:25.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/261: write d0/d13/d3b/d2f/f46 [839841,26802] 0 2024-04-24T22:45:25.992 INFO:tasks.workunit.client.1.smithi192.stdout:5/262: dread - d0/d13/f40 zero size 2024-04-24T22:45:25.993 INFO:tasks.workunit.client.1.smithi192.stdout:8/324: dread db/dc/d29/f4f [0,4194304] 0 2024-04-24T22:45:25.993 INFO:tasks.workunit.client.1.smithi192.stdout:6/305: mknod d0/d8/d3d/d3e/c6e 0 2024-04-24T22:45:25.995 INFO:tasks.workunit.client.0.smithi063.stdout:8/411: symlink d1/d11/d32/d6a/l88 0 2024-04-24T22:45:25.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/354: mkdir d2/d4/d12/d1a/d1d/d71 0 2024-04-24T22:45:25.998 INFO:tasks.workunit.client.1.smithi192.stdout:6/306: dread d0/d8/d29/f53 [0,4194304] 0 2024-04-24T22:45:25.999 INFO:tasks.workunit.client.0.smithi063.stdout:1/459: rename d3/da/dc/d11/f61 to d3/da/dc/d39/f98 0 2024-04-24T22:45:26.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/460: write d3/da/dc/d1f/d21/d42/f58 [986451,47611] 0 2024-04-24T22:45:26.001 INFO:tasks.workunit.client.0.smithi063.stdout:2/437: truncate d0/d2/d2d/d31/d89/d7/f64 3731121 0 2024-04-24T22:45:26.002 INFO:tasks.workunit.client.0.smithi063.stdout:1/461: write d3/da/dc/d11/f35 [9435889,22787] 0 2024-04-24T22:45:26.003 INFO:tasks.workunit.client.0.smithi063.stdout:2/438: write d0/d2/d2f/f6d [1382101,69809] 0 2024-04-24T22:45:26.004 INFO:tasks.workunit.client.0.smithi063.stdout:2/439: dread - d0/d2/d35/f82 zero size 2024-04-24T22:45:26.008 INFO:tasks.workunit.client.0.smithi063.stdout:2/440: dread d0/d2/d2d/d31/d89/d7/f5c [0,4194304] 0 2024-04-24T22:45:26.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/329: dread d2/db/d2a/f2b [0,4194304] 0 2024-04-24T22:45:26.009 INFO:tasks.workunit.client.0.smithi063.stdout:6/418: fdatasync db/f37 0 2024-04-24T22:45:26.010 INFO:tasks.workunit.client.0.smithi063.stdout:6/419: fsync db/d1d/f4e 0 2024-04-24T22:45:26.012 INFO:tasks.workunit.client.1.smithi192.stdout:0/396: creat d3/d6d/d79/f8b x:0 0 0 2024-04-24T22:45:26.013 INFO:tasks.workunit.client.0.smithi063.stdout:3/429: dwrite da/d40/f78 [0,4194304] 0 2024-04-24T22:45:26.017 INFO:tasks.workunit.client.0.smithi063.stdout:5/407: write f1 [1244513,130042] 0 2024-04-24T22:45:26.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/325: write db/dc/f1c [499148,23789] 0 2024-04-24T22:45:26.019 INFO:tasks.workunit.client.0.smithi063.stdout:8/412: read d1/d11/f29 [4435070,78257] 0 2024-04-24T22:45:26.021 INFO:tasks.workunit.client.0.smithi063.stdout:3/430: dwrite da/f3d [0,4194304] 0 2024-04-24T22:45:26.022 INFO:tasks.workunit.client.1.smithi192.stdout:2/434: dread d4/f41 [0,4194304] 0 2024-04-24T22:45:26.022 INFO:tasks.workunit.client.1.smithi192.stdout:0/397: dwrite d3/d27/d3c/d5d/f86 [0,4194304] 0 2024-04-24T22:45:26.022 INFO:tasks.workunit.client.1.smithi192.stdout:5/263: creat d0/d1b/d1c/d2c/f51 x:0 0 0 2024-04-24T22:45:26.022 INFO:tasks.workunit.client.1.smithi192.stdout:8/326: fsync db/dc/d29/f2b 0 2024-04-24T22:45:26.023 INFO:tasks.workunit.client.0.smithi063.stdout:8/413: read d1/d2/d3/f59 [2314047,84532] 0 2024-04-24T22:45:26.023 INFO:tasks.workunit.client.0.smithi063.stdout:8/414: write d1/d2/d3/f14 [661940,16331] 0 2024-04-24T22:45:26.027 INFO:tasks.workunit.client.1.smithi192.stdout:0/398: truncate d3/d27/d28/d2e/d4a/d5c/f3d 4386559 0 2024-04-24T22:45:26.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/355: creat d2/d27/d28/d38/f72 x:0 0 0 2024-04-24T22:45:26.039 INFO:tasks.workunit.client.1.smithi192.stdout:9/330: getdents d3/d59 0 2024-04-24T22:45:26.041 INFO:tasks.workunit.client.0.smithi063.stdout:7/340: creat d0/d4/de/d6f/f77 x:0 0 0 2024-04-24T22:45:26.042 INFO:tasks.workunit.client.0.smithi063.stdout:1/462: mkdir d3/da/d34/d99 0 2024-04-24T22:45:26.043 INFO:tasks.workunit.client.1.smithi192.stdout:4/277: getdents d2/d9/d25/d2e/d31/d36/d5b/d28/d2f 0 2024-04-24T22:45:26.044 INFO:tasks.workunit.client.1.smithi192.stdout:4/278: stat d2/d9/d25/d2e/d31/d36/f62 0 2024-04-24T22:45:26.045 INFO:tasks.workunit.client.1.smithi192.stdout:7/330: symlink d2/db/d2a/l66 0 2024-04-24T22:45:26.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/331: read f1 [664179,80602] 0 2024-04-24T22:45:26.050 INFO:tasks.workunit.client.1.smithi192.stdout:7/332: write d2/db/f45 [679649,79094] 0 2024-04-24T22:45:26.052 INFO:tasks.workunit.client.0.smithi063.stdout:5/408: unlink d5/d24/d49/d3b/l57 0 2024-04-24T22:45:26.056 INFO:tasks.workunit.client.1.smithi192.stdout:0/399: unlink d3/d27/f68 0 2024-04-24T22:45:26.057 INFO:tasks.workunit.client.0.smithi063.stdout:3/431: mkdir da/d39/d94 0 2024-04-24T22:45:26.059 INFO:tasks.workunit.client.1.smithi192.stdout:2/435: creat d4/d29/f87 x:0 0 0 2024-04-24T22:45:26.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/415: rename d1/d11/d32/l36 to d1/d70/l89 0 2024-04-24T22:45:26.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/416: stat d1/d2/c74 0 2024-04-24T22:45:26.064 INFO:tasks.workunit.client.1.smithi192.stdout:0/400: dwrite d3/d27/d28/d2e/d4a/d5c/f61 [0,4194304] 0 2024-04-24T22:45:26.066 INFO:tasks.workunit.client.1.smithi192.stdout:3/356: creat d2/d27/f73 x:0 0 0 2024-04-24T22:45:26.067 INFO:tasks.workunit.client.0.smithi063.stdout:7/341: symlink d0/d4/de/d6f/d2e/l78 0 2024-04-24T22:45:26.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/342: stat d0/d4/de/d6f/d2e/c51 0 2024-04-24T22:45:26.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/463: truncate d3/da/dc/f3a 907228 0 2024-04-24T22:45:26.071 INFO:tasks.workunit.client.0.smithi063.stdout:7/343: dread d0/d4/de/d17/d28/f64 [0,4194304] 0 2024-04-24T22:45:26.074 INFO:tasks.workunit.client.1.smithi192.stdout:4/279: mknod d2/d9/d25/d2e/c63 0 2024-04-24T22:45:26.076 INFO:tasks.workunit.client.1.smithi192.stdout:7/333: mkdir d2/db/d21/d42/d67 0 2024-04-24T22:45:26.077 INFO:tasks.workunit.client.1.smithi192.stdout:7/334: read d2/db/d21/f22 [3256093,17250] 0 2024-04-24T22:45:26.078 INFO:tasks.workunit.client.1.smithi192.stdout:5/264: mkdir d0/d13/d3b/d2f/d34/d4f/d52 0 2024-04-24T22:45:26.078 INFO:tasks.workunit.client.0.smithi063.stdout:9/411: link d6/d8/db/d2b/l64 d6/d8/db/d6b/d4f/d59/l84 0 2024-04-24T22:45:26.080 INFO:tasks.workunit.client.0.smithi063.stdout:5/409: readlink d5/d24/d36/l90 0 2024-04-24T22:45:26.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/432: creat da/d48/f95 x:0 0 0 2024-04-24T22:45:26.083 INFO:tasks.workunit.client.1.smithi192.stdout:0/401: read - d3/d27/d28/f56 zero size 2024-04-24T22:45:26.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/307: rename d0/d8/d29/f53 to d0/d8/f6f 0 2024-04-24T22:45:26.086 INFO:tasks.workunit.client.0.smithi063.stdout:9/412: dwrite d6/d2f/d42/f7b [0,4194304] 0 2024-04-24T22:45:26.087 INFO:tasks.workunit.client.0.smithi063.stdout:8/417: symlink d1/d11/d32/d6a/l8a 0 2024-04-24T22:45:26.089 INFO:tasks.workunit.client.1.smithi192.stdout:3/357: symlink d2/d4/d12/d1a/d1d/d55/l74 0 2024-04-24T22:45:26.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/433: dread da/f14 [0,4194304] 0 2024-04-24T22:45:26.090 INFO:tasks.workunit.client.0.smithi063.stdout:5/410: dread d5/d11/d18/f20 [0,4194304] 0 2024-04-24T22:45:26.092 INFO:tasks.workunit.client.0.smithi063.stdout:1/464: creat d3/da/d34/f9a x:0 0 0 2024-04-24T22:45:26.094 INFO:tasks.workunit.client.0.smithi063.stdout:5/411: write d5/d24/d36/f6c [103509,28278] 0 2024-04-24T22:45:26.096 INFO:tasks.workunit.client.0.smithi063.stdout:1/465: chown d3/da/d34/d6d/c8e 110729142 1 2024-04-24T22:45:26.097 INFO:tasks.workunit.client.0.smithi063.stdout:7/344: rename d0/f72 to d0/d4/de/d17/d35/f79 0 2024-04-24T22:45:26.097 INFO:tasks.workunit.client.0.smithi063.stdout:9/413: dwrite d6/f7 [0,4194304] 0 2024-04-24T22:45:26.099 INFO:tasks.workunit.client.1.smithi192.stdout:1/288: getdents d9/d15/d2d/d31 0 2024-04-24T22:45:26.099 INFO:tasks.workunit.client.1.smithi192.stdout:1/289: chown d9/d15/d2d/d31/d2f/c4b 123934 1 2024-04-24T22:45:26.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/420: creat db/d1d/f91 x:0 0 0 2024-04-24T22:45:26.103 INFO:tasks.workunit.client.1.smithi192.stdout:7/335: rmdir d2/db/d49 39 2024-04-24T22:45:26.106 INFO:tasks.workunit.client.1.smithi192.stdout:5/265: dread d0/d1b/d1c/d2c/f44 [0,4194304] 0 2024-04-24T22:45:26.109 INFO:tasks.workunit.client.1.smithi192.stdout:8/327: rename db/dc/d22/d31/d33/d62 to db/d5d/d63 0 2024-04-24T22:45:26.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/328: fsync db/dc/f1b 0 2024-04-24T22:45:26.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/329: chown db/f3d 42783001 1 2024-04-24T22:45:26.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/434: unlink da/d18/d47/d7b/c8b 0 2024-04-24T22:45:26.113 INFO:tasks.workunit.client.1.smithi192.stdout:3/358: creat d2/d62/f75 x:0 0 0 2024-04-24T22:45:26.115 INFO:tasks.workunit.client.1.smithi192.stdout:9/331: rmdir d3/dd/de/d17/d6a/d6f 0 2024-04-24T22:45:26.119 INFO:tasks.workunit.client.0.smithi063.stdout:9/414: symlink d6/d2f/l85 0 2024-04-24T22:45:26.119 INFO:tasks.workunit.client.0.smithi063.stdout:9/415: chown d6/d8/db/d2b 8 1 2024-04-24T22:45:26.120 INFO:tasks.workunit.client.1.smithi192.stdout:4/280: mkdir d2/d9/d25/d2e/d31/d36/d5b/d64 0 2024-04-24T22:45:26.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/441: link d0/d2/d27/d3a/l3b d0/d2/d2d/d31/d89/l8b 0 2024-04-24T22:45:26.128 INFO:tasks.workunit.client.0.smithi063.stdout:8/418: write d1/d2/f2f [327311,40021] 0 2024-04-24T22:45:26.129 INFO:tasks.workunit.client.1.smithi192.stdout:7/336: dwrite d2/db/f15 [0,4194304] 0 2024-04-24T22:45:26.131 INFO:tasks.workunit.client.0.smithi063.stdout:3/435: creat da/d18/d3a/f96 x:0 0 0 2024-04-24T22:45:26.133 INFO:tasks.workunit.client.0.smithi063.stdout:1/466: mknod d3/da/dc/d1f/d21/c9b 0 2024-04-24T22:45:26.139 INFO:tasks.workunit.client.0.smithi063.stdout:7/345: mkdir d0/d4/de/d17/d35/d6c/d7a 0 2024-04-24T22:45:26.139 INFO:tasks.workunit.client.0.smithi063.stdout:2/442: mknod d0/d2/d27/c8c 0 2024-04-24T22:45:26.140 INFO:tasks.workunit.client.1.smithi192.stdout:6/308: mknod d0/d8/d3d/d3e/d40/d44/c70 0 2024-04-24T22:45:26.140 INFO:tasks.workunit.client.1.smithi192.stdout:1/290: dwrite d9/dd/f2b [0,4194304] 0 2024-04-24T22:45:26.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/436: dwrite da/d18/f8d [0,4194304] 0 2024-04-24T22:45:26.140 INFO:tasks.workunit.client.0.smithi063.stdout:2/443: fsync d0/d2/f1e 0 2024-04-24T22:45:26.141 INFO:tasks.workunit.client.1.smithi192.stdout:1/291: dread - d9/d15/f49 zero size 2024-04-24T22:45:26.142 INFO:tasks.workunit.client.1.smithi192.stdout:3/359: unlink d2/d4/d12/f53 0 2024-04-24T22:45:26.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/292: fdatasync d9/dd/f29 0 2024-04-24T22:45:26.143 INFO:tasks.workunit.client.0.smithi063.stdout:3/437: write da/d18/d47/d4d/f8c [541895,37675] 0 2024-04-24T22:45:26.143 INFO:tasks.workunit.client.0.smithi063.stdout:6/421: unlink db/d1d/f91 0 2024-04-24T22:45:26.146 INFO:tasks.workunit.client.1.smithi192.stdout:9/332: rename d3/dd/de/c20 to d3/d24/c75 0 2024-04-24T22:45:26.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/467: rmdir d3/da/dc/d1f/d2b 39 2024-04-24T22:45:26.148 INFO:tasks.workunit.client.1.smithi192.stdout:3/360: read d2/d4/d12/f42 [48051,29510] 0 2024-04-24T22:45:26.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/468: stat d3/f2c 0 2024-04-24T22:45:26.149 INFO:tasks.workunit.client.0.smithi063.stdout:9/416: mknod d6/d8/c86 0 2024-04-24T22:45:26.149 INFO:tasks.workunit.client.0.smithi063.stdout:2/444: dwrite d0/d2/d2d/f81 [0,4194304] 0 2024-04-24T22:45:26.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/436: getdents d4/d9 0 2024-04-24T22:45:26.153 INFO:tasks.workunit.client.1.smithi192.stdout:2/437: truncate d4/d29/f86 680971 0 2024-04-24T22:45:26.154 INFO:tasks.workunit.client.1.smithi192.stdout:5/266: mkdir d0/d13/d53 0 2024-04-24T22:45:26.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/402: creat d3/d27/d28/d2e/d4a/f8c x:0 0 0 2024-04-24T22:45:26.156 INFO:tasks.workunit.client.1.smithi192.stdout:5/267: write d0/d13/d3b/f3a [225173,119077] 0 2024-04-24T22:45:26.156 INFO:tasks.workunit.client.1.smithi192.stdout:2/438: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:26.156 INFO:tasks.workunit.client.1.smithi192.stdout:0/403: chown d3/f10 27311106 1 2024-04-24T22:45:26.157 INFO:tasks.workunit.client.1.smithi192.stdout:0/404: chown d3/d27/d3c/d64 135 1 2024-04-24T22:45:26.160 INFO:tasks.workunit.client.0.smithi063.stdout:9/417: dwrite d6/d8/db/d6b/d4f/f79 [0,4194304] 0 2024-04-24T22:45:26.163 INFO:tasks.workunit.client.0.smithi063.stdout:9/418: write d6/d8/d1f/d20/d22/d31/f68 [801024,121773] 0 2024-04-24T22:45:26.164 INFO:tasks.workunit.client.0.smithi063.stdout:3/438: mknod da/d39/d94/c97 0 2024-04-24T22:45:26.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/422: symlink db/d1d/d2d/l92 0 2024-04-24T22:45:26.169 INFO:tasks.workunit.client.0.smithi063.stdout:1/469: fdatasync d3/da/dc/f3a 0 2024-04-24T22:45:26.172 INFO:tasks.workunit.client.0.smithi063.stdout:9/419: dwrite d6/d8/db/d2b/d36/f40 [0,4194304] 0 2024-04-24T22:45:26.178 INFO:tasks.workunit.client.1.smithi192.stdout:2/439: mknod d4/d29/d60/c88 0 2024-04-24T22:45:26.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/423: rename db/d1d/d20/d3e/d47 to db/d1d/d20/d3f/d5f/d93 0 2024-04-24T22:45:26.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/424: chown db/d1d/d20/d29 13 1 2024-04-24T22:45:26.182 INFO:tasks.workunit.client.1.smithi192.stdout:8/330: rmdir db/d5d/d63 0 2024-04-24T22:45:26.188 INFO:tasks.workunit.client.0.smithi063.stdout:9/420: mknod d6/d8/db/d6b/c87 0 2024-04-24T22:45:26.188 INFO:tasks.workunit.client.1.smithi192.stdout:8/331: dwrite db/f56 [0,4194304] 0 2024-04-24T22:45:26.189 INFO:tasks.workunit.client.1.smithi192.stdout:8/332: truncate f3 2203768 0 2024-04-24T22:45:26.191 INFO:tasks.workunit.client.1.smithi192.stdout:1/293: symlink d9/d15/d2d/l5f 0 2024-04-24T22:45:26.191 INFO:tasks.workunit.client.0.smithi063.stdout:2/445: write d0/d2/d2d/d31/d89/d7/f13 [3023804,120790] 0 2024-04-24T22:45:26.192 INFO:tasks.workunit.client.0.smithi063.stdout:6/425: rmdir db/d1d/d20/d3f/d7f 39 2024-04-24T22:45:26.193 INFO:tasks.workunit.client.0.smithi063.stdout:2/446: chown d0/d2/d27/c83 26986 1 2024-04-24T22:45:26.193 INFO:tasks.workunit.client.0.smithi063.stdout:6/426: write db/f1a [3282731,117667] 0 2024-04-24T22:45:26.195 INFO:tasks.workunit.client.0.smithi063.stdout:1/470: unlink d3/da/dc/d1f/d21/d42/l7b 0 2024-04-24T22:45:26.199 INFO:tasks.workunit.client.1.smithi192.stdout:9/333: truncate d3/dd/de/f16 2558003 0 2024-04-24T22:45:26.201 INFO:tasks.workunit.client.1.smithi192.stdout:7/337: link d2/db/l1f d2/db/d3a/l68 0 2024-04-24T22:45:26.202 INFO:tasks.workunit.client.0.smithi063.stdout:6/427: stat db/d1d/d20/c4a 0 2024-04-24T22:45:26.203 INFO:tasks.workunit.client.1.smithi192.stdout:5/268: mknod d0/d1b/d50/c54 0 2024-04-24T22:45:26.204 INFO:tasks.workunit.client.0.smithi063.stdout:0/369: sync 2024-04-24T22:45:26.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/447: dwrite d0/d2/d2f/f58 [0,4194304] 0 2024-04-24T22:45:26.210 INFO:tasks.workunit.client.1.smithi192.stdout:6/309: creat d0/f71 x:0 0 0 2024-04-24T22:45:26.215 INFO:tasks.workunit.client.1.smithi192.stdout:3/361: rename d2/d4/d12/d1a/f5d to d2/f76 0 2024-04-24T22:45:26.217 INFO:tasks.workunit.client.0.smithi063.stdout:6/428: dwrite db/d1d/d20/f50 [0,4194304] 0 2024-04-24T22:45:26.217 INFO:tasks.workunit.client.1.smithi192.stdout:3/362: truncate d2/d27/f4a 1015300 0 2024-04-24T22:45:26.218 INFO:tasks.workunit.client.1.smithi192.stdout:4/281: getdents d2/d9/d25/d2e/d31/d36/d5b/d28 0 2024-04-24T22:45:26.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/448: dwrite d0/d2/d2d/d31/d89/d48/f6f [0,4194304] 0 2024-04-24T22:45:26.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/363: read d2/d27/d28/f63 [289367,100313] 0 2024-04-24T22:45:26.220 INFO:tasks.workunit.client.1.smithi192.stdout:0/405: write d3/d27/d28/d2e/f4c [1500083,84110] 0 2024-04-24T22:45:26.221 INFO:tasks.workunit.client.0.smithi063.stdout:9/421: dread d6/d8/d1f/d20/d22/f3b [0,4194304] 0 2024-04-24T22:45:26.225 INFO:tasks.workunit.client.1.smithi192.stdout:7/338: creat d2/db/d25/f69 x:0 0 0 2024-04-24T22:45:26.226 INFO:tasks.workunit.client.1.smithi192.stdout:7/339: fdatasync d2/db/f15 0 2024-04-24T22:45:26.227 INFO:tasks.workunit.client.1.smithi192.stdout:7/340: chown d2/db/d36/c46 254782565 1 2024-04-24T22:45:26.231 INFO:tasks.workunit.client.1.smithi192.stdout:8/333: unlink db/dc/c2d 0 2024-04-24T22:45:26.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/341: dread d2/db/f15 [0,4194304] 0 2024-04-24T22:45:26.234 INFO:tasks.workunit.client.1.smithi192.stdout:2/440: rename d4/d29/f51 to d4/d29/d60/f89 0 2024-04-24T22:45:26.237 INFO:tasks.workunit.client.1.smithi192.stdout:4/282: symlink d2/d9/d25/d2e/d31/d36/l65 0 2024-04-24T22:45:26.239 INFO:tasks.workunit.client.1.smithi192.stdout:5/269: dread d0/fc [0,4194304] 0 2024-04-24T22:45:26.241 INFO:tasks.workunit.client.1.smithi192.stdout:9/334: mknod d3/dd/de/d17/c76 0 2024-04-24T22:45:26.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/439: getdents da 0 2024-04-24T22:45:26.243 INFO:tasks.workunit.client.1.smithi192.stdout:8/334: mknod db/d5d/c64 0 2024-04-24T22:45:26.245 INFO:tasks.workunit.client.0.smithi063.stdout:2/449: dread d0/d2/d63/f71 [0,4194304] 0 2024-04-24T22:45:26.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/429: write db/d1d/d20/d3f/d7f/f88 [4936487,42923] 0 2024-04-24T22:45:26.248 INFO:tasks.workunit.client.0.smithi063.stdout:9/422: creat d6/d8/d53/f88 x:0 0 0 2024-04-24T22:45:26.250 INFO:tasks.workunit.client.0.smithi063.stdout:6/430: fsync db/d1d/d2d/f42 0 2024-04-24T22:45:26.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/450: read d0/d2/d2d/d31/d89/d7/fc [1705324,14146] 0 2024-04-24T22:45:26.250 INFO:tasks.workunit.client.0.smithi063.stdout:0/370: dwrite d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:26.251 INFO:tasks.workunit.client.1.smithi192.stdout:4/283: creat d2/d9/d25/d2e/f66 x:0 0 0 2024-04-24T22:45:26.251 INFO:tasks.workunit.client.0.smithi063.stdout:6/431: read - db/d1d/d20/d3e/f87 zero size 2024-04-24T22:45:26.253 INFO:tasks.workunit.client.0.smithi063.stdout:0/371: write d6/da/d2b/d60/f76 [1230700,59367] 0 2024-04-24T22:45:26.259 INFO:tasks.workunit.client.1.smithi192.stdout:8/335: write db/dc/f18 [2012363,114692] 0 2024-04-24T22:45:26.260 INFO:tasks.workunit.client.0.smithi063.stdout:2/451: dwrite d0/d2/d39/f65 [0,4194304] 0 2024-04-24T22:45:26.261 INFO:tasks.workunit.client.0.smithi063.stdout:3/440: link da/d18/d3a/f69 da/d39/f98 0 2024-04-24T22:45:26.264 INFO:tasks.workunit.client.0.smithi063.stdout:9/423: symlink d6/d8/d1f/l89 0 2024-04-24T22:45:26.267 INFO:tasks.workunit.client.1.smithi192.stdout:9/335: mknod d3/dd/c77 0 2024-04-24T22:45:26.270 INFO:tasks.workunit.client.1.smithi192.stdout:6/310: getdents d0/d8/d3d 0 2024-04-24T22:45:26.270 INFO:tasks.workunit.client.0.smithi063.stdout:6/432: write db/d1d/d20/f50 [5162034,5274] 0 2024-04-24T22:45:26.271 INFO:tasks.workunit.client.0.smithi063.stdout:0/372: creat d6/d64/d21/d25/d62/d63/f82 x:0 0 0 2024-04-24T22:45:26.272 INFO:tasks.workunit.client.1.smithi192.stdout:3/364: dwrite d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:26.272 INFO:tasks.workunit.client.0.smithi063.stdout:3/441: dwrite da/d18/d3a/f3f [0,4194304] 0 2024-04-24T22:45:26.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/452: dwrite d0/d2/d27/f85 [0,4194304] 0 2024-04-24T22:45:26.276 INFO:tasks.workunit.client.1.smithi192.stdout:9/336: dread d3/dd/f30 [0,4194304] 0 2024-04-24T22:45:26.279 INFO:tasks.workunit.client.0.smithi063.stdout:2/453: dread d0/d2/d39/f65 [0,4194304] 0 2024-04-24T22:45:26.282 INFO:tasks.workunit.client.1.smithi192.stdout:1/294: rename d9/d15/d2d/l3b to d9/dd/l60 0 2024-04-24T22:45:26.286 INFO:tasks.workunit.client.1.smithi192.stdout:5/270: creat d0/d13/d3b/f55 x:0 0 0 2024-04-24T22:45:26.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/295: dread d9/dd/f19 [0,4194304] 0 2024-04-24T22:45:26.292 INFO:tasks.workunit.client.1.smithi192.stdout:6/311: creat d0/d8/d29/d4c/d4d/f72 x:0 0 0 2024-04-24T22:45:26.293 INFO:tasks.workunit.client.0.smithi063.stdout:3/442: write da/d18/d47/f85 [1065785,116607] 0 2024-04-24T22:45:26.295 INFO:tasks.workunit.client.1.smithi192.stdout:3/365: write d2/d4/f3e [190723,9605] 0 2024-04-24T22:45:26.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/337: mknod d3/dd/d62/c78 0 2024-04-24T22:45:26.298 INFO:tasks.workunit.client.1.smithi192.stdout:2/441: dwrite d4/f27 [0,4194304] 0 2024-04-24T22:45:26.298 INFO:tasks.workunit.client.0.smithi063.stdout:9/424: mkdir d6/d8/d8a 0 2024-04-24T22:45:26.299 INFO:tasks.workunit.client.1.smithi192.stdout:9/338: read d3/dd/de/f2f [468196,87302] 0 2024-04-24T22:45:26.300 INFO:tasks.workunit.client.0.smithi063.stdout:6/433: mkdir db/d1d/d20/d3f/d94 0 2024-04-24T22:45:26.301 INFO:tasks.workunit.client.1.smithi192.stdout:9/339: truncate d3/d24/f44 166456 0 2024-04-24T22:45:26.302 INFO:tasks.workunit.client.1.smithi192.stdout:7/342: rename d2/db/d36/l4f to d2/db/d21/d42/d67/l6a 0 2024-04-24T22:45:26.303 INFO:tasks.workunit.client.1.smithi192.stdout:9/340: fsync d3/dd/de/d17/d6a/f54 0 2024-04-24T22:45:26.303 INFO:tasks.workunit.client.0.smithi063.stdout:6/434: write db/d1d/d20/f40 [520538,31734] 0 2024-04-24T22:45:26.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/443: symlink da/d18/d47/d4d/l99 0 2024-04-24T22:45:26.304 INFO:tasks.workunit.client.1.smithi192.stdout:7/343: write d2/db/d2a/f2e [970246,40097] 0 2024-04-24T22:45:26.304 INFO:tasks.workunit.client.1.smithi192.stdout:5/271: mknod d0/d1b/d1c/d2c/c56 0 2024-04-24T22:45:26.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/425: dwrite d6/f7 [0,4194304] 0 2024-04-24T22:45:26.307 INFO:tasks.workunit.client.0.smithi063.stdout:3/444: truncate da/d39/f7a 153362 0 2024-04-24T22:45:26.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/272: write d0/d13/d3b/d2f/f46 [172717,84485] 0 2024-04-24T22:45:26.307 INFO:tasks.workunit.client.1.smithi192.stdout:1/296: symlink d9/d15/d2d/d31/d2f/l61 0 2024-04-24T22:45:26.307 INFO:tasks.workunit.client.1.smithi192.stdout:1/297: readlink d9/d15/d2d/d31/l56 0 2024-04-24T22:45:26.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/273: readlink d0/d13/d19/l4c 0 2024-04-24T22:45:26.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/454: symlink d0/d2/d2d/d31/d89/d7/d86/l8d 0 2024-04-24T22:45:26.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/274: dread - d0/d1b/d1c/d2c/f51 zero size 2024-04-24T22:45:26.310 INFO:tasks.workunit.client.1.smithi192.stdout:6/312: creat d0/d8/d3d/d3e/d40/d44/f73 x:0 0 0 2024-04-24T22:45:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.309+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff94c104930 con 0x7ff93806b860 2024-04-24T22:45:26.311 INFO:tasks.workunit.client.1.smithi192.stdout:6/313: dread - d0/d8/d3d/f55 zero size 2024-04-24T22:45:26.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/336: creat db/f65 x:0 0 0 2024-04-24T22:45:26.312 INFO:tasks.workunit.client.0.smithi063.stdout:6/435: write db/d1d/d20/d3f/d7f/d68/d70/f67 [871862,107762] 0 2024-04-24T22:45:26.313 INFO:tasks.workunit.client.0.smithi063.stdout:3/445: mknod da/d18/d47/d4d/c9a 0 2024-04-24T22:45:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.311+0000 7ff942ffd700 1 -- 172.21.15.63:0/1417208477 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff93c00f040 con 0x7ff93806b860 2024-04-24T22:45:26.316 INFO:tasks.workunit.client.0.smithi063.stdout:2/455: mkdir d0/d2/d2d/d31/d89/d7/d86/d8e 0 2024-04-24T22:45:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 msgr2=0x7ff93806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 0x7ff93806dd10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009b80 tx=0x7ff93c009d60 comp rx=0 tx=0).stop 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 msgr2=0x7ff94c10b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b970 tx=0x7ff948009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 shutdown_connections 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff93806b860 0x7ff93806dd10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff94c0fc290 0x7ff94c10b730 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 --2- 172.21.15.63:0/1417208477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff94c0fcbe0 0x7ff94c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 >> 172.21.15.63:0/1417208477 conn(0x7ff94c0f6b80 msgr2=0x7ff94c0fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 shutdown_connections 2024-04-24T22:45:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.315+0000 7ff9546cf700 1 -- 172.21.15.63:0/1417208477 wait complete. 2024-04-24T22:45:26.319 INFO:tasks.workunit.client.1.smithi192.stdout:2/442: unlink d4/f2a 0 2024-04-24T22:45:26.321 INFO:tasks.workunit.client.0.smithi063.stdout:3/446: rmdir da/d39/d94 39 2024-04-24T22:45:26.324 INFO:tasks.workunit.client.0.smithi063.stdout:2/456: mknod d0/d2/d2d/d31/d89/d1c/d3f/c8f 0 2024-04-24T22:45:26.324 INFO:tasks.workunit.client.0.smithi063.stdout:3/447: dread da/f3d [0,4194304] 0 2024-04-24T22:45:26.325 INFO:tasks.workunit.client.0.smithi063.stdout:3/448: chown da/d39 395709418 1 2024-04-24T22:45:26.329 INFO:tasks.workunit.client.1.smithi192.stdout:4/284: rename l0 to d2/d9/d25/d2e/d31/d36/l67 0 2024-04-24T22:45:26.329 INFO:tasks.workunit.client.1.smithi192.stdout:4/285: chown d2/f3 2261 1 2024-04-24T22:45:26.330 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:45:26.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/341: creat d3/dd/de/d39/f79 x:0 0 0 2024-04-24T22:45:26.338 INFO:tasks.workunit.client.0.smithi063.stdout:6/436: getdents db/d1d/d20/d84 0 2024-04-24T22:45:26.344 INFO:tasks.workunit.client.1.smithi192.stdout:1/298: creat d9/d15/d2d/d31/f62 x:0 0 0 2024-04-24T22:45:26.344 INFO:tasks.workunit.client.1.smithi192.stdout:6/314: creat d0/d8/d29/d4c/f74 x:0 0 0 2024-04-24T22:45:26.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/337: creat db/dc/d22/f66 x:0 0 0 2024-04-24T22:45:26.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/338: write db/dc/d22/f5e [358858,115697] 0 2024-04-24T22:45:26.345 INFO:tasks.workunit.client.0.smithi063.stdout:3/449: rmdir da/d18/d47 39 2024-04-24T22:45:26.345 INFO:tasks.workunit.client.0.smithi063.stdout:6/437: dwrite db/d1c/f89 [0,4194304] 0 2024-04-24T22:45:26.345 INFO:tasks.workunit.client.1.smithi192.stdout:3/366: truncate d2/d4/d12/f1c 2365612 0 2024-04-24T22:45:26.346 INFO:tasks.workunit.client.1.smithi192.stdout:3/367: dread - d2/d4/d12/d1a/d1d/d55/d5f/f68 zero size 2024-04-24T22:45:26.349 INFO:tasks.workunit.client.0.smithi063.stdout:3/450: mkdir da/d18/d47/d4d/d9b 0 2024-04-24T22:45:26.354 INFO:tasks.workunit.client.1.smithi192.stdout:5/275: rename d0/d13/d3b/d2f/f2a to d0/d13/d19/f57 0 2024-04-24T22:45:26.356 INFO:tasks.workunit.client.0.smithi063.stdout:3/451: creat da/d40/f9c x:0 0 0 2024-04-24T22:45:26.356 INFO:tasks.workunit.client.1.smithi192.stdout:4/286: truncate d2/d9/d25/f23 176755 0 2024-04-24T22:45:26.357 INFO:tasks.workunit.client.0.smithi063.stdout:2/457: write d0/d2/d2d/d31/d89/d7/f5c [551974,1369] 0 2024-04-24T22:45:26.362 INFO:tasks.workunit.client.0.smithi063.stdout:6/438: link db/d1c/l1e db/d1c/l95 0 2024-04-24T22:45:26.362 INFO:tasks.workunit.client.0.smithi063.stdout:3/452: dwrite da/f15 [0,4194304] 0 2024-04-24T22:45:26.365 INFO:tasks.workunit.client.0.smithi063.stdout:6/439: read db/d1d/d2d/f42 [436923,110634] 0 2024-04-24T22:45:26.366 INFO:tasks.workunit.client.1.smithi192.stdout:7/344: dwrite d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:26.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/443: dread d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:26.368 INFO:tasks.workunit.client.0.smithi063.stdout:6/440: write db/d1d/d20/d3e/f4f [3942516,120563] 0 2024-04-24T22:45:26.368 INFO:tasks.workunit.client.1.smithi192.stdout:2/444: read d4/d9/d37/f4a [400815,54749] 0 2024-04-24T22:45:26.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/315: rename d0/d8/d3d/d3e/d40/c42 to d0/d8/d3d/c75 0 2024-04-24T22:45:26.372 INFO:tasks.workunit.client.1.smithi192.stdout:5/276: symlink d0/d13/d3b/d2f/d34/d4f/l58 0 2024-04-24T22:45:26.374 INFO:tasks.workunit.client.0.smithi063.stdout:3/453: symlink da/d39/l9d 0 2024-04-24T22:45:26.376 INFO:tasks.workunit.client.1.smithi192.stdout:4/287: creat d2/d9/d25/f68 x:0 0 0 2024-04-24T22:45:26.380 INFO:tasks.workunit.client.0.smithi063.stdout:6/441: rmdir db/d1d/d20/d3f/d5f/d93 39 2024-04-24T22:45:26.381 INFO:tasks.workunit.client.0.smithi063.stdout:3/454: dwrite da/d18/f61 [0,4194304] 0 2024-04-24T22:45:26.381 INFO:tasks.workunit.client.0.smithi063.stdout:6/442: dread - db/d1d/d20/d3e/f87 zero size 2024-04-24T22:45:26.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/339: symlink db/dc/l67 0 2024-04-24T22:45:26.384 INFO:tasks.workunit.client.1.smithi192.stdout:9/342: dwrite d3/dd/de/d17/f23 [0,4194304] 0 2024-04-24T22:45:26.385 INFO:tasks.workunit.client.1.smithi192.stdout:8/340: truncate db/dc/d22/f23 754364 0 2024-04-24T22:45:26.386 INFO:tasks.workunit.client.1.smithi192.stdout:7/345: creat d2/db/d21/d42/d5b/f6b x:0 0 0 2024-04-24T22:45:26.387 INFO:tasks.workunit.client.0.smithi063.stdout:3/455: dread da/d18/f8d [0,4194304] 0 2024-04-24T22:45:26.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/458: link d0/d2/d2d/l5a d0/dd/d72/d77/l90 0 2024-04-24T22:45:26.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/459: readlink d0/l4a 0 2024-04-24T22:45:26.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/346: chown d2/db/d36/f38 173284943 1 2024-04-24T22:45:26.389 INFO:tasks.workunit.client.0.smithi063.stdout:6/443: dread db/d1c/f58 [0,4194304] 0 2024-04-24T22:45:26.389 INFO:tasks.workunit.client.1.smithi192.stdout:2/445: creat d4/d9/d37/f8a x:0 0 0 2024-04-24T22:45:26.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/316: rmdir d0/d8/d29 39 2024-04-24T22:45:26.393 INFO:tasks.workunit.client.1.smithi192.stdout:6/317: stat d0/d8/f6f 0 2024-04-24T22:45:26.398 INFO:tasks.workunit.client.1.smithi192.stdout:9/343: dwrite d3/d24/f41 [0,4194304] 0 2024-04-24T22:45:26.398 INFO:tasks.workunit.client.0.smithi063.stdout:6/444: dread db/d1d/d2d/d48/f51 [4194304,4194304] 0 2024-04-24T22:45:26.399 INFO:tasks.workunit.client.0.smithi063.stdout:6/445: truncate db/d1c/f90 29864 0 2024-04-24T22:45:26.400 INFO:tasks.workunit.client.0.smithi063.stdout:6/446: read db/f10 [1552849,65547] 0 2024-04-24T22:45:26.401 INFO:tasks.workunit.client.1.smithi192.stdout:8/341: dwrite f3 [0,4194304] 0 2024-04-24T22:45:26.404 INFO:tasks.workunit.client.1.smithi192.stdout:2/446: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:26.404 INFO:tasks.workunit.client.0.smithi063.stdout:3/456: mkdir da/d18/d9e 0 2024-04-24T22:45:26.406 INFO:tasks.workunit.client.0.smithi063.stdout:3/457: readlink da/d18/d47/l75 0 2024-04-24T22:45:26.410 INFO:tasks.workunit.client.0.smithi063.stdout:6/447: dwrite db/d1c/f90 [0,4194304] 0 2024-04-24T22:45:26.413 INFO:tasks.workunit.client.1.smithi192.stdout:6/318: chown d0/d8/d29/d5a/d63 52 1 2024-04-24T22:45:26.414 INFO:tasks.workunit.client.1.smithi192.stdout:6/319: fdatasync d0/d8/d29/d5a/f6b 0 2024-04-24T22:45:26.419 INFO:tasks.workunit.client.1.smithi192.stdout:9/344: unlink l0 0 2024-04-24T22:45:26.420 INFO:tasks.workunit.client.0.smithi063.stdout:6/448: stat db/d1c/l95 0 2024-04-24T22:45:26.425 INFO:tasks.workunit.client.0.smithi063.stdout:6/449: rmdir db/d1d/d2d/d69/d82 39 2024-04-24T22:45:26.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/342: mknod db/d43/c68 0 2024-04-24T22:45:26.427 INFO:tasks.workunit.client.0.smithi063.stdout:2/460: dread d0/d2/d2d/d31/d89/d7/f64 [0,4194304] 0 2024-04-24T22:45:26.427 INFO:tasks.workunit.client.1.smithi192.stdout:9/345: dwrite d3/dd/de/d17/f48 [0,4194304] 0 2024-04-24T22:45:26.428 INFO:tasks.workunit.client.0.smithi063.stdout:3/458: link da/d18/d47/d4d/c6b da/d18/d47/d7b/c9f 0 2024-04-24T22:45:26.429 INFO:tasks.workunit.client.0.smithi063.stdout:2/461: write d0/d2/d2f/f6d [1815750,108457] 0 2024-04-24T22:45:26.430 INFO:tasks.workunit.client.1.smithi192.stdout:2/447: stat d4/d29/c71 0 2024-04-24T22:45:26.430 INFO:tasks.workunit.client.0.smithi063.stdout:2/462: write d0/d2/d2f/f58 [3173095,89993] 0 2024-04-24T22:45:26.433 INFO:tasks.workunit.client.1.smithi192.stdout:7/347: creat d2/db/d49/f6c x:0 0 0 2024-04-24T22:45:26.435 INFO:tasks.workunit.client.0.smithi063.stdout:3/459: creat da/fa0 x:0 0 0 2024-04-24T22:45:26.436 INFO:tasks.workunit.client.1.smithi192.stdout:6/320: unlink d0/d8/d3d/d3e/c48 0 2024-04-24T22:45:26.438 INFO:tasks.workunit.client.0.smithi063.stdout:6/450: mkdir db/d1d/d2d/d69/d82/d96 0 2024-04-24T22:45:26.440 INFO:tasks.workunit.client.0.smithi063.stdout:2/463: mknod d0/d2/d2d/d31/d89/d48/c91 0 2024-04-24T22:45:26.442 INFO:tasks.workunit.client.0.smithi063.stdout:2/464: read - d0/dd/d72/d77/f78 zero size 2024-04-24T22:45:26.442 INFO:tasks.workunit.client.1.smithi192.stdout:8/343: rename db/dc/d29/f2c to db/d43/f69 0 2024-04-24T22:45:26.443 INFO:tasks.workunit.client.1.smithi192.stdout:4/288: dwrite d2/d9/d25/d2e/d31/d36/d5b/f37 [0,4194304] 0 2024-04-24T22:45:26.443 INFO:tasks.workunit.client.1.smithi192.stdout:8/344: readlink db/dc/d29/l36 0 2024-04-24T22:45:26.444 INFO:tasks.workunit.client.1.smithi192.stdout:8/345: stat db/d5d 0 2024-04-24T22:45:26.445 INFO:tasks.workunit.client.1.smithi192.stdout:2/448: creat d4/d9/f8b x:0 0 0 2024-04-24T22:45:26.446 INFO:tasks.workunit.client.1.smithi192.stdout:5/277: getdents d0/d13/d3b 0 2024-04-24T22:45:26.447 INFO:tasks.workunit.client.0.smithi063.stdout:6/451: dwrite db/d1d/d20/d29/d5d/f6b [0,4194304] 0 2024-04-24T22:45:26.451 INFO:tasks.workunit.client.0.smithi063.stdout:6/452: rmdir db/d1d/d20/d29 39 2024-04-24T22:45:26.452 INFO:tasks.workunit.client.0.smithi063.stdout:2/465: dwrite d0/d2/f47 [0,4194304] 0 2024-04-24T22:45:26.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/346: truncate d3/dd/de/f12 4175510 0 2024-04-24T22:45:26.462 INFO:tasks.workunit.client.0.smithi063.stdout:2/466: readlink d0/d2/d27/d3a/l3b 0 2024-04-24T22:45:26.462 INFO:tasks.workunit.client.1.smithi192.stdout:8/346: dwrite db/dc/f41 [0,4194304] 0 2024-04-24T22:45:26.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/348: link f0 d2/db/d21/d42/d5b/d63/f6d 0 2024-04-24T22:45:26.469 INFO:tasks.workunit.client.1.smithi192.stdout:8/347: dwrite db/dc/f1b [0,4194304] 0 2024-04-24T22:45:26.471 INFO:tasks.workunit.client.0.smithi063.stdout:2/467: unlink d0/f57 0 2024-04-24T22:45:26.474 INFO:tasks.workunit.client.0.smithi063.stdout:8/419: sync 2024-04-24T22:45:26.474 INFO:tasks.workunit.client.0.smithi063.stdout:5/412: sync 2024-04-24T22:45:26.474 INFO:tasks.workunit.client.0.smithi063.stdout:4/380: sync 2024-04-24T22:45:26.476 INFO:tasks.workunit.client.1.smithi192.stdout:8/348: dwrite db/dc/f13 [0,4194304] 0 2024-04-24T22:45:26.477 INFO:tasks.workunit.client.0.smithi063.stdout:8/420: read - d1/d11/d32/f65 zero size 2024-04-24T22:45:26.479 INFO:tasks.workunit.client.1.smithi192.stdout:4/289: symlink d2/l69 0 2024-04-24T22:45:26.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/413: write d5/d1d/d7d/f52 [1049050,61924] 0 2024-04-24T22:45:26.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/468: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:26.480 INFO:tasks.workunit.client.0.smithi063.stdout:8/421: chown d1/d2/fd 0 1 2024-04-24T22:45:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 -- 172.21.15.63:0/4147287779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fbcb0 msgr2=0x7f87780fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 --2- 172.21.15.63:0/4147287779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fbcb0 0x7f87780fc0c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8768004940 tx=0x7f8768009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 -- 172.21.15.63:0/4147287779 shutdown_connections 2024-04-24T22:45:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 --2- 172.21.15.63:0/4147287779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87780fcf60 0x7f87780fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 --2- 172.21.15.63:0/4147287779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fbcb0 0x7f87780fc0c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 -- 172.21.15.63:0/4147287779 >> 172.21.15.63:0/4147287779 conn(0x7f87780f7430 msgr2=0x7f87780f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 -- 172.21.15.63:0/4147287779 shutdown_connections 2024-04-24T22:45:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.481+0000 7f877d5e3700 1 -- 172.21.15.63:0/4147287779 wait complete. 2024-04-24T22:45:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.483+0000 7f877d5e3700 1 Processor -- start 2024-04-24T22:45:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f877d5e3700 1 -- start start 2024-04-24T22:45:26.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f877d5e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87780fbcb0 0x7f8778114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f877d5e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f877d5e3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778114b70 con 0x7f87780fcf60 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f877d5e3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781153c0 con 0x7f87780fbcb0 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.484+0000 7f87767fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47284/0 (socket says 172.21.15.63:47284) 2024-04-24T22:45:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 -- 172.21.15.63:0/472452401 learned_addr learned my addr 172.21.15.63:0/472452401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:26.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 -- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87780fbcb0 msgr2=0x7f8778114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87780fbcb0 0x7f8778114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 -- 172.21.15.63:0/472452401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768005040 con 0x7f87780fcf60 2024-04-24T22:45:26.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.485+0000 7f87767fc700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f876c002a50 tx=0x7f876c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:26.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.486+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f876c007cc0 con 0x7f87780fcf60 2024-04-24T22:45:26.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/453: creat db/d1d/d20/d3f/d7f/f97 x:0 0 0 2024-04-24T22:45:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.486+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87781156a0 con 0x7f87780fcf60 2024-04-24T22:45:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.486+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8778115be0 con 0x7f87780fcf60 2024-04-24T22:45:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.486+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f876c00f040 con 0x7f87780fcf60 2024-04-24T22:45:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.486+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f876c00daf0 con 0x7f87780fcf60 2024-04-24T22:45:26.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.488+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640052e0 con 0x7f87780fcf60 2024-04-24T22:45:26.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/278: mknod d0/d13/d53/c59 0 2024-04-24T22:45:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.489+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f876c00dc50 con 0x7f87780fcf60 2024-04-24T22:45:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.490+0000 7f875f7fe700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 0x7f876006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.490+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f876c089cb0 con 0x7f87780fcf60 2024-04-24T22:45:26.492 INFO:tasks.workunit.client.1.smithi192.stdout:7/349: mkdir d2/db/d3a/d6e 0 2024-04-24T22:45:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.492+0000 7f8776ffd700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 0x7f876006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:26.495 INFO:tasks.workunit.client.1.smithi192.stdout:6/321: creat d0/d8/f76 x:0 0 0 2024-04-24T22:45:26.496 INFO:tasks.workunit.client.0.smithi063.stdout:4/381: creat d2/d8/d36/d3a/d50/f77 x:0 0 0 2024-04-24T22:45:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.495+0000 7f8776ffd700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 0x7f876006dc90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8768004f90 tx=0x7f8768009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:26.497 INFO:tasks.workunit.client.1.smithi192.stdout:7/350: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:26.498 INFO:tasks.workunit.client.0.smithi063.stdout:4/382: write d2/d19/f4d [2389291,41055] 0 2024-04-24T22:45:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.497+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f876c057220 con 0x7f87780fcf60 2024-04-24T22:45:26.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/414: mkdir d5/d11/d23/d2d/d62/d73/d95 0 2024-04-24T22:45:26.506 INFO:tasks.workunit.client.0.smithi063.stdout:5/415: mknod d5/d1d/d7d/c96 0 2024-04-24T22:45:26.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/460: dread da/d18/d3a/f3f [4194304,4194304] 0 2024-04-24T22:45:26.508 INFO:tasks.workunit.client.1.smithi192.stdout:2/449: mkdir d4/d9/d10/d6c/d8c 0 2024-04-24T22:45:26.508 INFO:tasks.workunit.client.0.smithi063.stdout:5/416: dread - d5/d1d/f93 zero size 2024-04-24T22:45:26.508 INFO:tasks.workunit.client.0.smithi063.stdout:3/461: readlink da/d18/d3a/l8f 0 2024-04-24T22:45:26.508 INFO:tasks.workunit.client.0.smithi063.stdout:6/454: mknod db/d1d/d20/d3f/d5f/c98 0 2024-04-24T22:45:26.509 INFO:tasks.workunit.client.0.smithi063.stdout:4/383: mknod d2/c78 0 2024-04-24T22:45:26.514 INFO:tasks.workunit.client.0.smithi063.stdout:4/384: symlink d2/d19/d33/l79 0 2024-04-24T22:45:26.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/462: dread da/d40/f78 [0,4194304] 0 2024-04-24T22:45:26.516 INFO:tasks.workunit.client.0.smithi063.stdout:4/385: write d2/f26 [5006737,28986] 0 2024-04-24T22:45:26.516 INFO:tasks.workunit.client.0.smithi063.stdout:8/422: dread d1/d21/f6f [0,4194304] 0 2024-04-24T22:45:26.518 INFO:tasks.workunit.client.1.smithi192.stdout:4/290: creat d2/d9/d25/d2e/d31/d36/d5b/f6a x:0 0 0 2024-04-24T22:45:26.518 INFO:tasks.workunit.client.0.smithi063.stdout:8/423: write d1/d11/d49/d54/d7b/f7f [633542,40207] 0 2024-04-24T22:45:26.519 INFO:tasks.workunit.client.1.smithi192.stdout:4/291: chown d2/db 0 1 2024-04-24T22:45:26.520 INFO:tasks.workunit.client.0.smithi063.stdout:6/455: creat db/d1d/d20/d29/d5d/f99 x:0 0 0 2024-04-24T22:45:26.520 INFO:tasks.workunit.client.0.smithi063.stdout:6/456: read db/f1a [4837481,105212] 0 2024-04-24T22:45:26.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/351: creat d2/db/f6f x:0 0 0 2024-04-24T22:45:26.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/424: dread d1/d2/f2f [0,4194304] 0 2024-04-24T22:45:26.525 INFO:tasks.workunit.client.1.smithi192.stdout:9/347: dwrite d3/f21 [0,4194304] 0 2024-04-24T22:45:26.526 INFO:tasks.workunit.client.0.smithi063.stdout:3/463: read - da/f1d zero size 2024-04-24T22:45:26.530 INFO:tasks.workunit.client.1.smithi192.stdout:4/292: dwrite d2/d9/d25/f68 [0,4194304] 0 2024-04-24T22:45:26.532 INFO:tasks.workunit.client.1.smithi192.stdout:4/293: chown d2/d9/d25 4299 1 2024-04-24T22:45:26.533 INFO:tasks.workunit.client.1.smithi192.stdout:5/279: write d0/d13/d19/f1d [2495380,24751] 0 2024-04-24T22:45:26.534 INFO:tasks.workunit.client.1.smithi192.stdout:4/294: stat d2/d9/d25/d2e/d31/d36/d5b/d28/c43 0 2024-04-24T22:45:26.534 INFO:tasks.workunit.client.1.smithi192.stdout:5/280: chown d0/d1b/d1c/d2c/c56 13 1 2024-04-24T22:45:26.537 INFO:tasks.workunit.client.1.smithi192.stdout:9/348: dread d3/d24/f4e [0,4194304] 0 2024-04-24T22:45:26.537 INFO:tasks.workunit.client.1.smithi192.stdout:2/450: dread d4/d9/d37/f55 [0,4194304] 0 2024-04-24T22:45:26.538 INFO:tasks.workunit.client.0.smithi063.stdout:2/469: dwrite d0/d2/d27/f34 [0,4194304] 0 2024-04-24T22:45:26.541 INFO:tasks.workunit.client.0.smithi063.stdout:5/417: rename d5/d11/d23/f66 to d5/d24/d49/f97 0 2024-04-24T22:45:26.541 INFO:tasks.workunit.client.1.smithi192.stdout:6/322: dwrite d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:26.544 INFO:tasks.workunit.client.0.smithi063.stdout:6/457: creat db/d1d/d2d/d69/d82/f9a x:0 0 0 2024-04-24T22:45:26.547 INFO:tasks.workunit.client.0.smithi063.stdout:6/458: write db/d1c/f58 [964760,128967] 0 2024-04-24T22:45:26.549 INFO:tasks.workunit.client.0.smithi063.stdout:6/459: readlink db/d1d/d20/d3f/d7f/d68/d70/l79 0 2024-04-24T22:45:26.550 INFO:tasks.workunit.client.0.smithi063.stdout:2/470: dwrite d0/d2/d2f/f6d [0,4194304] 0 2024-04-24T22:45:26.551 INFO:tasks.workunit.client.0.smithi063.stdout:6/460: write db/d1d/d2d/d48/f6e [1297663,84032] 0 2024-04-24T22:45:26.552 INFO:tasks.workunit.client.0.smithi063.stdout:2/471: write d0/d2/d27/f29 [3866750,114531] 0 2024-04-24T22:45:26.552 INFO:tasks.workunit.client.0.smithi063.stdout:3/464: mkdir da/d18/d3a/da1 0 2024-04-24T22:45:26.556 INFO:tasks.workunit.client.0.smithi063.stdout:3/465: stat da/d18/d47/l50 0 2024-04-24T22:45:26.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/349: rename d3/dd/de/d17/f27 to d3/dd/de/f7a 0 2024-04-24T22:45:26.559 INFO:tasks.workunit.client.1.smithi192.stdout:6/323: mknod d0/d8/d18/d2e/c77 0 2024-04-24T22:45:26.560 INFO:tasks.workunit.client.1.smithi192.stdout:7/352: write d2/f2c [1750183,123984] 0 2024-04-24T22:45:26.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/324: write d0/d8/d29/d4c/f74 [385791,63840] 0 2024-04-24T22:45:26.565 INFO:tasks.workunit.client.1.smithi192.stdout:4/295: mkdir d2/d9/d6b 0 2024-04-24T22:45:26.570 INFO:tasks.workunit.client.1.smithi192.stdout:2/451: write d4/d9/f52 [1422001,43278] 0 2024-04-24T22:45:26.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/350: mknod d3/d59/c7b 0 2024-04-24T22:45:26.574 INFO:tasks.workunit.client.1.smithi192.stdout:9/351: read - d3/dd/d62/f6b zero size 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.1.smithi192.stdout:5/281: dwrite d0/d13/d19/f1a [0,4194304] 0 2024-04-24T22:45:26.575 INFO:tasks.workunit.client.1.smithi192.stdout:7/353: creat d2/db/d49/f70 x:0 0 0 2024-04-24T22:45:26.576 INFO:tasks.workunit.client.1.smithi192.stdout:5/282: chown d0/d1b/d1c/f3e 735 1 2024-04-24T22:45:26.578 INFO:tasks.workunit.client.0.smithi063.stdout:6/461: creat db/d1d/d20/d84/f9b x:0 0 0 2024-04-24T22:45:26.579 INFO:tasks.workunit.client.1.smithi192.stdout:6/325: write d0/d8/f17 [2270514,90928] 0 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.1.smithi192.stdout:5/283: write d0/d1b/f45 [672766,61908] 0 2024-04-24T22:45:26.581 INFO:tasks.workunit.client.1.smithi192.stdout:2/452: rename d4/d9/d10/f53 to d4/d9/d37/f8d 0 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.1.smithi192.stdout:7/354: mknod d2/db/d21/d42/d67/c71 0 2024-04-24T22:45:26.584 INFO:tasks.workunit.client.1.smithi192.stdout:2/453: dread - d4/d29/f87 zero size 2024-04-24T22:45:26.587 INFO:tasks.workunit.client.1.smithi192.stdout:9/352: dwrite d3/dd/de/f19 [0,4194304] 0 2024-04-24T22:45:26.589 INFO:tasks.workunit.client.1.smithi192.stdout:5/284: rmdir d0/d13/d3b 39 2024-04-24T22:45:26.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/425: rename d1/d70 to d1/d11/d8b 0 2024-04-24T22:45:26.594 INFO:tasks.workunit.client.1.smithi192.stdout:6/326: mkdir d0/d8/d3d/d3e/d78 0 2024-04-24T22:45:26.594 INFO:tasks.workunit.client.0.smithi063.stdout:7/346: sync 2024-04-24T22:45:26.595 INFO:tasks.workunit.client.0.smithi063.stdout:1/471: sync 2024-04-24T22:45:26.595 INFO:tasks.workunit.client.1.smithi192.stdout:6/327: read - d0/d8/d29/d5a/f6b zero size 2024-04-24T22:45:26.595 INFO:tasks.workunit.client.0.smithi063.stdout:1/472: read - d3/da/d34/f9a zero size 2024-04-24T22:45:26.596 INFO:tasks.workunit.client.0.smithi063.stdout:2/472: creat d0/d2/d2d/d31/f92 x:0 0 0 2024-04-24T22:45:26.597 INFO:tasks.workunit.client.1.smithi192.stdout:2/454: creat d4/d9/d10/d6c/f8e x:0 0 0 2024-04-24T22:45:26.597 INFO:tasks.workunit.client.0.smithi063.stdout:1/473: read - d3/da/dc/d39/d5a/f8f zero size 2024-04-24T22:45:26.597 INFO:tasks.workunit.client.0.smithi063.stdout:2/473: write d0/dd/d62/d7c/f61 [1563050,34065] 0 2024-04-24T22:45:26.599 INFO:tasks.workunit.client.1.smithi192.stdout:9/353: rmdir d3/dd/de/d39 39 2024-04-24T22:45:26.602 INFO:tasks.workunit.client.1.smithi192.stdout:7/355: symlink d2/db/l72 0 2024-04-24T22:45:26.602 INFO:tasks.workunit.client.0.smithi063.stdout:3/466: getdents da 0 2024-04-24T22:45:26.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/426: readlink d1/d11/d32/d38/l5f 0 2024-04-24T22:45:26.604 INFO:tasks.workunit.client.0.smithi063.stdout:8/427: fsync d1/d11/d49/f7c 0 2024-04-24T22:45:26.604 INFO:tasks.workunit.client.0.smithi063.stdout:3/467: read da/fc [823602,56516] 0 2024-04-24T22:45:26.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/428: truncate d1/d21/f83 1042515 0 2024-04-24T22:45:26.605 INFO:tasks.workunit.client.0.smithi063.stdout:7/347: creat d0/d4/de/d17/d35/d69/f7b x:0 0 0 2024-04-24T22:45:26.608 INFO:tasks.workunit.client.0.smithi063.stdout:5/418: truncate d5/d24/d44/f5c 265479 0 2024-04-24T22:45:26.610 INFO:tasks.workunit.client.0.smithi063.stdout:1/474: chown d3/c1d 2634 1 2024-04-24T22:45:26.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/429: dwrite d1/f2c [0,4194304] 0 2024-04-24T22:45:26.612 INFO:tasks.workunit.client.1.smithi192.stdout:7/356: creat d2/db/d36/f73 x:0 0 0 2024-04-24T22:45:26.612 INFO:tasks.workunit.client.1.smithi192.stdout:2/455: dwrite d4/d29/d60/f63 [0,4194304] 0 2024-04-24T22:45:26.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/462: write db/f2a [343513,74905] 0 2024-04-24T22:45:26.613 INFO:tasks.workunit.client.1.smithi192.stdout:7/357: dread - d2/db/f6f zero size 2024-04-24T22:45:26.614 INFO:tasks.workunit.client.0.smithi063.stdout:2/474: dread - d0/d2/d27/d3a/f84 zero size 2024-04-24T22:45:26.617 INFO:tasks.workunit.client.0.smithi063.stdout:1/475: dread d3/da/dc/d11/f46 [0,4194304] 0 2024-04-24T22:45:26.619 INFO:tasks.workunit.client.1.smithi192.stdout:5/285: creat d0/d1b/d1c/d2c/d3c/f5a x:0 0 0 2024-04-24T22:45:26.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/328: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:26.623 INFO:tasks.workunit.client.0.smithi063.stdout:2/475: dwrite d0/d2/d2d/d31/d89/d1c/d3f/f76 [0,4194304] 0 2024-04-24T22:45:26.623 INFO:tasks.workunit.client.1.smithi192.stdout:9/354: link d3/dd/f3a d3/d5d/f7c 0 2024-04-24T22:45:26.624 INFO:tasks.workunit.client.1.smithi192.stdout:2/456: dread d4/f31 [0,4194304] 0 2024-04-24T22:45:26.625 INFO:tasks.workunit.client.1.smithi192.stdout:9/355: chown d3/dd/c77 1293 1 2024-04-24T22:45:26.625 INFO:tasks.workunit.client.0.smithi063.stdout:3/468: creat da/fa2 x:0 0 0 2024-04-24T22:45:26.626 INFO:tasks.workunit.client.1.smithi192.stdout:2/457: truncate d4/d9/d37/f8a 1901 0 2024-04-24T22:45:26.627 INFO:tasks.workunit.client.1.smithi192.stdout:2/458: read d4/f69 [1502128,48678] 0 2024-04-24T22:45:26.630 INFO:tasks.workunit.client.0.smithi063.stdout:8/430: dread d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:26.634 INFO:tasks.workunit.client.1.smithi192.stdout:9/356: mkdir d3/d59/d7d 0 2024-04-24T22:45:26.637 INFO:tasks.workunit.client.0.smithi063.stdout:1/476: mkdir d3/da/dc/d11/d5e/d9c 0 2024-04-24T22:45:26.638 INFO:tasks.workunit.client.1.smithi192.stdout:6/329: dwrite d0/d8/f14 [0,4194304] 0 2024-04-24T22:45:26.639 INFO:tasks.workunit.client.1.smithi192.stdout:2/459: fsync d4/d29/d60/f47 0 2024-04-24T22:45:26.642 INFO:tasks.workunit.client.0.smithi063.stdout:2/476: mknod d0/dd/d72/c93 0 2024-04-24T22:45:26.643 INFO:tasks.workunit.client.1.smithi192.stdout:5/286: mkdir d0/d5b 0 2024-04-24T22:45:26.648 INFO:tasks.workunit.client.1.smithi192.stdout:9/357: creat d3/d24/f7e x:0 0 0 2024-04-24T22:45:26.651 INFO:tasks.workunit.client.0.smithi063.stdout:9/426: sync 2024-04-24T22:45:26.651 INFO:tasks.workunit.client.0.smithi063.stdout:0/373: sync 2024-04-24T22:45:26.651 INFO:tasks.workunit.client.0.smithi063.stdout:4/386: sync 2024-04-24T22:45:26.651 INFO:tasks.workunit.client.0.smithi063.stdout:4/387: chown d2/f1c 45681 1 2024-04-24T22:45:26.655 INFO:tasks.workunit.client.0.smithi063.stdout:8/431: creat d1/d11/d32/d38/f8c x:0 0 0 2024-04-24T22:45:26.656 INFO:tasks.workunit.client.1.smithi192.stdout:5/287: creat d0/d1b/d1c/d2c/f5c x:0 0 0 2024-04-24T22:45:26.659 INFO:tasks.workunit.client.1.smithi192.stdout:9/358: creat d3/dd/d53/f7f x:0 0 0 2024-04-24T22:45:26.660 INFO:tasks.workunit.client.0.smithi063.stdout:6/463: write db/d1d/d20/f2b [1393620,90793] 0 2024-04-24T22:45:26.662 INFO:tasks.workunit.client.1.smithi192.stdout:7/358: dwrite d2/db/d21/f22 [4194304,4194304] 0 2024-04-24T22:45:26.663 INFO:tasks.workunit.client.1.smithi192.stdout:7/359: dread - d2/db/d25/f69 zero size 2024-04-24T22:45:26.664 INFO:tasks.workunit.client.0.smithi063.stdout:7/348: creat d0/d4/de/f7c x:0 0 0 2024-04-24T22:45:26.665 INFO:tasks.workunit.client.1.smithi192.stdout:5/288: fsync d0/d1b/f37 0 2024-04-24T22:45:26.667 INFO:tasks.workunit.client.0.smithi063.stdout:7/349: read d0/f7 [522140,37195] 0 2024-04-24T22:45:26.668 INFO:tasks.workunit.client.0.smithi063.stdout:7/350: fsync d0/f73 0 2024-04-24T22:45:26.668 INFO:tasks.workunit.client.1.smithi192.stdout:7/360: read d2/f2c [3089225,5067] 0 2024-04-24T22:45:26.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/359: read - d3/dd/de/d39/f70 zero size 2024-04-24T22:45:26.669 INFO:tasks.workunit.client.1.smithi192.stdout:7/361: write d2/db/f20 [11564317,12529] 0 2024-04-24T22:45:26.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/360: dread - d3/dd/de/d17/f68 zero size 2024-04-24T22:45:26.671 INFO:tasks.workunit.client.1.smithi192.stdout:7/362: fsync d2/db/d25/f69 0 2024-04-24T22:45:26.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/361: chown d3/dd/de/d17/d33 0 1 2024-04-24T22:45:26.672 INFO:tasks.workunit.client.0.smithi063.stdout:9/427: symlink d6/d2f/d42/l8b 0 2024-04-24T22:45:26.672 INFO:tasks.workunit.client.1.smithi192.stdout:7/363: chown f0 165571 1 2024-04-24T22:45:26.672 INFO:tasks.workunit.client.1.smithi192.stdout:9/362: readlink d3/dd/de/d17/l67 0 2024-04-24T22:45:26.673 INFO:tasks.workunit.client.0.smithi063.stdout:9/428: stat d6/d8/db/d6b/d4f/f79 0 2024-04-24T22:45:26.675 INFO:tasks.workunit.client.0.smithi063.stdout:5/419: write d5/d11/d23/d2d/d62/d73/f74 [141346,83987] 0 2024-04-24T22:45:26.675 INFO:tasks.workunit.client.0.smithi063.stdout:0/374: read d6/dd/f4d [1700552,998] 0 2024-04-24T22:45:26.679 INFO:tasks.workunit.client.1.smithi192.stdout:6/330: getdents d0/d8/d22 0 2024-04-24T22:45:26.680 INFO:tasks.workunit.client.1.smithi192.stdout:6/331: dread - d0/d8/d3d/f5f zero size 2024-04-24T22:45:26.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/351: unlink d0/d4/de/d6f/d5c/d63/l67 0 2024-04-24T22:45:26.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/352: stat d0/d4/de/f10 0 2024-04-24T22:45:26.686 INFO:tasks.workunit.client.1.smithi192.stdout:7/364: mknod d2/db/d21/d42/d5b/c74 0 2024-04-24T22:45:26.688 INFO:tasks.workunit.client.1.smithi192.stdout:9/363: truncate d3/dd/de/f1a 236129 0 2024-04-24T22:45:26.690 INFO:tasks.workunit.client.1.smithi192.stdout:5/289: truncate d0/d1b/f25 247849 0 2024-04-24T22:45:26.691 INFO:tasks.workunit.client.1.smithi192.stdout:5/290: readlink d0/d13/d19/l4c 0 2024-04-24T22:45:26.691 INFO:tasks.workunit.client.0.smithi063.stdout:5/420: symlink d5/d24/d36/l98 0 2024-04-24T22:45:26.692 INFO:tasks.workunit.client.1.smithi192.stdout:6/332: rename d0/d8/d3d/d3e/d78 to d0/d8/d18/d79 0 2024-04-24T22:45:26.693 INFO:tasks.workunit.client.0.smithi063.stdout:5/421: readlink d5/d1d/l56 0 2024-04-24T22:45:26.693 INFO:tasks.workunit.client.0.smithi063.stdout:4/388: dwrite d2/d8/d5c/f67 [0,4194304] 0 2024-04-24T22:45:26.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/364: truncate d3/d24/f4e 1731230 0 2024-04-24T22:45:26.698 INFO:tasks.workunit.client.1.smithi192.stdout:9/365: readlink d3/dd/de/d17/d6a/l6e 0 2024-04-24T22:45:26.699 INFO:tasks.workunit.client.0.smithi063.stdout:8/432: link d1/d22/f5b d1/d11/d8b/f8d 0 2024-04-24T22:45:26.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/422: dwrite d5/d11/d23/f4e [0,4194304] 0 2024-04-24T22:45:26.700 INFO:tasks.workunit.client.0.smithi063.stdout:8/433: fdatasync d1/d11/d49/f72 0 2024-04-24T22:45:26.702 INFO:tasks.workunit.client.0.smithi063.stdout:1/477: creat d3/da/dc/d1f/d2b/d79/d96/f9d x:0 0 0 2024-04-24T22:45:26.703 INFO:tasks.workunit.client.0.smithi063.stdout:1/478: fsync d3/da/dc/d39/f66 0 2024-04-24T22:45:26.705 INFO:tasks.workunit.client.0.smithi063.stdout:7/353: dread d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:26.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/479: stat d3/da/dc/d39/c95 0 2024-04-24T22:45:26.707 INFO:tasks.workunit.client.0.smithi063.stdout:9/429: getdents d6/d8/d8a 0 2024-04-24T22:45:26.707 INFO:tasks.workunit.client.0.smithi063.stdout:7/354: fsync d0/d4/de/d17/d28/d31/f6a 0 2024-04-24T22:45:26.707 INFO:tasks.workunit.client.1.smithi192.stdout:6/333: mknod d0/d8/d18/c7a 0 2024-04-24T22:45:26.709 INFO:tasks.workunit.client.1.smithi192.stdout:6/334: write d0/d8/d3d/f49 [41027,108937] 0 2024-04-24T22:45:26.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/469: mkdir da/d18/d9e/da3 0 2024-04-24T22:45:26.714 INFO:tasks.workunit.client.0.smithi063.stdout:7/355: dwrite d0/d4/de/d6f/f5b [0,4194304] 0 2024-04-24T22:45:26.714 INFO:tasks.workunit.client.0.smithi063.stdout:3/470: write da/d18/d47/f6c [380113,129216] 0 2024-04-24T22:45:26.716 INFO:tasks.workunit.client.0.smithi063.stdout:7/356: fsync d0/d4/de/d17/d35/d69/f7b 0 2024-04-24T22:45:26.718 INFO:tasks.workunit.client.1.smithi192.stdout:5/291: symlink d0/d13/d3b/l5d 0 2024-04-24T22:45:26.720 INFO:tasks.workunit.client.0.smithi063.stdout:8/434: creat d1/d11/d49/d54/f8e x:0 0 0 2024-04-24T22:45:26.723 INFO:tasks.workunit.client.0.smithi063.stdout:3/471: dwrite da/d18/d47/f6a [0,4194304] 0 2024-04-24T22:45:26.723 INFO:tasks.workunit.client.1.smithi192.stdout:6/335: mkdir d0/d8/d29/d4c/d7b 0 2024-04-24T22:45:26.725 INFO:tasks.workunit.client.1.smithi192.stdout:9/366: symlink d3/dd/de/d17/l80 0 2024-04-24T22:45:26.725 INFO:tasks.workunit.client.1.smithi192.stdout:6/336: read d0/d8/d18/d2e/f51 [2194305,18656] 0 2024-04-24T22:45:26.730 INFO:tasks.workunit.client.1.smithi192.stdout:7/365: getdents d2/db/d21/d42/d5b 0 2024-04-24T22:45:26.735 INFO:tasks.workunit.client.1.smithi192.stdout:7/366: dread d2/db/d21/f22 [4194304,4194304] 0 2024-04-24T22:45:26.736 INFO:tasks.workunit.client.1.smithi192.stdout:9/367: unlink d3/d24/f52 0 2024-04-24T22:45:26.737 INFO:tasks.workunit.client.1.smithi192.stdout:9/368: dread - d3/dd/d53/d60/f74 zero size 2024-04-24T22:45:26.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/375: link d6/da/f3d d6/da/dc/d26/f83 0 2024-04-24T22:45:26.741 INFO:tasks.workunit.client.1.smithi192.stdout:5/292: fdatasync d0/d1b/d1c/f20 0 2024-04-24T22:45:26.747 INFO:tasks.workunit.client.1.smithi192.stdout:7/367: creat d2/db/d21/d42/d5b/f75 x:0 0 0 2024-04-24T22:45:26.748 INFO:tasks.workunit.client.0.smithi063.stdout:3/472: rmdir da/d18/d3a/d63 39 2024-04-24T22:45:26.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/430: dread d6/d8/f5b [0,4194304] 0 2024-04-24T22:45:26.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/480: link d3/da/dc/d1f/d2b/f32 d3/da/dc/d1f/d21/d49/f9e 0 2024-04-24T22:45:26.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/389: link d2/d8/d3c/f75 d2/d8/d36/d3a/f7a 0 2024-04-24T22:45:26.752 INFO:tasks.workunit.client.0.smithi063.stdout:8/435: dwrite d1/d2/fd [4194304,4194304] 0 2024-04-24T22:45:26.753 INFO:tasks.workunit.client.0.smithi063.stdout:7/357: symlink d0/d4/de/d6f/l7d 0 2024-04-24T22:45:26.754 INFO:tasks.workunit.client.0.smithi063.stdout:8/436: dread d1/d21/f61 [0,4194304] 0 2024-04-24T22:45:26.754 INFO:tasks.workunit.client.0.smithi063.stdout:5/423: rmdir d5/d11/d23/d83 0 2024-04-24T22:45:26.755 INFO:tasks.workunit.client.1.smithi192.stdout:7/368: chown d2/db/l1f 1 1 2024-04-24T22:45:26.757 INFO:tasks.workunit.client.1.smithi192.stdout:6/337: rmdir d0/d8/d18/d79 0 2024-04-24T22:45:26.758 INFO:tasks.workunit.client.0.smithi063.stdout:8/437: dwrite d1/d11/d32/d38/f67 [0,4194304] 0 2024-04-24T22:45:26.759 INFO:tasks.workunit.client.0.smithi063.stdout:9/431: fsync d6/d8/d1f/d20/d22/f6d 0 2024-04-24T22:45:26.760 INFO:tasks.workunit.client.0.smithi063.stdout:8/438: chown d1/d22/d33/c34 19 1 2024-04-24T22:45:26.767 INFO:tasks.workunit.client.1.smithi192.stdout:7/369: fsync d2/db/d21/d42/f4d 0 2024-04-24T22:45:26.769 INFO:tasks.workunit.client.1.smithi192.stdout:7/370: fsync d2/db/f15 0 2024-04-24T22:45:26.771 INFO:tasks.workunit.client.1.smithi192.stdout:9/369: getdents d3 0 2024-04-24T22:45:26.773 INFO:tasks.workunit.client.1.smithi192.stdout:5/293: dwrite d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:26.774 INFO:tasks.workunit.client.1.smithi192.stdout:5/294: fsync d0/d1b/f45 0 2024-04-24T22:45:26.779 INFO:tasks.workunit.client.1.smithi192.stdout:9/370: dwrite d3/dd/de/d17/d6a/f2e [0,4194304] 0 2024-04-24T22:45:26.784 INFO:tasks.workunit.client.1.smithi192.stdout:9/371: dread d3/dd/de/d17/f23 [0,4194304] 0 2024-04-24T22:45:26.785 INFO:tasks.workunit.client.0.smithi063.stdout:4/390: mknod d2/c7b 0 2024-04-24T22:45:26.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.816+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8764000bf0 con 0x7f876006b7e0 2024-04-24T22:45:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.817+0000 7f875f7fe700 1 -- 172.21.15.63:0/472452401 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8764000bf0 con 0x7f876006b7e0 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 msgr2=0x7f876006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 0x7f876006dc90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8768004f90 tx=0x7f8768009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 msgr2=0x7f8778114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f876c002a50 tx=0x7f876c0046b0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 shutdown_connections 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f876006b7e0 0x7f876006dc90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87780fbcb0 0x7f8778114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.819+0000 7f877d5e3700 1 --2- 172.21.15.63:0/472452401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87780fcf60 0x7f8778114570 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.820+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 >> 172.21.15.63:0/472452401 conn(0x7f87780f7430 msgr2=0x7f87780f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.820+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 shutdown_connections 2024-04-24T22:45:26.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.820+0000 7f877d5e3700 1 -- 172.21.15.63:0/472452401 wait complete. 2024-04-24T22:45:26.829 INFO:tasks.workunit.client.0.smithi063.stdout:6/464: sync 2024-04-24T22:45:26.829 INFO:tasks.workunit.client.0.smithi063.stdout:2/477: sync 2024-04-24T22:45:26.830 INFO:tasks.workunit.client.0.smithi063.stdout:6/465: write db/f37 [4539012,117483] 0 2024-04-24T22:45:26.830 INFO:tasks.workunit.client.0.smithi063.stdout:2/478: write d0/d2/d2d/f4d [720832,56665] 0 2024-04-24T22:45:26.833 INFO:tasks.workunit.client.0.smithi063.stdout:2/479: write d0/d2/d2d/d31/d89/d48/f5d [721052,111426] 0 2024-04-24T22:45:26.838 INFO:tasks.workunit.client.0.smithi063.stdout:2/480: dwrite d0/d2/d2d/f4d [0,4194304] 0 2024-04-24T22:45:26.846 INFO:tasks.workunit.client.0.smithi063.stdout:2/481: dwrite d0/d2/d35/f82 [0,4194304] 0 2024-04-24T22:45:26.847 INFO:tasks.workunit.client.0.smithi063.stdout:2/482: dread - d0/d2/d2d/d31/d89/d1c/d3f/f7d zero size 2024-04-24T22:45:26.848 INFO:tasks.workunit.client.0.smithi063.stdout:2/483: chown d0/d2/d35/l5f 64641531 1 2024-04-24T22:45:26.855 INFO:tasks.workunit.client.1.smithi192.stdout:5/295: rmdir d0/d1b/d1c/d2c 39 2024-04-24T22:45:26.857 INFO:tasks.workunit.client.0.smithi063.stdout:0/376: creat d6/da/dc/d26/f84 x:0 0 0 2024-04-24T22:45:26.862 INFO:tasks.workunit.client.1.smithi192.stdout:6/338: rename d0/d8/d18/f26 to d0/d8/d29/f7c 0 2024-04-24T22:45:26.863 INFO:tasks.workunit.client.0.smithi063.stdout:9/432: write d6/d8/f17 [2134802,23831] 0 2024-04-24T22:45:26.864 INFO:tasks.workunit.client.0.smithi063.stdout:1/481: mkdir d3/da/dc/d1f/d21/d9f 0 2024-04-24T22:45:26.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/339: write d0/d8/d3d/f5f [832488,43442] 0 2024-04-24T22:45:26.865 INFO:tasks.workunit.client.1.smithi192.stdout:6/340: chown d0/d8/d3d/f62 564 1 2024-04-24T22:45:26.865 INFO:tasks.workunit.client.0.smithi063.stdout:9/433: write d6/d8/db/d6b/f23 [319646,103835] 0 2024-04-24T22:45:26.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/358: rename d0/d4/de/c49 to d0/d4/de/d17/d28/d31/c7e 0 2024-04-24T22:45:26.875 INFO:tasks.workunit.client.0.smithi063.stdout:4/391: rmdir d2/d8/d3c/d6f 39 2024-04-24T22:45:26.881 INFO:tasks.workunit.client.0.smithi063.stdout:8/439: dwrite d1/d2/d17/f4b [0,4194304] 0 2024-04-24T22:45:26.883 INFO:tasks.workunit.client.0.smithi063.stdout:1/482: dwrite d3/da/dc/d11/f41 [0,4194304] 0 2024-04-24T22:45:26.883 INFO:tasks.workunit.client.0.smithi063.stdout:9/434: dwrite d6/d8/db/d6b/f1e [0,4194304] 0 2024-04-24T22:45:26.886 INFO:tasks.workunit.client.0.smithi063.stdout:9/435: fsync d6/d8/d1f/f61 0 2024-04-24T22:45:26.887 INFO:tasks.workunit.client.1.smithi192.stdout:7/371: rename d2/db/l72 to d2/db/d2a/d57/l76 0 2024-04-24T22:45:26.889 INFO:tasks.workunit.client.0.smithi063.stdout:5/424: sync 2024-04-24T22:45:26.890 INFO:tasks.workunit.client.1.smithi192.stdout:6/341: mknod d0/d8/d29/d58/c7d 0 2024-04-24T22:45:26.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/359: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:26.896 INFO:tasks.workunit.client.1.smithi192.stdout:9/372: link d3/dd/d53/f7f d3/dd/de/d17/d6a/f81 0 2024-04-24T22:45:26.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/473: creat da/d39/fa4 x:0 0 0 2024-04-24T22:45:26.901 INFO:tasks.workunit.client.0.smithi063.stdout:6/466: symlink db/d1d/d20/l9c 0 2024-04-24T22:45:26.901 INFO:tasks.workunit.client.0.smithi063.stdout:2/484: dwrite d0/d2/d2f/f6d [4194304,4194304] 0 2024-04-24T22:45:26.902 INFO:tasks.workunit.client.1.smithi192.stdout:9/373: creat d3/dd/de/d17/d33/f82 x:0 0 0 2024-04-24T22:45:26.903 INFO:tasks.workunit.client.0.smithi063.stdout:8/440: mknod d1/d21/c8f 0 2024-04-24T22:45:26.904 INFO:tasks.workunit.client.1.smithi192.stdout:7/372: mkdir d2/db/d3a/d6e/d77 0 2024-04-24T22:45:26.905 INFO:tasks.workunit.client.0.smithi063.stdout:8/441: write d1/d11/d3c/f55 [1513762,91617] 0 2024-04-24T22:45:26.907 INFO:tasks.workunit.client.0.smithi063.stdout:8/442: write d1/d11/d32/d38/f67 [2202444,4065] 0 2024-04-24T22:45:26.907 INFO:tasks.workunit.client.1.smithi192.stdout:6/342: mkdir d0/d8/d3d/d3e/d7e 0 2024-04-24T22:45:26.908 INFO:tasks.workunit.client.1.smithi192.stdout:6/343: dread - d0/f71 zero size 2024-04-24T22:45:26.911 INFO:tasks.workunit.client.1.smithi192.stdout:9/374: rename d3/dd/de/d39/l47 to d3/dd/d62/d6d/l83 0 2024-04-24T22:45:26.913 INFO:tasks.workunit.client.1.smithi192.stdout:7/373: dwrite d2/db/d2a/f61 [0,4194304] 0 2024-04-24T22:45:26.918 INFO:tasks.workunit.client.0.smithi063.stdout:3/474: creat da/d18/d9e/fa5 x:0 0 0 2024-04-24T22:45:26.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/344: unlink d0/c1f 0 2024-04-24T22:45:26.922 INFO:tasks.workunit.client.1.smithi192.stdout:5/296: dwrite d0/f18 [0,4194304] 0 2024-04-24T22:45:26.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/375: mknod d3/dd/d62/c84 0 2024-04-24T22:45:26.924 INFO:tasks.workunit.client.1.smithi192.stdout:9/376: chown d3/d59/c7b 7072 1 2024-04-24T22:45:26.924 INFO:tasks.workunit.client.0.smithi063.stdout:6/467: symlink db/d1c/l9d 0 2024-04-24T22:45:26.925 INFO:tasks.workunit.client.1.smithi192.stdout:9/377: dread - d3/dd/de/f51 zero size 2024-04-24T22:45:26.926 INFO:tasks.workunit.client.1.smithi192.stdout:7/374: mknod d2/db/d21/d42/d67/c78 0 2024-04-24T22:45:26.927 INFO:tasks.workunit.client.1.smithi192.stdout:7/375: fdatasync d2/db/d49/f70 0 2024-04-24T22:45:26.928 INFO:tasks.workunit.client.0.smithi063.stdout:1/483: write d3/da/dc/d1f/d21/d49/f9e [1381840,52497] 0 2024-04-24T22:45:26.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/297: dread d0/d1b/f24 [0,4194304] 0 2024-04-24T22:45:26.933 INFO:tasks.workunit.client.1.smithi192.stdout:9/378: mkdir d3/dd/de/d39/d85 0 2024-04-24T22:45:26.936 INFO:tasks.workunit.client.0.smithi063.stdout:8/443: readlink d1/l19 0 2024-04-24T22:45:26.937 INFO:tasks.workunit.client.0.smithi063.stdout:8/444: dread - d1/d11/d49/d54/f8e zero size 2024-04-24T22:45:26.938 INFO:tasks.workunit.client.0.smithi063.stdout:9/436: mkdir d6/d8/db/d1c/d78/d8c 0 2024-04-24T22:45:26.940 INFO:tasks.workunit.client.1.smithi192.stdout:5/298: mknod d0/d13/d53/c5e 0 2024-04-24T22:45:26.941 INFO:tasks.workunit.client.1.smithi192.stdout:5/299: fsync d0/d13/d19/f4b 0 2024-04-24T22:45:26.941 INFO:tasks.workunit.client.0.smithi063.stdout:7/360: symlink d0/d4/de/d17/l7f 0 2024-04-24T22:45:26.941 INFO:tasks.workunit.client.0.smithi063.stdout:7/361: chown d0/c27 666 1 2024-04-24T22:45:26.942 INFO:tasks.workunit.client.1.smithi192.stdout:9/379: creat d3/dd/d62/d6d/f86 x:0 0 0 2024-04-24T22:45:26.942 INFO:tasks.workunit.client.1.smithi192.stdout:9/380: chown d3/f43 134347852 1 2024-04-24T22:45:26.942 INFO:tasks.workunit.client.1.smithi192.stdout:7/376: truncate f0 245976 0 2024-04-24T22:45:26.943 INFO:tasks.workunit.client.1.smithi192.stdout:9/381: write d3/dd/de/d17/d6a/f2e [3805589,62714] 0 2024-04-24T22:45:26.945 INFO:tasks.workunit.client.0.smithi063.stdout:4/392: link d2/f2e d2/f7c 0 2024-04-24T22:45:26.946 INFO:tasks.workunit.client.1.smithi192.stdout:9/382: write d3/dd/d62/d6d/f86 [552244,31807] 0 2024-04-24T22:45:26.946 INFO:tasks.workunit.client.1.smithi192.stdout:6/345: creat d0/f7f x:0 0 0 2024-04-24T22:45:26.947 INFO:tasks.workunit.client.0.smithi063.stdout:6/468: rename db/d1d/d20/d84/f9b to db/d1d/d20/d84/f9e 0 2024-04-24T22:45:26.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/393: chown d2/d8/d5c/f67 1985044 1 2024-04-24T22:45:26.949 INFO:tasks.workunit.client.0.smithi063.stdout:0/377: getdents d6/d3a 0 2024-04-24T22:45:26.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/362: dwrite d0/d4/de/d17/d35/d69/f7b [0,4194304] 0 2024-04-24T22:45:26.956 INFO:tasks.workunit.client.0.smithi063.stdout:1/484: truncate d3/da/dc/f3a 1072954 0 2024-04-24T22:45:26.956 INFO:tasks.workunit.client.1.smithi192.stdout:7/377: truncate d2/db/d21/f28 5599833 0 2024-04-24T22:45:26.956 INFO:tasks.workunit.client.1.smithi192.stdout:5/300: rename d0/d1b/d1c to d0/d13/d3b/d2f/d34/d5f 0 2024-04-24T22:45:26.957 INFO:tasks.workunit.client.1.smithi192.stdout:7/378: dread d2/db/f15 [0,4194304] 0 2024-04-24T22:45:26.957 INFO:tasks.workunit.client.1.smithi192.stdout:9/383: mknod d3/dd/de/d17/c87 0 2024-04-24T22:45:26.957 INFO:tasks.workunit.client.1.smithi192.stdout:7/379: write d2/db/d49/f70 [268307,40986] 0 2024-04-24T22:45:26.957 INFO:tasks.workunit.client.1.smithi192.stdout:9/384: chown d3/f43 56415001 1 2024-04-24T22:45:26.958 INFO:tasks.workunit.client.1.smithi192.stdout:7/380: write d2/db/d21/f39 [73585,30079] 0 2024-04-24T22:45:26.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/475: mknod da/d18/d3a/ca6 0 2024-04-24T22:45:26.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/425: dwrite d5/d11/f6a [0,4194304] 0 2024-04-24T22:45:26.961 INFO:tasks.workunit.client.1.smithi192.stdout:5/301: creat d0/d13/d19/f60 x:0 0 0 2024-04-24T22:45:26.962 INFO:tasks.workunit.client.1.smithi192.stdout:5/302: chown d0/d13/d3b/d2f 195062 1 2024-04-24T22:45:26.962 INFO:tasks.workunit.client.0.smithi063.stdout:3/476: rename da to da/d18/d47/d4d/d62/da7 22 2024-04-24T22:45:26.965 INFO:tasks.workunit.client.0.smithi063.stdout:1/485: dread d3/da/dc/d1f/f4d [4194304,4194304] 0 2024-04-24T22:45:26.967 INFO:tasks.workunit.client.0.smithi063.stdout:1/486: chown d3/da/dc/d11/d5e/d9c 1592 1 2024-04-24T22:45:26.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/394: creat d2/d8/d36/f7d x:0 0 0 2024-04-24T22:45:26.969 INFO:tasks.workunit.client.1.smithi192.stdout:7/381: unlink d2/f6 0 2024-04-24T22:45:26.969 INFO:tasks.workunit.client.0.smithi063.stdout:1/487: chown d3/da/dc/d11/d5e/c7c 315 1 2024-04-24T22:45:26.969 INFO:tasks.workunit.client.1.smithi192.stdout:7/382: chown d2/db/d2a/f3c 2147 1 2024-04-24T22:45:26.971 INFO:tasks.workunit.client.0.smithi063.stdout:6/469: creat db/d1d/d20/d29/f9f x:0 0 0 2024-04-24T22:45:26.972 INFO:tasks.workunit.client.1.smithi192.stdout:5/303: rename d0/d13/d19/f48 to d0/d13/d3b/d2f/d34/d4f/d52/f61 0 2024-04-24T22:45:26.973 INFO:tasks.workunit.client.1.smithi192.stdout:5/304: dread - d0/d13/d19/f4b zero size 2024-04-24T22:45:26.973 INFO:tasks.workunit.client.1.smithi192.stdout:5/305: fsync d0/f31 0 2024-04-24T22:45:26.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/378: dread d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:26.974 INFO:tasks.workunit.client.0.smithi063.stdout:9/437: mknod d6/d8/d1f/d20/c8d 0 2024-04-24T22:45:26.974 INFO:tasks.workunit.client.1.smithi192.stdout:5/306: stat d0/d13/d3b/d2f/d34 0 2024-04-24T22:45:26.974 INFO:tasks.workunit.client.1.smithi192.stdout:9/385: mkdir d3/d59/d7d/d88 0 2024-04-24T22:45:26.976 INFO:tasks.workunit.client.1.smithi192.stdout:5/307: stat d0/d13/d3b/d2f/d34/d4f/d52 0 2024-04-24T22:45:26.977 INFO:tasks.workunit.client.0.smithi063.stdout:8/445: mknod d1/d11/d49/d54/d79/c90 0 2024-04-24T22:45:26.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/308: chown d0/c21 4160 1 2024-04-24T22:45:26.980 INFO:tasks.workunit.client.0.smithi063.stdout:1/488: dwrite d3/da/dc/d1f/f71 [0,4194304] 0 2024-04-24T22:45:26.980 INFO:tasks.workunit.client.0.smithi063.stdout:8/446: chown d1/d11/d49/f72 5135014 1 2024-04-24T22:45:26.984 INFO:tasks.workunit.client.0.smithi063.stdout:3/477: stat da/f15 0 2024-04-24T22:45:26.990 INFO:tasks.workunit.client.1.smithi192.stdout:5/309: chown d0/d13/d3b/d2f/d34/d5f/d2c/f5c 45 1 2024-04-24T22:45:26.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/310: fdatasync d0/d13/d3b/d2f/d34/d5f/f3e 0 2024-04-24T22:45:26.992 INFO:tasks.workunit.client.0.smithi063.stdout:6/470: dread db/f37 [0,4194304] 0 2024-04-24T22:45:26.993 INFO:tasks.workunit.client.1.smithi192.stdout:7/383: mknod d2/db/d3a/d6e/d77/c79 0 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.991+0000 7f2d150c7700 1 -- 172.21.15.63:0/2558876026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 msgr2=0x7f2d100fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.991+0000 7f2d150c7700 1 --2- 172.21.15.63:0/2558876026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d100fd2a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2d00004940 tx=0x7f2d00009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 -- 172.21.15.63:0/2558876026 shutdown_connections 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 --2- 172.21.15.63:0/2558876026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 0x7f2d100fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 --2- 172.21.15.63:0/2558876026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d100fd2a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 -- 172.21.15.63:0/2558876026 >> 172.21.15.63:0/2558876026 conn(0x7f2d100f85c0 msgr2=0x7f2d100fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 -- 172.21.15.63:0/2558876026 shutdown_connections 2024-04-24T22:45:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.992+0000 7f2d150c7700 1 -- 172.21.15.63:0/2558876026 wait complete. 2024-04-24T22:45:26.995 INFO:tasks.workunit.client.1.smithi192.stdout:7/384: write d2/db/f15 [1049207,69313] 0 2024-04-24T22:45:26.995 INFO:tasks.workunit.client.1.smithi192.stdout:7/385: chown d2/db/d25/l3b 67 1 2024-04-24T22:45:26.995 INFO:tasks.workunit.client.1.smithi192.stdout:9/386: mkdir d3/dd/de/d17/d89 0 2024-04-24T22:45:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 Processor -- start 2024-04-24T22:45:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 -- start start 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 0x7f2d100702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d0ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d0ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47304/0 (socket says 172.21.15.63:47304) 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d0ed9d700 1 -- 172.21.15.63:0/3803661009 learned_addr learned my addr 172.21.15.63:0/3803661009 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100708f0 con 0x7f2d100fce90 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.993+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10070a60 con 0x7f2d100fe090 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d0e59c700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 0x7f2d100702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d0ed9d700 1 -- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 msgr2=0x7f2d100702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d0ed9d700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 0x7f2d100702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d0ed9d700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00005040 con 0x7f2d100fce90 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d0ed9d700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d00000c00 tx=0x7f2d00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0000e030 con 0x7f2d100fce90 2024-04-24T22:45:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d10070ce0 con 0x7f2d100fce90 2024-04-24T22:45:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d10115b20 con 0x7f2d100fce90 2024-04-24T22:45:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d00013420 con 0x7f2d100fce90 2024-04-24T22:45:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.994+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0000bee0 con 0x7f2d100fce90 2024-04-24T22:45:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.996+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc0052e0 con 0x7f2d100fce90 2024-04-24T22:45:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.996+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f2d0001c750 con 0x7f2d100fce90 2024-04-24T22:45:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.997+0000 7f2cf77fe700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 0x7f2cf806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.997+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2d0008bd70 con 0x7f2d100fce90 2024-04-24T22:45:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.997+0000 7f2d0e59c700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 0x7f2cf806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:26.998+0000 7f2d0e59c700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 0x7f2cf806dc40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009a80 tx=0x7f2d04008040 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:27.000 INFO:tasks.workunit.client.1.smithi192.stdout:9/387: creat d3/d59/d7d/f8a x:0 0 0 2024-04-24T22:45:27.001 INFO:tasks.workunit.client.0.smithi063.stdout:7/363: link d0/d4/de/f25 d0/d4/de/d6f/d5d/f80 0 2024-04-24T22:45:27.002 INFO:tasks.workunit.client.1.smithi192.stdout:7/386: mkdir d2/db/d2a/d7a 0 2024-04-24T22:45:27.003 INFO:tasks.workunit.client.0.smithi063.stdout:1/489: dread d3/da/dc/f3a [0,4194304] 0 2024-04-24T22:45:27.003 INFO:tasks.workunit.client.0.smithi063.stdout:5/426: truncate d5/f5b 1172686 0 2024-04-24T22:45:27.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.002+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2d000559e0 con 0x7f2d100fce90 2024-04-24T22:45:27.007 INFO:tasks.workunit.client.0.smithi063.stdout:5/427: write d5/d11/f6a [5167219,113509] 0 2024-04-24T22:45:27.008 INFO:tasks.workunit.client.1.smithi192.stdout:5/311: mkdir d0/d1b/d62 0 2024-04-24T22:45:27.008 INFO:tasks.workunit.client.0.smithi063.stdout:5/428: readlink d5/d11/d18/l2e 0 2024-04-24T22:45:27.008 INFO:tasks.workunit.client.0.smithi063.stdout:5/429: chown d5/d1d/d7d 3699720 1 2024-04-24T22:45:27.009 INFO:tasks.workunit.client.0.smithi063.stdout:5/430: chown d5/d1d/d7d/l61 44 1 2024-04-24T22:45:27.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/388: unlink d3/dd/de/d17/l67 0 2024-04-24T22:45:27.013 INFO:tasks.workunit.client.0.smithi063.stdout:6/471: unlink db/d1d/d20/d84/f8f 0 2024-04-24T22:45:27.014 INFO:tasks.workunit.client.1.smithi192.stdout:7/387: dread d2/db/d21/f28 [4194304,4194304] 0 2024-04-24T22:45:27.014 INFO:tasks.workunit.client.1.smithi192.stdout:5/312: symlink d0/d13/d19/l63 0 2024-04-24T22:45:27.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/472: chown db/d1d/d2d/l92 1217649 1 2024-04-24T22:45:27.015 INFO:tasks.workunit.client.1.smithi192.stdout:5/313: dread - d0/d13/f43 zero size 2024-04-24T22:45:27.017 INFO:tasks.workunit.client.0.smithi063.stdout:0/379: dwrite d6/da/f3d [0,4194304] 0 2024-04-24T22:45:27.018 INFO:tasks.workunit.client.1.smithi192.stdout:9/389: rename d3/dd/de/d17/d33 to d3/dd/de/d17/d89/d8b 0 2024-04-24T22:45:27.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/388: dread d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:27.019 INFO:tasks.workunit.client.0.smithi063.stdout:2/485: sync 2024-04-24T22:45:27.019 INFO:tasks.workunit.client.1.smithi192.stdout:5/314: dread d0/d1b/f45 [0,4194304] 0 2024-04-24T22:45:27.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/389: write d2/db/d2a/f61 [975209,38593] 0 2024-04-24T22:45:27.026 INFO:tasks.workunit.client.0.smithi063.stdout:6/473: dwrite db/d1d/d2d/d48/f6e [0,4194304] 0 2024-04-24T22:45:27.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/380: dread d6/da/dc/d26/f83 [0,4194304] 0 2024-04-24T22:45:27.028 INFO:tasks.workunit.client.1.smithi192.stdout:5/315: dwrite d0/d13/d3b/d2f/d34/d5f/f3e [0,4194304] 0 2024-04-24T22:45:27.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/447: mkdir d1/d11/d49/d91 0 2024-04-24T22:45:27.034 INFO:tasks.workunit.client.0.smithi063.stdout:8/448: write d1/d11/d32/d38/f67 [3750389,109393] 0 2024-04-24T22:45:27.038 INFO:tasks.workunit.client.0.smithi063.stdout:6/474: dwrite db/d1d/d20/d3e/f74 [0,4194304] 0 2024-04-24T22:45:27.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/475: dwrite db/d1d/d20/f50 [0,4194304] 0 2024-04-24T22:45:27.054 INFO:tasks.workunit.client.0.smithi063.stdout:2/486: dread d0/d2/f4b [0,4194304] 0 2024-04-24T22:45:27.080 INFO:tasks.workunit.client.0.smithi063.stdout:7/364: fdatasync d0/d4/de/f61 0 2024-04-24T22:45:27.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/390: rename d3/dd/de/d39 to d3/dd/de/d17/d89/d8b/d8c 0 2024-04-24T22:45:27.086 INFO:tasks.workunit.client.0.smithi063.stdout:1/490: fdatasync d3/da/dc/f3a 0 2024-04-24T22:45:27.087 INFO:tasks.workunit.client.1.smithi192.stdout:5/316: rename d0/d13/d3b/d2f/d34 to d0/d13/d19/d64 0 2024-04-24T22:45:27.089 INFO:tasks.workunit.client.0.smithi063.stdout:3/478: mknod da/d40/ca8 0 2024-04-24T22:45:27.090 INFO:tasks.workunit.client.1.smithi192.stdout:9/391: unlink d3/dd/de/d17/d6a/f54 0 2024-04-24T22:45:27.091 INFO:tasks.workunit.client.1.smithi192.stdout:9/392: truncate d3/f31 712070 0 2024-04-24T22:45:27.092 INFO:tasks.workunit.client.0.smithi063.stdout:4/395: creat d2/f7e x:0 0 0 2024-04-24T22:45:27.092 INFO:tasks.workunit.client.1.smithi192.stdout:9/393: dread - d3/dd/d62/f6b zero size 2024-04-24T22:45:27.094 INFO:tasks.workunit.client.1.smithi192.stdout:7/390: getdents d2/db/d21/d42/d67 0 2024-04-24T22:45:27.099 INFO:tasks.workunit.client.1.smithi192.stdout:7/391: unlink d2/db/d36/f38 0 2024-04-24T22:45:27.100 INFO:tasks.workunit.client.1.smithi192.stdout:7/392: chown d2/db/d3a/l3e 16548 1 2024-04-24T22:45:27.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/393: chown d2/db/d21/d42/f4d 324428346 1 2024-04-24T22:45:27.102 INFO:tasks.workunit.client.1.smithi192.stdout:5/317: creat d0/d1b/f65 x:0 0 0 2024-04-24T22:45:27.103 INFO:tasks.workunit.client.0.smithi063.stdout:9/438: rmdir d6/d8/db/d1c/d78/d8c 0 2024-04-24T22:45:27.109 INFO:tasks.workunit.client.1.smithi192.stdout:9/394: dread d3/f4 [0,4194304] 0 2024-04-24T22:45:27.111 INFO:tasks.workunit.client.0.smithi063.stdout:8/449: rmdir d1/d11/d3c 39 2024-04-24T22:45:27.114 INFO:tasks.workunit.client.1.smithi192.stdout:5/318: mkdir d0/d13/d19/d64/d5f/d66 0 2024-04-24T22:45:27.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/395: creat d3/f8d x:0 0 0 2024-04-24T22:45:27.116 INFO:tasks.workunit.client.0.smithi063.stdout:6/476: mkdir db/da0 0 2024-04-24T22:45:27.119 INFO:tasks.workunit.client.1.smithi192.stdout:5/319: creat d0/d13/d19/d64/d4f/d52/f67 x:0 0 0 2024-04-24T22:45:27.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/394: dwrite d2/fa [0,4194304] 0 2024-04-24T22:45:27.121 INFO:tasks.workunit.client.1.smithi192.stdout:7/395: truncate d2/db/d21/f39 1064160 0 2024-04-24T22:45:27.121 INFO:tasks.workunit.client.1.smithi192.stdout:7/396: readlink d2/db/d3a/l68 0 2024-04-24T22:45:27.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/491: creat d3/da/d34/d6d/fa0 x:0 0 0 2024-04-24T22:45:27.122 INFO:tasks.workunit.client.0.smithi063.stdout:9/439: dread d6/d2f/d42/f7b [0,4194304] 0 2024-04-24T22:45:27.123 INFO:tasks.workunit.client.0.smithi063.stdout:9/440: readlink d6/d2f/d42/l74 0 2024-04-24T22:45:27.126 INFO:tasks.workunit.client.0.smithi063.stdout:4/396: creat d2/d8/d36/d3a/f7f x:0 0 0 2024-04-24T22:45:27.128 INFO:tasks.workunit.client.1.smithi192.stdout:7/397: rmdir d2/db/d49 39 2024-04-24T22:45:27.129 INFO:tasks.workunit.client.0.smithi063.stdout:2/487: write d0/d2/d27/d3a/f84 [102111,47742] 0 2024-04-24T22:45:27.130 INFO:tasks.workunit.client.1.smithi192.stdout:5/320: creat d0/d13/d19/d64/d5f/d66/f68 x:0 0 0 2024-04-24T22:45:27.131 INFO:tasks.workunit.client.1.smithi192.stdout:9/396: readlink d3/dd/de/lf 0 2024-04-24T22:45:27.132 INFO:tasks.workunit.client.0.smithi063.stdout:3/479: write da/d18/f30 [469734,126221] 0 2024-04-24T22:45:27.132 INFO:tasks.workunit.client.1.smithi192.stdout:7/398: fdatasync d2/db/d21/d42/f4d 0 2024-04-24T22:45:27.134 INFO:tasks.workunit.client.1.smithi192.stdout:9/397: mknod d3/dd/d53/c8e 0 2024-04-24T22:45:27.135 INFO:tasks.workunit.client.1.smithi192.stdout:9/398: readlink d3/dd/l2a 0 2024-04-24T22:45:27.135 INFO:tasks.workunit.client.0.smithi063.stdout:0/381: fdatasync d6/da/dc/f1e 0 2024-04-24T22:45:27.136 INFO:tasks.workunit.client.0.smithi063.stdout:8/450: mknod d1/d2/d17/c92 0 2024-04-24T22:45:27.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/451: chown d1/c3e 14987 1 2024-04-24T22:45:27.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/399: truncate d2/f17 2472188 0 2024-04-24T22:45:27.138 INFO:tasks.workunit.client.0.smithi063.stdout:6/477: unlink db/d1d/d2d/d48/c80 0 2024-04-24T22:45:27.138 INFO:tasks.workunit.client.0.smithi063.stdout:7/365: dread d0/f4f [0,4194304] 0 2024-04-24T22:45:27.140 INFO:tasks.workunit.client.1.smithi192.stdout:7/400: mknod d2/db/d2a/c7b 0 2024-04-24T22:45:27.144 INFO:tasks.workunit.client.0.smithi063.stdout:1/492: creat d3/da/d34/fa1 x:0 0 0 2024-04-24T22:45:27.145 INFO:tasks.workunit.client.0.smithi063.stdout:1/493: fdatasync d3/da/d34/f9a 0 2024-04-24T22:45:27.147 INFO:tasks.workunit.client.1.smithi192.stdout:9/399: write d3/dd/de/d17/f35 [215474,52709] 0 2024-04-24T22:45:27.149 INFO:tasks.workunit.client.1.smithi192.stdout:7/401: unlink d2/db/d25/f47 0 2024-04-24T22:45:27.151 INFO:tasks.workunit.client.0.smithi063.stdout:5/431: rename d5/le to d5/d24/d49/l99 0 2024-04-24T22:45:27.153 INFO:tasks.workunit.client.1.smithi192.stdout:7/402: dread d2/db/f45 [0,4194304] 0 2024-04-24T22:45:27.154 INFO:tasks.workunit.client.1.smithi192.stdout:7/403: chown d2/db/d21/f22 260 1 2024-04-24T22:45:27.158 INFO:tasks.workunit.client.0.smithi063.stdout:0/382: symlink d6/dd/l85 0 2024-04-24T22:45:27.160 INFO:tasks.workunit.client.1.smithi192.stdout:9/400: creat d3/dd/de/d17/f8f x:0 0 0 2024-04-24T22:45:27.163 INFO:tasks.workunit.client.1.smithi192.stdout:9/401: fdatasync d3/d24/f7e 0 2024-04-24T22:45:27.164 INFO:tasks.workunit.client.0.smithi063.stdout:2/488: mkdir d0/d2/d6c/d7a/d94 0 2024-04-24T22:45:27.165 INFO:tasks.workunit.client.1.smithi192.stdout:9/402: symlink d3/d24/l90 0 2024-04-24T22:45:27.167 INFO:tasks.workunit.client.0.smithi063.stdout:2/489: write d0/d2/d27/f29 [4406826,32773] 0 2024-04-24T22:45:27.168 INFO:tasks.workunit.client.0.smithi063.stdout:1/494: dread d3/f19 [0,4194304] 0 2024-04-24T22:45:27.168 INFO:tasks.workunit.client.0.smithi063.stdout:0/383: write d6/da/f29 [2938436,83187] 0 2024-04-24T22:45:27.168 INFO:tasks.workunit.client.1.smithi192.stdout:7/404: truncate d2/db/d3a/f48 711145 0 2024-04-24T22:45:27.170 INFO:tasks.workunit.client.0.smithi063.stdout:9/441: mkdir d6/d8/d1f/d20/d2a/d4b/d5e/d8e 0 2024-04-24T22:45:27.172 INFO:tasks.workunit.client.0.smithi063.stdout:5/432: dwrite d5/d24/d49/f84 [0,4194304] 0 2024-04-24T22:45:27.172 INFO:tasks.workunit.client.1.smithi192.stdout:9/403: dwrite d3/dd/de/d17/f68 [0,4194304] 0 2024-04-24T22:45:27.175 INFO:tasks.workunit.client.0.smithi063.stdout:8/452: creat d1/d11/f93 x:0 0 0 2024-04-24T22:45:27.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/433: write d5/f6d [726736,23375] 0 2024-04-24T22:45:27.177 INFO:tasks.workunit.client.0.smithi063.stdout:6/478: creat db/d1d/d20/fa1 x:0 0 0 2024-04-24T22:45:27.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/442: read d6/d2f/f35 [3989629,106668] 0 2024-04-24T22:45:27.179 INFO:tasks.workunit.client.0.smithi063.stdout:7/366: dwrite d0/d4/f3a [0,4194304] 0 2024-04-24T22:45:27.184 INFO:tasks.workunit.client.0.smithi063.stdout:1/495: fsync d3/da/dc/d1f/f22 0 2024-04-24T22:45:27.189 INFO:tasks.workunit.client.0.smithi063.stdout:0/384: dwrite d6/d3a/f7a [0,4194304] 0 2024-04-24T22:45:27.192 INFO:tasks.workunit.client.0.smithi063.stdout:2/490: dread d0/d2/d2d/d31/d89/d48/f5d [0,4194304] 0 2024-04-24T22:45:27.195 INFO:tasks.workunit.client.1.smithi192.stdout:7/405: dwrite d2/db/d21/f39 [0,4194304] 0 2024-04-24T22:45:27.195 INFO:tasks.workunit.client.1.smithi192.stdout:7/406: link d2/db/d21/l55 d2/db/d3a/d6e/d77/l7c 0 2024-04-24T22:45:27.196 INFO:tasks.workunit.client.0.smithi063.stdout:8/453: write d1/d11/d3c/f69 [1646665,78293] 0 2024-04-24T22:45:27.196 INFO:tasks.workunit.client.1.smithi192.stdout:7/407: truncate d2/db/f20 12854553 0 2024-04-24T22:45:27.196 INFO:tasks.workunit.client.0.smithi063.stdout:2/491: chown d0/d2/d35/f82 357102 1 2024-04-24T22:45:27.197 INFO:tasks.workunit.client.1.smithi192.stdout:9/404: getdents d3/d59 0 2024-04-24T22:45:27.199 INFO:tasks.workunit.client.0.smithi063.stdout:0/385: read d6/d64/d21/f36 [1603826,120439] 0 2024-04-24T22:45:27.199 INFO:tasks.workunit.client.0.smithi063.stdout:9/443: unlink d6/d2f/l37 0 2024-04-24T22:45:27.201 INFO:tasks.workunit.client.1.smithi192.stdout:9/405: write d3/f43 [2725615,17592] 0 2024-04-24T22:45:27.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/480: rename da/cd to da/d39/ca9 0 2024-04-24T22:45:27.206 INFO:tasks.workunit.client.1.smithi192.stdout:9/406: mkdir d3/dd/d62/d6d/d91 0 2024-04-24T22:45:27.207 INFO:tasks.workunit.client.0.smithi063.stdout:7/367: chown d0/d2f/l58 0 1 2024-04-24T22:45:27.207 INFO:tasks.workunit.client.1.smithi192.stdout:9/407: chown d3/d59/d7d 341328 1 2024-04-24T22:45:27.208 INFO:tasks.workunit.client.1.smithi192.stdout:9/408: read d3/dd/de/f19 [954978,30947] 0 2024-04-24T22:45:27.210 INFO:tasks.workunit.client.0.smithi063.stdout:9/444: dread d6/d8/d53/f82 [0,4194304] 0 2024-04-24T22:45:27.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/445: readlink d6/d2f/l85 0 2024-04-24T22:45:27.213 INFO:tasks.workunit.client.1.smithi192.stdout:9/409: creat d3/dd/d62/f92 x:0 0 0 2024-04-24T22:45:27.214 INFO:tasks.workunit.client.1.smithi192.stdout:9/410: truncate d3/f43 4899510 0 2024-04-24T22:45:27.215 INFO:tasks.workunit.client.1.smithi192.stdout:9/411: dread - d3/dd/de/d17/d89/d8b/d8c/f70 zero size 2024-04-24T22:45:27.216 INFO:tasks.workunit.client.0.smithi063.stdout:2/492: creat d0/d2/d35/f95 x:0 0 0 2024-04-24T22:45:27.216 INFO:tasks.workunit.client.0.smithi063.stdout:2/493: readlink d0/d2/d2d/d31/d89/l38 0 2024-04-24T22:45:27.217 INFO:tasks.workunit.client.0.smithi063.stdout:4/397: sync 2024-04-24T22:45:27.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/494: write d0/d2/d2d/d31/d89/d7/f13 [4840758,4266] 0 2024-04-24T22:45:27.219 INFO:tasks.workunit.client.1.smithi192.stdout:9/412: unlink d3/c5 0 2024-04-24T22:45:27.222 INFO:tasks.workunit.client.1.smithi192.stdout:9/413: write d3/dd/de/f55 [292763,113829] 0 2024-04-24T22:45:27.223 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:26 smithi063 ceph-mon[26912]: pgmap v11: 65 pgs: 65 active+clean; 1.1 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 14 MiB/s rd, 144 MiB/s wr, 293 op/s 2024-04-24T22:45:27.223 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:26 smithi063 ceph-mon[26912]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:27.224 INFO:tasks.workunit.client.1.smithi192.stdout:9/414: write d3/dd/de/d17/f48 [2988615,15030] 0 2024-04-24T22:45:27.229 INFO:tasks.workunit.client.1.smithi192.stdout:9/415: dread d3/dd/de/d17/f35 [0,4194304] 0 2024-04-24T22:45:27.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/408: dwrite d2/db/d21/d42/d5b/d63/f6d [0,4194304] 0 2024-04-24T22:45:27.231 INFO:tasks.workunit.client.0.smithi063.stdout:1/496: dread d3/da/dc/d1f/d21/d49/f9e [0,4194304] 0 2024-04-24T22:45:27.231 INFO:tasks.workunit.client.1.smithi192.stdout:7/409: chown d2/db/d21/d42/d5b/f5d 8084780 1 2024-04-24T22:45:27.232 INFO:tasks.workunit.client.0.smithi063.stdout:1/497: dread - d3/da/dc/d11/f5f zero size 2024-04-24T22:45:27.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/410: chown d2/db/d21/d42/d67/c78 28248636 1 2024-04-24T22:45:27.238 INFO:tasks.workunit.client.1.smithi192.stdout:9/416: truncate d3/dd/de/d17/d89/d8b/f3e 1583090 0 2024-04-24T22:45:27.239 INFO:tasks.workunit.client.1.smithi192.stdout:7/411: read d2/f1a [5534143,105131] 0 2024-04-24T22:45:27.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/412: dread - d2/db/f60 zero size 2024-04-24T22:45:27.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/413: stat d2/db/d25 0 2024-04-24T22:45:27.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/498: dread d3/da/dc/d11/d5e/d6f/f90 [4194304,4194304] 0 2024-04-24T22:45:27.248 INFO:tasks.workunit.client.0.smithi063.stdout:1/499: stat d3/da/dc/d11/f46 0 2024-04-24T22:45:27.251 INFO:tasks.workunit.client.1.smithi192.stdout:0/406: sync 2024-04-24T22:45:27.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/407: write d3/d27/d3c/d3b/f65 [669071,63147] 0 2024-04-24T22:45:27.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/417: symlink d3/dd/de/d17/d89/d8b/l93 0 2024-04-24T22:45:27.254 INFO:tasks.workunit.client.1.smithi192.stdout:9/418: readlink d3/dd/de/d17/d89/d8b/l93 0 2024-04-24T22:45:27.257 INFO:tasks.workunit.client.1.smithi192.stdout:0/408: chown d3/d27/d28/d2e/d3f/l7b 4 1 2024-04-24T22:45:27.258 INFO:tasks.workunit.client.0.smithi063.stdout:5/434: truncate d5/f5b 1620863 0 2024-04-24T22:45:27.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/435: read d5/d11/f3c [2136985,32748] 0 2024-04-24T22:45:27.260 INFO:tasks.workunit.client.0.smithi063.stdout:6/479: mknod db/d1d/d20/d3f/d5f/d93/ca2 0 2024-04-24T22:45:27.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/409: creat d3/d27/d3c/d3b/f8d x:0 0 0 2024-04-24T22:45:27.261 INFO:tasks.workunit.client.0.smithi063.stdout:6/480: chown db/d1d/d20/l9c 75 1 2024-04-24T22:45:27.261 INFO:tasks.workunit.client.1.smithi192.stdout:0/410: chown d3/d27/d28/d2e/d3f/l7b 0 1 2024-04-24T22:45:27.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/411: truncate d3/d6d/d79/f7a 334422 0 2024-04-24T22:45:27.263 INFO:tasks.workunit.client.0.smithi063.stdout:0/386: creat d6/d64/d21/f86 x:0 0 0 2024-04-24T22:45:27.264 INFO:tasks.workunit.client.1.smithi192.stdout:7/414: creat d2/db/d25/f7d x:0 0 0 2024-04-24T22:45:27.264 INFO:tasks.workunit.client.1.smithi192.stdout:0/412: read d3/d27/d28/f24 [3548151,98355] 0 2024-04-24T22:45:27.264 INFO:tasks.workunit.client.0.smithi063.stdout:0/387: write d6/d64/d24/f59 [437226,104771] 0 2024-04-24T22:45:27.265 INFO:tasks.workunit.client.0.smithi063.stdout:0/388: chown d6/d64/d24/d42/l4b 11377 1 2024-04-24T22:45:27.266 INFO:tasks.workunit.client.0.smithi063.stdout:7/368: rename d0/d4/c4e to d0/d4/de/d17/d28/d31/c81 0 2024-04-24T22:45:27.268 INFO:tasks.workunit.client.1.smithi192.stdout:0/413: unlink d3/d6d/c75 0 2024-04-24T22:45:27.270 INFO:tasks.workunit.client.1.smithi192.stdout:7/415: creat d2/db/d21/d42/d67/f7e x:0 0 0 2024-04-24T22:45:27.273 INFO:tasks.workunit.client.1.smithi192.stdout:0/414: readlink d3/d27/d3c/d3b/l7d 0 2024-04-24T22:45:27.273 INFO:tasks.workunit.client.1.smithi192.stdout:0/415: readlink d3/d27/d3c/d6c/l70 0 2024-04-24T22:45:27.274 INFO:tasks.workunit.client.1.smithi192.stdout:0/416: write d3/d6d/d79/f8b [608523,112282] 0 2024-04-24T22:45:27.275 INFO:tasks.workunit.client.1.smithi192.stdout:7/416: symlink d2/db/d21/d42/l7f 0 2024-04-24T22:45:27.277 INFO:tasks.workunit.client.1.smithi192.stdout:0/417: creat d3/d27/d28/f8e x:0 0 0 2024-04-24T22:45:27.280 INFO:tasks.workunit.client.1.smithi192.stdout:0/418: creat d3/d27/d3c/d64/f8f x:0 0 0 2024-04-24T22:45:27.284 INFO:tasks.workunit.client.1.smithi192.stdout:0/419: unlink d3/d27/d28/f3e 0 2024-04-24T22:45:27.285 INFO:tasks.workunit.client.0.smithi063.stdout:1/500: readlink d3/da/dc/d1f/d21/l84 0 2024-04-24T22:45:27.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/419: dwrite d3/dd/de/f16 [0,4194304] 0 2024-04-24T22:45:27.292 INFO:tasks.workunit.client.1.smithi192.stdout:0/420: rmdir d3/d6d 39 2024-04-24T22:45:27.292 INFO:tasks.workunit.client.0.smithi063.stdout:5/436: chown d5/d24/d2c/f50 491 1 2024-04-24T22:45:27.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/446: write d6/d8/d1f/d20/d22/f6d [883105,77279] 0 2024-04-24T22:45:27.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/437: write d5/d24/d49/d3b/f48 [1635594,105527] 0 2024-04-24T22:45:27.296 INFO:tasks.workunit.client.0.smithi063.stdout:8/454: truncate d1/d2/d17/f3d 1879868 0 2024-04-24T22:45:27.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/420: mknod d3/dd/c94 0 2024-04-24T22:45:27.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/421: chown d3/dd/de/c73 1554 1 2024-04-24T22:45:27.299 INFO:tasks.workunit.client.1.smithi192.stdout:7/417: write d2/db/f45 [1653513,98036] 0 2024-04-24T22:45:27.299 INFO:tasks.workunit.client.0.smithi063.stdout:9/447: dwrite d6/f6c [0,4194304] 0 2024-04-24T22:45:27.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/418: chown d2/db/d36/c53 72592 1 2024-04-24T22:45:27.301 INFO:tasks.workunit.client.0.smithi063.stdout:0/389: mkdir d6/da/dc/d26/d33/d87 0 2024-04-24T22:45:27.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/421: symlink d3/d27/l90 0 2024-04-24T22:45:27.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/422: read - d3/d27/f71 zero size 2024-04-24T22:45:27.303 INFO:tasks.workunit.client.0.smithi063.stdout:7/369: symlink d0/d4/de/d17/d28/d31/l82 0 2024-04-24T22:45:27.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/398: mkdir d2/d80 0 2024-04-24T22:45:27.306 INFO:tasks.workunit.client.1.smithi192.stdout:7/419: mkdir d2/db/d21/d42/d5b/d63/d80 0 2024-04-24T22:45:27.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/420: fdatasync d2/db/f50 0 2024-04-24T22:45:27.308 INFO:tasks.workunit.client.1.smithi192.stdout:0/423: creat d3/d6d/f91 x:0 0 0 2024-04-24T22:45:27.309 INFO:tasks.workunit.client.1.smithi192.stdout:7/421: write d2/db/d21/d42/d67/f7e [274925,101380] 0 2024-04-24T22:45:27.310 INFO:tasks.workunit.client.0.smithi063.stdout:1/501: mkdir d3/da/da2 0 2024-04-24T22:45:27.315 INFO:tasks.workunit.client.1.smithi192.stdout:0/424: unlink d3/d27/d28/f26 0 2024-04-24T22:45:27.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/425: truncate d3/d27/d28/d2e/d4a/f7f 904716 0 2024-04-24T22:45:27.317 INFO:tasks.workunit.client.1.smithi192.stdout:7/422: mknod d2/db/c81 0 2024-04-24T22:45:27.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.318+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cfc000bf0 con 0x7f2cf806b790 2024-04-24T22:45:27.321 INFO:tasks.workunit.client.0.smithi063.stdout:8/455: symlink d1/d11/d49/d54/l94 0 2024-04-24T22:45:27.323 INFO:tasks.workunit.client.0.smithi063.stdout:9/448: creat d6/d8/db/d6b/f8f x:0 0 0 2024-04-24T22:45:27.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/423: mkdir d2/db/d25/d82 0 2024-04-24T22:45:27.325 INFO:tasks.workunit.client.0.smithi063.stdout:7/370: symlink d0/d4/de/d17/d35/l83 0 2024-04-24T22:45:27.326 INFO:tasks.workunit.client.0.smithi063.stdout:7/371: write d0/f73 [4167122,44985] 0 2024-04-24T22:45:27.329 INFO:tasks.workunit.client.0.smithi063.stdout:3/481: write da/f4c [1499982,116751] 0 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.331+0000 7f2cf77fe700 1 -- 172.21.15.63:0/3803661009 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f2cfc000bf0 con 0x7f2cf806b790 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 2m ago 8m 24.5M - 0.25.0 c8568f914cd2 f1e9967ae2bb 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 2m ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (7m) 22s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 2m ago 8m 85.4M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (2m) 2m ago 2m 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (3m) 2m ago 3m 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (2m) 22s ago 2m 157M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:45:27.333 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (2m) 22s ago 2m 12.9M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (40s) 22s ago 7m 125M - 19.0.0-2445-gd2c45d97 f68e37b807ef a965183fccc3 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 2m ago 10m 46.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (7m) 22s ago 7m 39.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 2m ago 8m 15.4M - 1.5.0 0da6a335fe13 b97c5a0b7f9e 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 22s ago 7m 15.7M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 6m 48.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 2m ago 6m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:45:27.334 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 2m ago 5m 44.4M 4288M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:45:27.335 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (5m) 22s ago 5m 82.3M 5312M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:45:27.335 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (4m) 22s ago 4m 74.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:45:27.335 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (4m) 22s ago 4m 72.7M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:45:27.335 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 2m ago 7m 53.9M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:45:27.335 INFO:tasks.workunit.client.0.smithi063.stdout:0/390: symlink d6/l88 0 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 msgr2=0x7f2cf806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 0x7f2cf806dc40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009a80 tx=0x7f2d04008040 comp rx=0 tx=0).stop 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 msgr2=0x7f2d10071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d00000c00 tx=0x7f2d00009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 shutdown_connections 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f2cf806b790 0x7f2cf806dc40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d100fce90 0x7f2d10071de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 --2- 172.21.15.63:0/3803661009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d100fe090 0x7f2d100702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 >> 172.21.15.63:0/3803661009 conn(0x7f2d100f85c0 msgr2=0x7f2d100f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 shutdown_connections 2024-04-24T22:45:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.334+0000 7f2d150c7700 1 -- 172.21.15.63:0/3803661009 wait complete. 2024-04-24T22:45:27.337 INFO:tasks.workunit.client.0.smithi063.stdout:4/399: mkdir d2/d80/d81 0 2024-04-24T22:45:27.337 INFO:tasks.workunit.client.0.smithi063.stdout:2/495: getdents d0/d2/d27/d3a 0 2024-04-24T22:45:27.338 INFO:tasks.workunit.client.0.smithi063.stdout:5/438: dwrite d5/d11/d18/f20 [0,4194304] 0 2024-04-24T22:45:27.339 INFO:tasks.workunit.client.0.smithi063.stdout:6/481: getdents db/d1d 0 2024-04-24T22:45:27.341 INFO:tasks.workunit.client.1.smithi192.stdout:7/424: dwrite d2/f1a [4194304,4194304] 0 2024-04-24T22:45:27.355 INFO:tasks.workunit.client.1.smithi192.stdout:7/425: write d2/f17 [3259181,85865] 0 2024-04-24T22:45:27.363 INFO:tasks.workunit.client.1.smithi192.stdout:7/426: dwrite d2/db/d2a/f61 [0,4194304] 0 2024-04-24T22:45:27.364 INFO:tasks.workunit.client.1.smithi192.stdout:7/427: symlink d2/db/d3a/d6e/d77/l83 0 2024-04-24T22:45:27.419 INFO:tasks.workunit.client.1.smithi192.stdout:7/428: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:27.425 INFO:tasks.workunit.client.1.smithi192.stdout:7/429: dwrite d2/db/d49/f70 [0,4194304] 0 2024-04-24T22:45:27.425 INFO:tasks.workunit.client.1.smithi192.stdout:7/430: stat d2/db/f50 0 2024-04-24T22:45:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:26 smithi192 ceph-mon[33991]: pgmap v11: 65 pgs: 65 active+clean; 1.1 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 14 MiB/s rd, 144 MiB/s wr, 293 op/s 2024-04-24T22:45:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:26 smithi192 ceph-mon[33991]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:27.429 INFO:tasks.workunit.client.1.smithi192.stdout:7/431: unlink f1 0 2024-04-24T22:45:27.430 INFO:tasks.workunit.client.1.smithi192.stdout:7/432: write d2/db/d21/d42/d67/f7e [939677,47236] 0 2024-04-24T22:45:27.432 INFO:tasks.workunit.client.1.smithi192.stdout:7/433: creat d2/db/d21/f84 x:0 0 0 2024-04-24T22:45:27.437 INFO:tasks.workunit.client.0.smithi063.stdout:1/502: link d3/f3c d3/fa3 0 2024-04-24T22:45:27.438 INFO:tasks.workunit.client.0.smithi063.stdout:9/449: truncate d6/d2f/d42/f7b 1686203 0 2024-04-24T22:45:27.440 INFO:tasks.workunit.client.0.smithi063.stdout:9/450: chown d6/d8/d8a 60331033 1 2024-04-24T22:45:27.442 INFO:tasks.workunit.client.0.smithi063.stdout:3/482: symlink da/d39/laa 0 2024-04-24T22:45:27.442 INFO:tasks.workunit.client.0.smithi063.stdout:1/503: symlink d3/da/dc/d11/d5e/la4 0 2024-04-24T22:45:27.443 INFO:tasks.workunit.client.0.smithi063.stdout:5/439: link d5/f6d d5/d11/d76/f9a 0 2024-04-24T22:45:27.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/434: truncate d2/f1a 6885718 0 2024-04-24T22:45:27.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/400: dwrite d2/d8/d3c/d6f/f76 [0,4194304] 0 2024-04-24T22:45:27.445 INFO:tasks.workunit.client.0.smithi063.stdout:8/456: truncate d1/d2/d17/f3d 1276620 0 2024-04-24T22:45:27.449 INFO:tasks.workunit.client.1.smithi192.stdout:7/435: rename d2/db/d21/c52 to d2/db/d21/c85 0 2024-04-24T22:45:27.450 INFO:tasks.workunit.client.0.smithi063.stdout:0/391: getdents d6 0 2024-04-24T22:45:27.452 INFO:tasks.workunit.client.1.smithi192.stdout:7/436: fsync d2/db/d25/f69 0 2024-04-24T22:45:27.456 INFO:tasks.workunit.client.1.smithi192.stdout:7/437: mkdir d2/db/d2a/d86 0 2024-04-24T22:45:27.457 INFO:tasks.workunit.client.0.smithi063.stdout:6/482: dwrite db/d1d/d2d/f36 [0,4194304] 0 2024-04-24T22:45:27.457 INFO:tasks.workunit.client.0.smithi063.stdout:3/483: dread - da/d18/d47/d4d/d62/f6f zero size 2024-04-24T22:45:27.457 INFO:tasks.workunit.client.0.smithi063.stdout:2/496: dwrite d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:27.457 INFO:tasks.workunit.client.0.smithi063.stdout:1/504: rename d3/da/dc/d11/d5e/f88 to d3/da/dc/d11/fa5 0 2024-04-24T22:45:27.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/438: chown d2/db/d25/l3b 13095785 1 2024-04-24T22:45:27.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/439: chown d2/db 83435459 1 2024-04-24T22:45:27.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/505: write d3/da/dc/d1f/f22 [3930943,123407] 0 2024-04-24T22:45:27.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/392: mkdir d6/d64/d21/d25/d62/d63/d89 0 2024-04-24T22:45:27.468 INFO:tasks.workunit.client.0.smithi063.stdout:2/497: symlink d0/dd/d62/d7c/l96 0 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 -- 172.21.15.63:0/1674009060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06f0d0 msgr2=0x7fb72c108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 --2- 172.21.15.63:0/1674009060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06f0d0 0x7fb72c108f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb720004930 tx=0x7fb72000c040 comp rx=0 tx=0).stop 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 -- 172.21.15.63:0/1674009060 shutdown_connections 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 --2- 172.21.15.63:0/1674009060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06f0d0 0x7fb72c108f80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 --2- 172.21.15.63:0/1674009060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06e780 0x7fb72c06eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 -- 172.21.15.63:0/1674009060 >> 172.21.15.63:0/1674009060 conn(0x7fb72c06a360 msgr2=0x7fb72c06c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:27.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 -- 172.21.15.63:0/1674009060 shutdown_connections 2024-04-24T22:45:27.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.467+0000 7fb733e61700 1 -- 172.21.15.63:0/1674009060 wait complete. 2024-04-24T22:45:27.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.468+0000 7fb733e61700 1 Processor -- start 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.468+0000 7fb733e61700 1 -- start start 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.468+0000 7fb733e61700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06e780 0x7fb72c118330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb733e61700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb733e61700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c118e70 con 0x7fb72c06e780 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb733e61700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c118fe0 con 0x7fb72c06f0d0 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:49298/0 (socket says 172.21.15.63:49298) 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 -- 172.21.15.63:0/3875402426 learned_addr learned my addr 172.21.15.63:0/3875402426 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb731bfd700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06e780 0x7fb72c118330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 -- 172.21.15.63:0/3875402426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06e780 msgr2=0x7fb72c118330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06e780 0x7fb72c118330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.469+0000 7fb7313fc700 1 -- 172.21.15.63:0/3875402426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720005040 con 0x7fb72c06f0d0 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb7313fc700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb72000c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:27.472 INFO:tasks.workunit.client.0.smithi063.stdout:8/457: dwrite d1/f62 [0,4194304] 0 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72000a2b0 con 0x7fb72c06f0d0 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb72000a410 con 0x7fb72c06f0d0 2024-04-24T22:45:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb720012610 con 0x7fb72c06f0d0 2024-04-24T22:45:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c11d9e0 con 0x7fb72c06f0d0 2024-04-24T22:45:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.470+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c11dfa0 con 0x7fb72c06f0d0 2024-04-24T22:45:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.471+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c112140 con 0x7fb72c06f0d0 2024-04-24T22:45:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.472+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fb72001d9e0 con 0x7fb72c06f0d0 2024-04-24T22:45:27.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.473+0000 7fb72a7fc700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 0x7fb71406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:27.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.473+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb72008bf30 con 0x7fb72c06f0d0 2024-04-24T22:45:27.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.474+0000 7fb731bfd700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 0x7fb71406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:27.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.474+0000 7fb731bfd700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 0x7fb71406dd50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb71c007fd0 tx=0x7fb71c00b040 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:27.476 INFO:tasks.workunit.client.0.smithi063.stdout:3/484: creat da/d18/d9e/da3/fab x:0 0 0 2024-04-24T22:45:27.477 INFO:tasks.workunit.client.0.smithi063.stdout:9/451: dread d6/f30 [0,4194304] 0 2024-04-24T22:45:27.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:27.481+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb720059660 con 0x7fb72c06f0d0 2024-04-24T22:45:27.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/440: dwrite d5/d11/f2f [8388608,4194304] 0 2024-04-24T22:45:27.487 INFO:tasks.workunit.client.0.smithi063.stdout:4/401: dwrite d2/f7c [0,4194304] 0 2024-04-24T22:45:27.487 INFO:tasks.workunit.client.0.smithi063.stdout:1/506: creat d3/da/dc/d11/d5e/fa6 x:0 0 0 2024-04-24T22:45:27.488 INFO:tasks.workunit.client.1.smithi192.stdout:1/299: sync 2024-04-24T22:45:27.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/441: chown d5/d11/d76/f7b 13654829 1 2024-04-24T22:45:27.490 INFO:tasks.workunit.client.0.smithi063.stdout:1/507: write d3/da/f6a [1940959,82132] 0 2024-04-24T22:45:27.492 INFO:tasks.workunit.client.1.smithi192.stdout:1/300: truncate d9/d15/f1b 181542 0 2024-04-24T22:45:27.493 INFO:tasks.workunit.client.0.smithi063.stdout:5/442: write d5/d11/f2f [8766954,73062] 0 2024-04-24T22:45:27.493 INFO:tasks.workunit.client.1.smithi192.stdout:1/301: readlink d9/d15/d2d/d4a/l58 0 2024-04-24T22:45:27.493 INFO:tasks.workunit.client.0.smithi063.stdout:8/458: dread d1/d2/d17/f4b [0,4194304] 0 2024-04-24T22:45:27.493 INFO:tasks.workunit.client.0.smithi063.stdout:1/508: stat d3/da/dc/d1f/f4d 0 2024-04-24T22:45:27.494 INFO:tasks.workunit.client.0.smithi063.stdout:3/485: symlink da/lac 0 2024-04-24T22:45:27.496 INFO:tasks.workunit.client.0.smithi063.stdout:8/459: stat d1/d11/d49/d54/d7b 0 2024-04-24T22:45:27.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/509: fsync d3/da/dc/d11/f35 0 2024-04-24T22:45:27.498 INFO:tasks.workunit.client.0.smithi063.stdout:1/510: dread - d3/da/d34/d6d/f92 zero size 2024-04-24T22:45:27.500 INFO:tasks.workunit.client.0.smithi063.stdout:2/498: dread d0/d2/d2d/d31/d89/d1c/f2a [4194304,4194304] 0 2024-04-24T22:45:27.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/440: dwrite d2/db/d49/f70 [4194304,4194304] 0 2024-04-24T22:45:27.501 INFO:tasks.workunit.client.0.smithi063.stdout:2/499: chown d0/dd/d72/d77/f78 23784711 1 2024-04-24T22:45:27.503 INFO:tasks.workunit.client.1.smithi192.stdout:1/302: dread d9/f14 [0,4194304] 0 2024-04-24T22:45:27.504 INFO:tasks.workunit.client.0.smithi063.stdout:5/443: mknod d5/d24/d36/c9b 0 2024-04-24T22:45:27.505 INFO:tasks.workunit.client.1.smithi192.stdout:1/303: write d9/dd/f2b [4649457,29606] 0 2024-04-24T22:45:27.505 INFO:tasks.workunit.client.0.smithi063.stdout:8/460: dwrite d1/d2/d3/f82 [0,4194304] 0 2024-04-24T22:45:27.518 INFO:tasks.workunit.client.0.smithi063.stdout:6/483: dread db/d1d/d2d/d48/f63 [0,4194304] 0 2024-04-24T22:45:27.518 INFO:tasks.workunit.client.0.smithi063.stdout:2/500: dwrite d0/d2/d2d/d31/d68/f74 [0,4194304] 0 2024-04-24T22:45:27.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/511: symlink d3/da/dc/d39/la7 0 2024-04-24T22:45:27.522 INFO:tasks.workunit.client.0.smithi063.stdout:6/484: write db/d1c/f90 [4468770,99911] 0 2024-04-24T22:45:27.525 INFO:tasks.workunit.client.0.smithi063.stdout:2/501: write d0/d2/d27/f34 [2832117,80103] 0 2024-04-24T22:45:27.525 INFO:tasks.workunit.client.0.smithi063.stdout:5/444: rename d5/d1d/d7d/c59 to d5/d24/d2c/c9c 0 2024-04-24T22:45:27.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/393: dwrite d6/da/dc/f1e [0,4194304] 0 2024-04-24T22:45:27.529 INFO:tasks.workunit.client.0.smithi063.stdout:4/402: rmdir d2/d8/d5c/d73 0 2024-04-24T22:45:27.530 INFO:tasks.workunit.client.1.smithi192.stdout:7/441: dwrite d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:27.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/442: symlink d2/db/d21/d42/d5b/l87 0 2024-04-24T22:45:27.537 INFO:tasks.workunit.client.0.smithi063.stdout:9/452: dread d6/d8/f17 [0,4194304] 0 2024-04-24T22:45:27.539 INFO:tasks.workunit.client.1.smithi192.stdout:7/443: symlink d2/db/d3a/l88 0 2024-04-24T22:45:27.544 INFO:tasks.workunit.client.0.smithi063.stdout:6/485: dwrite db/d1c/f90 [0,4194304] 0 2024-04-24T22:45:27.544 INFO:tasks.workunit.client.0.smithi063.stdout:4/403: mkdir d2/d19/d82 0 2024-04-24T22:45:27.545 INFO:tasks.workunit.client.1.smithi192.stdout:7/444: creat d2/f89 x:0 0 0 2024-04-24T22:45:27.547 INFO:tasks.workunit.client.0.smithi063.stdout:5/445: fdatasync d5/d1d/d7d/d67/f78 0 2024-04-24T22:45:27.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/445: dread d2/db/d2a/f2b [0,4194304] 0 2024-04-24T22:45:27.552 INFO:tasks.workunit.client.0.smithi063.stdout:2/502: dread d0/d2/d2d/d31/d68/f74 [0,4194304] 0 2024-04-24T22:45:27.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/372: sync 2024-04-24T22:45:27.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/486: write db/d1d/d20/f2b [5475498,8019] 0 2024-04-24T22:45:27.554 INFO:tasks.workunit.client.0.smithi063.stdout:4/404: mknod d2/d19/d33/c83 0 2024-04-24T22:45:27.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/446: rmdir d2/db/d2a/d7a 0 2024-04-24T22:45:27.556 INFO:tasks.workunit.client.0.smithi063.stdout:1/512: dwrite d3/da/f5c [0,4194304] 0 2024-04-24T22:45:27.557 INFO:tasks.workunit.client.0.smithi063.stdout:8/461: dwrite d1/d2/fe [0,4194304] 0 2024-04-24T22:45:27.558 INFO:tasks.workunit.client.0.smithi063.stdout:5/446: creat d5/d11/d76/d92/f9d x:0 0 0 2024-04-24T22:45:27.560 INFO:tasks.workunit.client.1.smithi192.stdout:7/447: mknod d2/db/d25/d82/c8a 0 2024-04-24T22:45:27.561 INFO:tasks.workunit.client.0.smithi063.stdout:9/453: getdents d6/d8/d1f 0 2024-04-24T22:45:27.563 INFO:tasks.workunit.client.0.smithi063.stdout:8/462: readlink d1/d11/l16 0 2024-04-24T22:45:27.564 INFO:tasks.workunit.client.0.smithi063.stdout:2/503: symlink d0/d2/d6c/l97 0 2024-04-24T22:45:27.566 INFO:tasks.workunit.client.0.smithi063.stdout:6/487: chown db/d1d/d20/d3f/d5f/d93/f54 2 1 2024-04-24T22:45:27.568 INFO:tasks.workunit.client.0.smithi063.stdout:0/394: dread d6/dd/f4d [0,4194304] 0 2024-04-24T22:45:27.569 INFO:tasks.workunit.client.1.smithi192.stdout:7/448: dwrite d2/db/d2a/f2e [0,4194304] 0 2024-04-24T22:45:27.569 INFO:tasks.workunit.client.0.smithi063.stdout:7/373: rename d0/f4f to d0/d4/de/d6f/d5c/d63/f84 0 2024-04-24T22:45:27.572 INFO:tasks.workunit.client.0.smithi063.stdout:3/486: dread da/f3b [0,4194304] 0 2024-04-24T22:45:27.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/449: write d2/db/f6f [564671,79174] 0 2024-04-24T22:45:27.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/447: readlink d5/d24/d49/l99 0 2024-04-24T22:45:27.576 INFO:tasks.workunit.client.0.smithi063.stdout:6/488: symlink db/d1d/la3 0 2024-04-24T22:45:27.578 INFO:tasks.workunit.client.0.smithi063.stdout:9/454: creat d6/d8/d1f/d20/d2a/d4b/d5e/d8e/f90 x:0 0 0 2024-04-24T22:45:27.579 INFO:tasks.workunit.client.0.smithi063.stdout:4/405: dwrite d2/d8/d36/d3a/d50/f77 [0,4194304] 0 2024-04-24T22:45:27.581 INFO:tasks.workunit.client.0.smithi063.stdout:7/374: unlink d0/d4/de/f21 0 2024-04-24T22:45:27.588 INFO:tasks.workunit.client.0.smithi063.stdout:1/513: dwrite d3/f63 [0,4194304] 0 2024-04-24T22:45:27.588 INFO:tasks.workunit.client.0.smithi063.stdout:9/455: mknod d6/d8/db/d2b/c91 0 2024-04-24T22:45:27.591 INFO:tasks.workunit.client.0.smithi063.stdout:0/395: truncate d6/d64/d21/d25/d62/d75/f61 879627 0 2024-04-24T22:45:27.592 INFO:tasks.workunit.client.0.smithi063.stdout:2/504: dwrite d0/f40 [0,4194304] 0 2024-04-24T22:45:27.596 INFO:tasks.workunit.client.0.smithi063.stdout:7/375: creat d0/d4/de/d6f/d2e/f85 x:0 0 0 2024-04-24T22:45:27.597 INFO:tasks.workunit.client.0.smithi063.stdout:4/406: creat d2/d19/d82/f84 x:0 0 0 2024-04-24T22:45:27.598 INFO:tasks.workunit.client.0.smithi063.stdout:1/514: creat d3/da/d34/d68/fa8 x:0 0 0 2024-04-24T22:45:27.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/505: chown d0/d2/d39/l45 3 1 2024-04-24T22:45:27.599 INFO:tasks.workunit.client.0.smithi063.stdout:9/456: mknod d6/d8/d1f/d20/d2a/c92 0 2024-04-24T22:45:27.603 INFO:tasks.workunit.client.1.smithi192.stdout:7/450: truncate d2/db/d21/f28 2405579 0 2024-04-24T22:45:27.604 INFO:tasks.workunit.client.1.smithi192.stdout:7/451: chown d2/db/f50 417763 1 2024-04-24T22:45:27.607 INFO:tasks.workunit.client.0.smithi063.stdout:2/506: unlink d0/d2/d35/f82 0 2024-04-24T22:45:27.607 INFO:tasks.workunit.client.0.smithi063.stdout:0/396: mknod d6/d64/d21/d25/c8a 0 2024-04-24T22:45:27.608 INFO:tasks.workunit.client.1.smithi192.stdout:7/452: mknod d2/db/d2a/c8b 0 2024-04-24T22:45:27.610 INFO:tasks.workunit.client.0.smithi063.stdout:3/487: link da/d18/d47/d4d/d62/l93 da/lad 0 2024-04-24T22:45:27.611 INFO:tasks.workunit.client.1.smithi192.stdout:7/453: rmdir d2/db/d21/d42/d5b/d63 39 2024-04-24T22:45:27.612 INFO:tasks.workunit.client.0.smithi063.stdout:5/448: write d5/d11/d23/f69 [665283,125151] 0 2024-04-24T22:45:27.614 INFO:tasks.workunit.client.0.smithi063.stdout:4/407: unlink d2/d8/c28 0 2024-04-24T22:45:27.615 INFO:tasks.workunit.client.0.smithi063.stdout:1/515: dread d3/da/dc/f3a [0,4194304] 0 2024-04-24T22:45:27.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/457: mkdir d6/d8/db/d93 0 2024-04-24T22:45:27.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/488: creat da/d18/d47/d4d/fae x:0 0 0 2024-04-24T22:45:27.618 INFO:tasks.workunit.client.0.smithi063.stdout:2/507: write d0/d2/d2d/d31/d89/d48/f5d [511268,53790] 0 2024-04-24T22:45:27.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/463: dread d1/d2/d3/f59 [0,4194304] 0 2024-04-24T22:45:27.621 INFO:tasks.workunit.client.0.smithi063.stdout:5/449: mknod d5/d24/d49/c9e 0 2024-04-24T22:45:27.622 INFO:tasks.workunit.client.0.smithi063.stdout:4/408: write d2/d8/d36/d3a/f7f [973222,107300] 0 2024-04-24T22:45:27.623 INFO:tasks.workunit.client.0.smithi063.stdout:1/516: readlink d3/da/d34/d68/l70 0 2024-04-24T22:45:27.627 INFO:tasks.workunit.client.0.smithi063.stdout:3/489: write da/f3b [3844348,42087] 0 2024-04-24T22:45:27.627 INFO:tasks.workunit.client.0.smithi063.stdout:0/397: dwrite d6/d64/d21/d25/d62/d63/f82 [0,4194304] 0 2024-04-24T22:45:27.628 INFO:tasks.workunit.client.0.smithi063.stdout:6/489: link db/d1d/d20/d3f/d5f/c98 db/d1d/d20/d3f/d94/ca4 0 2024-04-24T22:45:27.630 INFO:tasks.workunit.client.0.smithi063.stdout:9/458: mkdir d6/d8/db/d2b/d48/d94 0 2024-04-24T22:45:27.630 INFO:tasks.workunit.client.0.smithi063.stdout:4/409: symlink d2/d80/l85 0 2024-04-24T22:45:27.632 INFO:tasks.workunit.client.0.smithi063.stdout:6/490: dread - db/d1d/d20/d3f/d7f/d68/f6d zero size 2024-04-24T22:45:27.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/459: fdatasync d6/d8/db/d6b/f8f 0 2024-04-24T22:45:27.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/517: symlink d3/da/dc/d1f/d21/d42/d6e/la9 0 2024-04-24T22:45:27.637 INFO:tasks.workunit.client.0.smithi063.stdout:0/398: creat d6/d64/d24/f8b x:0 0 0 2024-04-24T22:45:27.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/376: dread d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:27.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/508: write d0/d2/d2d/d31/d89/d7/f64 [360818,69700] 0 2024-04-24T22:45:27.641 INFO:tasks.workunit.client.0.smithi063.stdout:5/450: write d5/f5b [1927082,72408] 0 2024-04-24T22:45:27.641 INFO:tasks.workunit.client.0.smithi063.stdout:3/490: creat da/d39/d94/faf x:0 0 0 2024-04-24T22:45:27.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/410: truncate d2/f35 823427 0 2024-04-24T22:45:27.644 INFO:tasks.workunit.client.0.smithi063.stdout:8/464: dwrite d1/d2/d3/f59 [0,4194304] 0 2024-04-24T22:45:27.644 INFO:tasks.workunit.client.0.smithi063.stdout:6/491: symlink db/d85/la5 0 2024-04-24T22:45:27.648 INFO:tasks.workunit.client.0.smithi063.stdout:7/377: write d0/d2f/f37 [2984671,20595] 0 2024-04-24T22:45:27.649 INFO:tasks.workunit.client.0.smithi063.stdout:9/460: creat d6/d2f/d42/d81/f95 x:0 0 0 2024-04-24T22:45:27.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/518: mkdir d3/da/dc/d1f/d21/d9f/daa 0 2024-04-24T22:45:27.651 INFO:tasks.workunit.client.0.smithi063.stdout:3/491: symlink da/d18/d47/d4d/lb0 0 2024-04-24T22:45:27.651 INFO:tasks.workunit.client.0.smithi063.stdout:8/465: mknod d1/d11/d3c/c95 0 2024-04-24T22:45:27.652 INFO:tasks.workunit.client.0.smithi063.stdout:6/492: mkdir db/d1d/d20/d29/d5d/da6 0 2024-04-24T22:45:27.653 INFO:tasks.workunit.client.0.smithi063.stdout:2/509: creat d0/dd/d72/d77/d87/f98 x:0 0 0 2024-04-24T22:45:27.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/368: sync 2024-04-24T22:45:27.655 INFO:tasks.workunit.client.0.smithi063.stdout:4/411: rename d2/d8/d3c/d6f to d2/d8/d40/d51/d86 0 2024-04-24T22:45:27.656 INFO:tasks.workunit.client.0.smithi063.stdout:4/412: stat d2/fd 0 2024-04-24T22:45:27.656 INFO:tasks.workunit.client.0.smithi063.stdout:7/378: mkdir d0/d4/de/d17/d35/d69/d86 0 2024-04-24T22:45:27.658 INFO:tasks.workunit.client.0.smithi063.stdout:9/461: creat d6/d2f/d42/d81/f96 x:0 0 0 2024-04-24T22:45:27.659 INFO:tasks.workunit.client.0.smithi063.stdout:0/399: link d6/d64/d21/c57 d6/d64/d21/d25/d62/d63/d89/c8c 0 2024-04-24T22:45:27.664 INFO:tasks.workunit.client.0.smithi063.stdout:6/493: readlink db/d1d/d20/d3f/d5f/d93/l57 0 2024-04-24T22:45:27.665 INFO:tasks.workunit.client.0.smithi063.stdout:8/466: rename d1/d21/c5e to d1/d2/d3/d86/c96 0 2024-04-24T22:45:27.666 INFO:tasks.workunit.client.0.smithi063.stdout:2/510: dwrite d0/d2/d2f/f58 [0,4194304] 0 2024-04-24T22:45:27.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/379: mknod d0/d4/de/d6f/d5c/c87 0 2024-04-24T22:45:27.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/467: readlink d1/l53 0 2024-04-24T22:45:27.670 INFO:tasks.workunit.client.0.smithi063.stdout:1/519: dwrite d3/fd [4194304,4194304] 0 2024-04-24T22:45:27.671 INFO:tasks.workunit.client.1.smithi192.stdout:3/369: truncate d2/d4/d7/fd 1608433 0 2024-04-24T22:45:27.672 INFO:tasks.workunit.client.1.smithi192.stdout:3/370: chown d2/d27/f4a 222 1 2024-04-24T22:45:27.676 INFO:tasks.workunit.client.0.smithi063.stdout:5/451: dread d5/d24/d2c/f3e [0,4194304] 0 2024-04-24T22:45:27.677 INFO:tasks.workunit.client.0.smithi063.stdout:9/462: dwrite d6/d8/db/f62 [0,4194304] 0 2024-04-24T22:45:27.677 INFO:tasks.workunit.client.1.smithi192.stdout:3/371: unlink d2/d62/l6c 0 2024-04-24T22:45:27.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/413: mknod d2/d8/c87 0 2024-04-24T22:45:27.678 INFO:tasks.workunit.client.0.smithi063.stdout:8/468: write d1/d11/d3c/f55 [25218,67861] 0 2024-04-24T22:45:27.679 INFO:tasks.workunit.client.0.smithi063.stdout:6/494: creat db/d1c/fa7 x:0 0 0 2024-04-24T22:45:27.682 INFO:tasks.workunit.client.0.smithi063.stdout:5/452: write d5/d11/d76/f7b [5013214,49972] 0 2024-04-24T22:45:27.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/520: rename d3/da/dc/d11/d5e/d6f to d3/da/dc/d11/d5e/d6f/dab 22 2024-04-24T22:45:27.682 INFO:tasks.workunit.client.0.smithi063.stdout:2/511: creat d0/dd/f99 x:0 0 0 2024-04-24T22:45:27.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/492: write da/d39/f41 [3766740,37092] 0 2024-04-24T22:45:27.684 INFO:tasks.workunit.client.0.smithi063.stdout:0/400: write d6/da/d2b/f40 [668187,24228] 0 2024-04-24T22:45:27.687 INFO:tasks.workunit.client.0.smithi063.stdout:2/512: write d0/d2/d27/f85 [4265593,97217] 0 2024-04-24T22:45:27.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/463: creat d6/d8/db/d1c/f97 x:0 0 0 2024-04-24T22:45:27.688 INFO:tasks.workunit.client.0.smithi063.stdout:6/495: fdatasync db/d1d/d2d/f76 0 2024-04-24T22:45:27.692 INFO:tasks.workunit.client.0.smithi063.stdout:2/513: fdatasync d0/d2/d27/f29 0 2024-04-24T22:45:27.692 INFO:tasks.workunit.client.0.smithi063.stdout:8/469: truncate d1/d21/f61 1228254 0 2024-04-24T22:45:27.694 INFO:tasks.workunit.client.1.smithi192.stdout:3/372: write d2/d4/d12/d1a/d1d/d55/f32 [1306595,44564] 0 2024-04-24T22:45:27.694 INFO:tasks.workunit.client.0.smithi063.stdout:2/514: stat d0/d2/d2d/d31/f5b 0 2024-04-24T22:45:27.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/493: chown da/f14 1680242 1 2024-04-24T22:45:27.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/470: chown d1/d2/f1c 237 1 2024-04-24T22:45:27.695 INFO:tasks.workunit.client.0.smithi063.stdout:1/521: rmdir d3/da/dc/d1f/d21 39 2024-04-24T22:45:27.696 INFO:tasks.workunit.client.0.smithi063.stdout:6/496: symlink db/d1d/d20/d3f/d5f/d93/la8 0 2024-04-24T22:45:27.697 INFO:tasks.workunit.client.0.smithi063.stdout:8/471: dread - d1/d11/d8b/f8d zero size 2024-04-24T22:45:27.698 INFO:tasks.workunit.client.0.smithi063.stdout:6/497: stat db/d1d/d20/d3f/d7f 0 2024-04-24T22:45:27.698 INFO:tasks.workunit.client.0.smithi063.stdout:1/522: chown d3/da/dc/d1f/d2b/d79/d96 61544 1 2024-04-24T22:45:27.699 INFO:tasks.workunit.client.0.smithi063.stdout:7/380: truncate d0/f73 514192 0 2024-04-24T22:45:27.700 INFO:tasks.workunit.client.0.smithi063.stdout:2/515: truncate d0/dd/f49 298539 0 2024-04-24T22:45:27.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/516: write d0/dd/f75 [154749,54049] 0 2024-04-24T22:45:27.704 INFO:tasks.workunit.client.0.smithi063.stdout:5/453: rename d5/d24/d49/l99 to d5/d11/d23/l9f 0 2024-04-24T22:45:27.704 INFO:tasks.workunit.client.0.smithi063.stdout:5/454: chown d5/d24/d2c 4 1 2024-04-24T22:45:27.705 INFO:tasks.workunit.client.1.smithi192.stdout:3/373: creat d2/d4/d7/f77 x:0 0 0 2024-04-24T22:45:27.708 INFO:tasks.workunit.client.0.smithi063.stdout:1/523: fdatasync d3/da/dc/d39/f98 0 2024-04-24T22:45:27.709 INFO:tasks.workunit.client.1.smithi192.stdout:3/374: creat d2/d4/d12/d39/d3a/f78 x:0 0 0 2024-04-24T22:45:27.711 INFO:tasks.workunit.client.0.smithi063.stdout:7/381: fsync d0/d4/f19 0 2024-04-24T22:45:27.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/375: mknod d2/d4/d12/d65/c79 0 2024-04-24T22:45:27.715 INFO:tasks.workunit.client.0.smithi063.stdout:2/517: symlink d0/d2/d63/l9a 0 2024-04-24T22:45:27.716 INFO:tasks.workunit.client.0.smithi063.stdout:9/464: rename d6/f6c to d6/d8/d1f/d20/d2a/d4b/d5e/d8e/f98 0 2024-04-24T22:45:27.717 INFO:tasks.workunit.client.1.smithi192.stdout:3/376: dread d2/d27/f4a [0,4194304] 0 2024-04-24T22:45:27.719 INFO:tasks.workunit.client.0.smithi063.stdout:9/465: dread d6/d8/db/d6b/d4f/d59/f76 [0,4194304] 0 2024-04-24T22:45:27.720 INFO:tasks.workunit.client.0.smithi063.stdout:9/466: fsync d6/d8/d1f/d20/d22/d31/f68 0 2024-04-24T22:45:27.721 INFO:tasks.workunit.client.0.smithi063.stdout:9/467: read - d6/d8/db/d1c/f97 zero size 2024-04-24T22:45:27.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/498: mknod db/d1d/d20/d29/d5d/da6/ca9 0 2024-04-24T22:45:27.724 INFO:tasks.workunit.client.0.smithi063.stdout:7/382: mknod d0/d4/c88 0 2024-04-24T22:45:27.724 INFO:tasks.workunit.client.1.smithi192.stdout:3/377: dwrite d2/d4/d12/d39/d3a/f61 [0,4194304] 0 2024-04-24T22:45:27.727 INFO:tasks.workunit.client.1.smithi192.stdout:3/378: creat d2/d4/d7/d4f/f7a x:0 0 0 2024-04-24T22:45:27.727 INFO:tasks.workunit.client.0.smithi063.stdout:9/468: dread d6/d8/db/d2b/f3d [0,4194304] 0 2024-04-24T22:45:27.730 INFO:tasks.workunit.client.0.smithi063.stdout:5/455: mknod d5/d11/ca0 0 2024-04-24T22:45:27.730 INFO:tasks.workunit.client.0.smithi063.stdout:4/414: dread d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:27.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/379: mkdir d2/d4/d12/d1a/d1d/d7b 0 2024-04-24T22:45:27.733 INFO:tasks.workunit.client.0.smithi063.stdout:5/456: stat d5 0 2024-04-24T22:45:27.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/380: truncate d2/d4/f3e 4522770 0 2024-04-24T22:45:27.734 INFO:tasks.workunit.client.0.smithi063.stdout:1/524: write d3/da/dc/d11/f46 [4418763,45494] 0 2024-04-24T22:45:27.736 INFO:tasks.workunit.client.0.smithi063.stdout:2/518: mknod d0/d2/c9b 0 2024-04-24T22:45:27.737 INFO:tasks.workunit.client.0.smithi063.stdout:6/499: dwrite db/d1d/d2d/d69/d82/f9a [0,4194304] 0 2024-04-24T22:45:27.739 INFO:tasks.workunit.client.0.smithi063.stdout:2/519: write d0/d2/d2d/d31/f5b [1300903,83872] 0 2024-04-24T22:45:27.740 INFO:tasks.workunit.client.0.smithi063.stdout:0/401: rename d6/d64/d21/d25/d62/d63/d89/c8c to d6/d3a/c8d 0 2024-04-24T22:45:27.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/520: chown d0/d2/d2d/d31/d89/d7/d86/d8e 4074 1 2024-04-24T22:45:27.742 INFO:tasks.workunit.client.0.smithi063.stdout:9/469: creat d6/d8/d53/f99 x:0 0 0 2024-04-24T22:45:27.745 INFO:tasks.workunit.client.0.smithi063.stdout:3/494: getdents da/d40 0 2024-04-24T22:45:27.745 INFO:tasks.workunit.client.0.smithi063.stdout:9/470: write d6/d8/db/d1c/f97 [385960,110824] 0 2024-04-24T22:45:27.746 INFO:tasks.workunit.client.0.smithi063.stdout:4/415: dread - d2/d8/f5b zero size 2024-04-24T22:45:27.748 INFO:tasks.workunit.client.0.smithi063.stdout:1/525: creat d3/da/dc/d1f/d2b/d79/fac x:0 0 0 2024-04-24T22:45:27.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/471: dread d6/d8/db/d6b/d4f/d59/f76 [0,4194304] 0 2024-04-24T22:45:27.750 INFO:tasks.workunit.client.0.smithi063.stdout:6/500: symlink db/laa 0 2024-04-24T22:45:27.752 INFO:tasks.workunit.client.1.smithi192.stdout:3/381: dwrite d2/d4/d7/f2c [0,4194304] 0 2024-04-24T22:45:27.753 INFO:tasks.workunit.client.0.smithi063.stdout:2/521: dwrite d0/d2/d27/f29 [0,4194304] 0 2024-04-24T22:45:27.753 INFO:tasks.workunit.client.0.smithi063.stdout:7/383: rename d0/d4/de/d6f/f77 to d0/d4/de/d17/d35/d69/f89 0 2024-04-24T22:45:27.754 INFO:tasks.workunit.client.0.smithi063.stdout:0/402: mknod d6/da/dc/d26/d33/c8e 0 2024-04-24T22:45:27.758 INFO:tasks.workunit.client.0.smithi063.stdout:1/526: unlink d3/da/dc/d1f/d2b/d79/c86 0 2024-04-24T22:45:27.759 INFO:tasks.workunit.client.1.smithi192.stdout:3/382: rename d2/d4/d12/d39/d3a/f61 to d2/d4/d7/f7c 0 2024-04-24T22:45:27.759 INFO:tasks.workunit.client.0.smithi063.stdout:9/472: mkdir d6/d8/db/d1c/d78/d9a 0 2024-04-24T22:45:27.764 INFO:tasks.workunit.client.0.smithi063.stdout:0/403: read d6/da/f14 [4729737,120504] 0 2024-04-24T22:45:27.764 INFO:tasks.workunit.client.0.smithi063.stdout:2/522: unlink d0/dd/d72/c93 0 2024-04-24T22:45:27.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/416: dwrite d2/f5 [4194304,4194304] 0 2024-04-24T22:45:27.770 INFO:tasks.workunit.client.1.smithi192.stdout:3/383: dread d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:27.770 INFO:tasks.workunit.client.0.smithi063.stdout:9/473: chown d6/d8/db/d1c/c32 7190 1 2024-04-24T22:45:27.771 INFO:tasks.workunit.client.0.smithi063.stdout:5/457: getdents d5/d11/d23/d2d/d62/d73 0 2024-04-24T22:45:27.772 INFO:tasks.workunit.client.1.smithi192.stdout:3/384: fsync d2/d27/d28/f36 0 2024-04-24T22:45:27.773 INFO:tasks.workunit.client.0.smithi063.stdout:7/384: getdents d0/d4/de/d6f/d2e/d55 0 2024-04-24T22:45:27.776 INFO:tasks.workunit.client.0.smithi063.stdout:0/404: creat d6/d64/d21/d53/f8f x:0 0 0 2024-04-24T22:45:27.778 INFO:tasks.workunit.client.0.smithi063.stdout:9/474: creat d6/d8/db/d6b/d4f/d59/f9b x:0 0 0 2024-04-24T22:45:27.778 INFO:tasks.workunit.client.0.smithi063.stdout:4/417: dread d2/d8/d36/d3a/f7f [0,4194304] 0 2024-04-24T22:45:27.778 INFO:tasks.workunit.client.0.smithi063.stdout:5/458: chown d5/d24/d44/f5c 3249857 1 2024-04-24T22:45:27.781 INFO:tasks.workunit.client.0.smithi063.stdout:7/385: getdents d0/d4/de/d17/d35/d69/d86 0 2024-04-24T22:45:27.783 INFO:tasks.workunit.client.0.smithi063.stdout:9/475: fdatasync d6/d8/d53/f7a 0 2024-04-24T22:45:27.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/527: dwrite d3/da/dc/d11/fa5 [0,4194304] 0 2024-04-24T22:45:27.785 INFO:tasks.workunit.client.0.smithi063.stdout:7/386: fdatasync d0/d4/fb 0 2024-04-24T22:45:27.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/501: dread db/d1d/d2d/d48/f51 [0,4194304] 0 2024-04-24T22:45:27.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/502: readlink db/l31 0 2024-04-24T22:45:27.790 INFO:tasks.workunit.client.0.smithi063.stdout:9/476: write d6/d8/d1f/d20/d2a/d4b/d5e/d8e/f98 [104866,110566] 0 2024-04-24T22:45:27.794 INFO:tasks.workunit.client.0.smithi063.stdout:4/418: rename d2/d19/d33/c4c to d2/d8/c88 0 2024-04-24T22:45:27.795 INFO:tasks.workunit.client.0.smithi063.stdout:0/405: link d6/da/d2b/l34 d6/da/l90 0 2024-04-24T22:45:27.796 INFO:tasks.workunit.client.0.smithi063.stdout:5/459: link d5/d1d/d7d/l61 d5/d11/d23/d2d/d62/la1 0 2024-04-24T22:45:27.798 INFO:tasks.workunit.client.0.smithi063.stdout:1/528: fsync d3/da/dc/d1f/d21/f69 0 2024-04-24T22:45:27.802 INFO:tasks.workunit.client.0.smithi063.stdout:0/406: mkdir d6/d64/d21/d25/d62/d63/d91 0 2024-04-24T22:45:27.808 INFO:tasks.workunit.client.0.smithi063.stdout:2/523: dread d0/dd/f22 [0,4194304] 0 2024-04-24T22:45:27.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/385: dread d2/d4/d12/f49 [0,4194304] 0 2024-04-24T22:45:27.812 INFO:tasks.workunit.client.0.smithi063.stdout:5/460: dwrite d5/d1d/f64 [0,4194304] 0 2024-04-24T22:45:27.812 INFO:tasks.workunit.client.0.smithi063.stdout:9/477: dwrite d6/d8/f5b [0,4194304] 0 2024-04-24T22:45:27.820 INFO:tasks.workunit.client.0.smithi063.stdout:6/503: dread f3 [0,4194304] 0 2024-04-24T22:45:27.821 INFO:tasks.workunit.client.0.smithi063.stdout:7/387: dread d0/d4/fb [0,4194304] 0 2024-04-24T22:45:27.822 INFO:tasks.workunit.client.0.smithi063.stdout:1/529: link d3/da/dc/d11/l64 d3/da/dc/d39/d5a/lad 0 2024-04-24T22:45:27.823 INFO:tasks.workunit.client.0.smithi063.stdout:9/478: mknod d6/d8/db/d1c/d78/c9c 0 2024-04-24T22:45:27.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/407: dwrite d6/da/dc/d26/f5f [0,4194304] 0 2024-04-24T22:45:27.828 INFO:tasks.workunit.client.0.smithi063.stdout:5/461: creat d5/d24/d44/d71/fa2 x:0 0 0 2024-04-24T22:45:27.828 INFO:tasks.workunit.client.0.smithi063.stdout:1/530: truncate d3/da/dc/f8b 4435449 0 2024-04-24T22:45:27.830 INFO:tasks.workunit.client.0.smithi063.stdout:6/504: mkdir db/d1d/d20/d29/dab 0 2024-04-24T22:45:27.832 INFO:tasks.workunit.client.0.smithi063.stdout:6/505: chown db/d1d/d20/d3f/d7f/d68/d70 126496927 1 2024-04-24T22:45:27.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/479: mkdir d6/d8/d1f/d20/d2a/d4b/d5e/d9d 0 2024-04-24T22:45:27.835 INFO:tasks.workunit.client.0.smithi063.stdout:7/388: creat d0/d4/de/d17/f8a x:0 0 0 2024-04-24T22:45:27.837 INFO:tasks.workunit.client.0.smithi063.stdout:4/419: dread d2/d8/d5c/f67 [0,4194304] 0 2024-04-24T22:45:27.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/408: creat d6/d64/f92 x:0 0 0 2024-04-24T22:45:27.839 INFO:tasks.workunit.client.0.smithi063.stdout:5/462: read d5/d11/d18/f19 [86940,57827] 0 2024-04-24T22:45:27.842 INFO:tasks.workunit.client.0.smithi063.stdout:7/389: mkdir d0/d4/de/d6f/d5c/d8b 0 2024-04-24T22:45:27.842 INFO:tasks.workunit.client.0.smithi063.stdout:1/531: dwrite d3/da/dc/d11/f89 [0,4194304] 0 2024-04-24T22:45:27.844 INFO:tasks.workunit.client.0.smithi063.stdout:5/463: write d5/d11/d76/f7b [4513625,24311] 0 2024-04-24T22:45:27.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/506: creat db/d1d/d20/d3f/d7f/fac x:0 0 0 2024-04-24T22:45:27.846 INFO:tasks.workunit.client.0.smithi063.stdout:2/524: dread d0/d2/d2d/d31/d89/d7/f64 [0,4194304] 0 2024-04-24T22:45:27.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/420: creat d2/d8/d40/d51/d86/f89 x:0 0 0 2024-04-24T22:45:27.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/507: write db/d1d/d2d/f42 [3751501,60261] 0 2024-04-24T22:45:27.850 INFO:tasks.workunit.client.0.smithi063.stdout:1/532: unlink d3/da/dc/c16 0 2024-04-24T22:45:27.854 INFO:tasks.workunit.client.0.smithi063.stdout:7/390: unlink d0/d4/de/d17/d35/d69/f89 0 2024-04-24T22:45:27.857 INFO:tasks.workunit.client.0.smithi063.stdout:4/421: rename d2/d8/d3c/f72 to d2/d80/f8a 0 2024-04-24T22:45:27.859 INFO:tasks.workunit.client.0.smithi063.stdout:5/464: dwrite d5/d11/d23/f79 [0,4194304] 0 2024-04-24T22:45:27.861 INFO:tasks.workunit.client.0.smithi063.stdout:1/533: symlink d3/da/dc/d39/d5a/lae 0 2024-04-24T22:45:27.866 INFO:tasks.workunit.client.0.smithi063.stdout:4/422: read - d2/d8/d40/d51/f69 zero size 2024-04-24T22:45:27.867 INFO:tasks.workunit.client.0.smithi063.stdout:0/409: write d6/dd/f3f [1210165,17956] 0 2024-04-24T22:45:27.867 INFO:tasks.workunit.client.0.smithi063.stdout:2/525: dwrite d0/dd/f99 [0,4194304] 0 2024-04-24T22:45:27.871 INFO:tasks.workunit.client.1.smithi192.stdout:8/349: sync 2024-04-24T22:45:27.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/350: write db/dc/f5b [16174,37531] 0 2024-04-24T22:45:27.877 INFO:tasks.workunit.client.0.smithi063.stdout:7/391: dwrite d0/d4/de/d17/d35/f79 [0,4194304] 0 2024-04-24T22:45:27.882 INFO:tasks.workunit.client.0.smithi063.stdout:9/480: dread d6/d8/db/d2b/f34 [0,4194304] 0 2024-04-24T22:45:27.884 INFO:tasks.workunit.client.0.smithi063.stdout:2/526: dwrite d0/d2/d2d/f4d [0,4194304] 0 2024-04-24T22:45:27.885 INFO:tasks.workunit.client.1.smithi192.stdout:8/351: rename db/dc/l20 to db/dc/l6a 0 2024-04-24T22:45:27.887 INFO:tasks.workunit.client.0.smithi063.stdout:4/423: mknod d2/d8/c8b 0 2024-04-24T22:45:27.889 INFO:tasks.workunit.client.0.smithi063.stdout:7/392: rmdir d0/d4/de/d17/d35/d69/d86 0 2024-04-24T22:45:27.895 INFO:tasks.workunit.client.0.smithi063.stdout:4/424: unlink d2/d8/d36/d3a/d50/l54 0 2024-04-24T22:45:27.896 INFO:tasks.workunit.client.0.smithi063.stdout:7/393: mknod d0/d4/de/d17/d35/c8c 0 2024-04-24T22:45:27.898 INFO:tasks.workunit.client.0.smithi063.stdout:9/481: symlink d6/d8/d1f/d20/d2a/d4b/d5e/d9d/l9e 0 2024-04-24T22:45:27.899 INFO:tasks.workunit.client.0.smithi063.stdout:4/425: symlink d2/d80/l8c 0 2024-04-24T22:45:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:7/394: rename d0/f3d to d0/d4/de/f8d 0 2024-04-24T22:45:27.907 INFO:tasks.workunit.client.0.smithi063.stdout:5/465: dread d5/d24/f3d [0,4194304] 0 2024-04-24T22:45:27.908 INFO:tasks.workunit.client.0.smithi063.stdout:9/482: dwrite d6/d8/f71 [0,4194304] 0 2024-04-24T22:45:27.912 INFO:tasks.workunit.client.0.smithi063.stdout:6/508: dread db/d1d/d20/d3e/f5e [0,4194304] 0 2024-04-24T22:45:27.912 INFO:tasks.workunit.client.0.smithi063.stdout:9/483: fsync d6/d8/d53/f88 0 2024-04-24T22:45:27.912 INFO:tasks.workunit.client.0.smithi063.stdout:0/410: dwrite d6/dd/f54 [0,4194304] 0 2024-04-24T22:45:27.913 INFO:tasks.workunit.client.0.smithi063.stdout:1/534: dread d3/da/dc/f8b [0,4194304] 0 2024-04-24T22:45:27.915 INFO:tasks.workunit.client.0.smithi063.stdout:5/466: mkdir d5/d1d/da3 0 2024-04-24T22:45:27.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/426: truncate d2/d8/d5c/f67 1669980 0 2024-04-24T22:45:27.917 INFO:tasks.workunit.client.0.smithi063.stdout:9/484: write d6/d8/db/d2b/d36/f40 [1031813,72993] 0 2024-04-24T22:45:27.918 INFO:tasks.workunit.client.0.smithi063.stdout:7/395: fsync d0/d4/de/f12 0 2024-04-24T22:45:27.921 INFO:tasks.workunit.client.0.smithi063.stdout:5/467: rmdir d5/d24/d2c 39 2024-04-24T22:45:27.922 INFO:tasks.workunit.client.0.smithi063.stdout:2/527: dread d0/d2/d2d/f81 [0,4194304] 0 2024-04-24T22:45:27.923 INFO:tasks.workunit.client.0.smithi063.stdout:5/468: stat d5/ff 0 2024-04-24T22:45:27.928 INFO:tasks.workunit.client.0.smithi063.stdout:4/427: rename d2/d8/d40/d51/d86 to d2/d80/d8d 0 2024-04-24T22:45:27.930 INFO:tasks.workunit.client.0.smithi063.stdout:5/469: read d5/d11/f2f [4363885,19076] 0 2024-04-24T22:45:27.936 INFO:tasks.workunit.client.0.smithi063.stdout:7/396: dwrite d0/d4/de/d6f/f53 [4194304,4194304] 0 2024-04-24T22:45:27.936 INFO:tasks.workunit.client.0.smithi063.stdout:2/528: creat d0/dd/d72/d77/d87/f9c x:0 0 0 2024-04-24T22:45:27.937 INFO:tasks.workunit.client.0.smithi063.stdout:1/535: link d3/da/d34/f83 d3/da/dc/d11/d5e/d6f/faf 0 2024-04-24T22:45:27.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/470: truncate d5/d11/d23/f27 730306 0 2024-04-24T22:45:27.939 INFO:tasks.workunit.client.0.smithi063.stdout:6/509: dread db/d1d/d2d/f42 [0,4194304] 0 2024-04-24T22:45:27.946 INFO:tasks.workunit.client.0.smithi063.stdout:5/471: chown d5/d11/d23/d2d/d62/la1 62389 1 2024-04-24T22:45:27.947 INFO:tasks.workunit.client.0.smithi063.stdout:1/536: mknod d3/da/dc/d11/d5e/d9c/cb0 0 2024-04-24T22:45:27.947 INFO:tasks.workunit.client.0.smithi063.stdout:6/510: dread db/f2a [0,4194304] 0 2024-04-24T22:45:27.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/411: dread d6/da/dc/d26/f83 [0,4194304] 0 2024-04-24T22:45:27.949 INFO:tasks.workunit.client.0.smithi063.stdout:6/511: chown db/d1d/d20/f50 579895512 1 2024-04-24T22:45:27.949 INFO:tasks.workunit.client.0.smithi063.stdout:1/537: dread - d3/da/dc/d1f/d2b/d79/d96/f9d zero size 2024-04-24T22:45:27.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/412: dread d6/d64/d21/d25/d62/d75/f61 [0,4194304] 0 2024-04-24T22:45:27.952 INFO:tasks.workunit.client.0.smithi063.stdout:6/512: creat db/d1d/d20/d3f/d7f/d68/d70/fad x:0 0 0 2024-04-24T22:45:27.953 INFO:tasks.workunit.client.0.smithi063.stdout:9/485: dread d6/d8/db/d1c/f57 [0,4194304] 0 2024-04-24T22:45:27.954 INFO:tasks.workunit.client.0.smithi063.stdout:1/538: symlink d3/da/dc/d1f/d2b/d79/lb1 0 2024-04-24T22:45:27.954 INFO:tasks.workunit.client.0.smithi063.stdout:4/428: truncate d2/d8/f3e 137267 0 2024-04-24T22:45:27.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/397: write d0/d4/de/d17/f42 [241819,97692] 0 2024-04-24T22:45:27.958 INFO:tasks.workunit.client.0.smithi063.stdout:1/539: chown d3/da/dc/d1f/f71 741 1 2024-04-24T22:45:27.960 INFO:tasks.workunit.client.0.smithi063.stdout:9/486: symlink d6/d2f/l9f 0 2024-04-24T22:45:27.960 INFO:tasks.workunit.client.0.smithi063.stdout:6/513: creat db/d1d/d20/d84/fae x:0 0 0 2024-04-24T22:45:27.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/487: write d6/f7 [1845029,71440] 0 2024-04-24T22:45:27.963 INFO:tasks.workunit.client.0.smithi063.stdout:7/398: chown d0/d4/de/l14 0 1 2024-04-24T22:45:27.964 INFO:tasks.workunit.client.0.smithi063.stdout:1/540: rename d3/da/dc/f8b to d3/da/dc/d1f/fb2 0 2024-04-24T22:45:27.965 INFO:tasks.workunit.client.0.smithi063.stdout:2/529: dread d0/d2/f1e [0,4194304] 0 2024-04-24T22:45:27.966 INFO:tasks.workunit.client.0.smithi063.stdout:9/488: chown d6/d8/db/d6b/d4f/d59/l7f 634340286 1 2024-04-24T22:45:27.966 INFO:tasks.workunit.client.0.smithi063.stdout:1/541: write d3/f2c [3439909,60953] 0 2024-04-24T22:45:27.967 INFO:tasks.workunit.client.0.smithi063.stdout:6/514: creat db/d1d/d20/d3f/faf x:0 0 0 2024-04-24T22:45:27.970 INFO:tasks.workunit.client.0.smithi063.stdout:2/530: creat d0/d2/d2d/d31/d89/d7/f9d x:0 0 0 2024-04-24T22:45:27.970 INFO:tasks.workunit.client.0.smithi063.stdout:7/399: dread d0/d4/f19 [0,4194304] 0 2024-04-24T22:45:27.971 INFO:tasks.workunit.client.0.smithi063.stdout:6/515: creat db/d1d/d20/d3f/d7f/d68/fb0 x:0 0 0 2024-04-24T22:45:27.971 INFO:tasks.workunit.client.0.smithi063.stdout:9/489: mkdir d6/d8/db/d93/da0 0 2024-04-24T22:45:27.973 INFO:tasks.workunit.client.0.smithi063.stdout:2/531: write d0/d2/d27/f7f [196287,113311] 0 2024-04-24T22:45:27.974 INFO:tasks.workunit.client.0.smithi063.stdout:2/532: chown d0/d2/d63 2832 1 2024-04-24T22:45:27.976 INFO:tasks.workunit.client.0.smithi063.stdout:7/400: read d0/d4/de/d6f/d5c/d63/f84 [1213491,15266] 0 2024-04-24T22:45:27.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/516: getdents db/d1d/d2d/d69 0 2024-04-24T22:45:27.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/542: dwrite d3/da/d34/d68/fa8 [0,4194304] 0 2024-04-24T22:45:27.978 INFO:tasks.workunit.client.0.smithi063.stdout:2/533: write d0/d2/d27/f34 [3751017,81071] 0 2024-04-24T22:45:27.985 INFO:tasks.workunit.client.0.smithi063.stdout:7/401: mkdir d0/d4/de/d17/d35/d6c/d8e 0 2024-04-24T22:45:27.986 INFO:tasks.workunit.client.0.smithi063.stdout:2/534: symlink d0/dd/d62/l9e 0 2024-04-24T22:45:27.992 INFO:tasks.workunit.client.0.smithi063.stdout:2/535: write d0/dd/d72/d77/d87/f98 [910008,19753] 0 2024-04-24T22:45:27.993 INFO:tasks.workunit.client.0.smithi063.stdout:6/517: dwrite db/d1d/d20/d3f/d7f/d68/f6d [0,4194304] 0 2024-04-24T22:45:27.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/543: dwrite d3/da/dc/d39/f98 [4194304,4194304] 0 2024-04-24T22:45:27.997 INFO:tasks.workunit.client.0.smithi063.stdout:1/544: chown d3/da/l94 3 1 2024-04-24T22:45:27.998 INFO:tasks.workunit.client.0.smithi063.stdout:6/518: mknod db/d1d/d20/d29/dab/cb1 0 2024-04-24T22:45:28.002 INFO:tasks.workunit.client.0.smithi063.stdout:1/545: mknod d3/da/dc/cb3 0 2024-04-24T22:45:28.005 INFO:tasks.workunit.client.0.smithi063.stdout:2/536: read d0/dd/d72/d77/d87/f98 [36144,51617] 0 2024-04-24T22:45:28.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/546: unlink d3/da/dc/d11/fa5 0 2024-04-24T22:45:28.009 INFO:tasks.workunit.client.0.smithi063.stdout:2/537: read d0/d2/d2f/f58 [2505431,121497] 0 2024-04-24T22:45:28.009 INFO:tasks.workunit.client.0.smithi063.stdout:7/402: dread d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:28.010 INFO:tasks.workunit.client.0.smithi063.stdout:6/519: dwrite db/d1d/d20/d84/fae [0,4194304] 0 2024-04-24T22:45:28.012 INFO:tasks.workunit.client.0.smithi063.stdout:1/547: mkdir d3/da/dc/d1f/d21/db4 0 2024-04-24T22:45:28.015 INFO:tasks.workunit.client.0.smithi063.stdout:7/403: mknod d0/d4/de/d6f/d5c/d8b/c8f 0 2024-04-24T22:45:28.015 INFO:tasks.workunit.client.0.smithi063.stdout:2/538: fdatasync d0/d2/d2d/d31/d89/d1c/f2a 0 2024-04-24T22:45:28.016 INFO:tasks.workunit.client.0.smithi063.stdout:8/472: sync 2024-04-24T22:45:28.016 INFO:tasks.workunit.client.0.smithi063.stdout:3/495: sync 2024-04-24T22:45:28.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/404: mknod d0/d4/de/d17/c90 0 2024-04-24T22:45:28.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/405: chown d0/d4/de/d17 709400627 1 2024-04-24T22:45:28.022 INFO:tasks.workunit.client.0.smithi063.stdout:2/539: mknod d0/dd/d72/d77/c9f 0 2024-04-24T22:45:28.022 INFO:tasks.workunit.client.0.smithi063.stdout:3/496: mknod da/d18/cb1 0 2024-04-24T22:45:28.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/497: dread - da/fa2 zero size 2024-04-24T22:45:28.027 INFO:tasks.workunit.client.0.smithi063.stdout:1/548: rename d3/da/dc/d1f to d3/db5 0 2024-04-24T22:45:28.028 INFO:tasks.workunit.client.0.smithi063.stdout:7/406: dread d0/d4/de/d17/d35/f54 [0,4194304] 0 2024-04-24T22:45:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.034+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb72c10be70 con 0x7fb72c06f0d0 2024-04-24T22:45:28.036 INFO:tasks.workunit.client.0.smithi063.stdout:1/549: dwrite d3/db5/d21/f69 [0,4194304] 0 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.035+0000 7fb72a7fc700 1 -- 172.21.15.63:0/3875402426 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fb72000a580 con 0x7fb72c06f0d0 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:45:28.037 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:45:28.038 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 msgr2=0x7fb71406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 0x7fb71406dd50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb71c007fd0 tx=0x7fb71c00b040 comp rx=0 tx=0).stop 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 msgr2=0x7fb72c118870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb72000c040 comp rx=0 tx=0).stop 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 shutdown_connections 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fb71406b8a0 0x7fb71406dd50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb72c06e780 0x7fb72c118330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 --2- 172.21.15.63:0/3875402426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72c06f0d0 0x7fb72c118870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 >> 172.21.15.63:0/3875402426 conn(0x7fb72c06a360 msgr2=0x7fb72c10f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.038+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 shutdown_connections 2024-04-24T22:45:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.039+0000 7fb733e61700 1 -- 172.21.15.63:0/3875402426 wait complete. 2024-04-24T22:45:28.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/473: dread d1/d11/d3c/f55 [0,4194304] 0 2024-04-24T22:45:28.045 INFO:tasks.workunit.client.0.smithi063.stdout:8/474: dread d1/d21/f61 [0,4194304] 0 2024-04-24T22:45:28.046 INFO:tasks.workunit.client.0.smithi063.stdout:2/540: dread d0/d2/d2d/d31/d89/d7/f53 [0,4194304] 0 2024-04-24T22:45:28.047 INFO:tasks.workunit.client.0.smithi063.stdout:8/475: fsync d1/d2/f1c 0 2024-04-24T22:45:28.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/541: readlink d0/dd/l66 0 2024-04-24T22:45:28.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/476: symlink d1/d2/d17/l97 0 2024-04-24T22:45:28.051 INFO:tasks.workunit.client.0.smithi063.stdout:2/542: rename d0/d2/d6c/d7a to d0/d2/d2d/d31/d89/d7/d86/da0 0 2024-04-24T22:45:28.052 INFO:tasks.workunit.client.0.smithi063.stdout:2/543: write d0/d2/f47 [4350494,95093] 0 2024-04-24T22:45:28.054 INFO:tasks.workunit.client.0.smithi063.stdout:2/544: rmdir d0/d2/d2d/d31/d89/d48 39 2024-04-24T22:45:28.056 INFO:tasks.workunit.client.0.smithi063.stdout:2/545: write d0/d2/d2d/d31/d89/d1c/f2a [1994811,73984] 0 2024-04-24T22:45:28.058 INFO:tasks.workunit.client.1.smithi192.stdout:0/426: sync 2024-04-24T22:45:28.058 INFO:tasks.workunit.client.1.smithi192.stdout:4/296: sync 2024-04-24T22:45:28.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/427: dread - d3/d27/d28/d31/f85 zero size 2024-04-24T22:45:28.061 INFO:tasks.workunit.client.0.smithi063.stdout:1/550: dread d3/da/f6a [0,4194304] 0 2024-04-24T22:45:28.064 INFO:tasks.workunit.client.1.smithi192.stdout:0/428: symlink d3/l92 0 2024-04-24T22:45:28.065 INFO:tasks.workunit.client.1.smithi192.stdout:0/429: chown d3/d27/d3c/c5b 571368215 1 2024-04-24T22:45:28.065 INFO:tasks.workunit.client.1.smithi192.stdout:4/297: creat d2/d9/d25/d2e/d31/d36/d5b/d64/f6c x:0 0 0 2024-04-24T22:45:28.067 INFO:tasks.workunit.client.0.smithi063.stdout:1/551: rename d3/db5/f3e to d3/da/d34/d6d/fb6 0 2024-04-24T22:45:28.069 INFO:tasks.workunit.client.1.smithi192.stdout:0/430: mknod d3/d27/d28/d2e/d4a/d5c/d8a/c93 0 2024-04-24T22:45:28.070 INFO:tasks.workunit.client.0.smithi063.stdout:2/546: dwrite d0/d2/d39/f65 [0,4194304] 0 2024-04-24T22:45:28.071 INFO:tasks.workunit.client.0.smithi063.stdout:1/552: rename d3/db5/d21/d42 to d3/db5/d21/d9f/daa/db7 0 2024-04-24T22:45:28.072 INFO:tasks.workunit.client.0.smithi063.stdout:8/477: dread d1/f2c [0,4194304] 0 2024-04-24T22:45:28.074 INFO:tasks.workunit.client.1.smithi192.stdout:4/298: dwrite d2/d9/d25/f5a [0,4194304] 0 2024-04-24T22:45:28.074 INFO:tasks.workunit.client.0.smithi063.stdout:8/478: dread - d1/d11/d32/d38/f8c zero size 2024-04-24T22:45:28.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/553: symlink d3/db5/d21/d9f/daa/db7/d6e/lb8 0 2024-04-24T22:45:28.084 INFO:tasks.workunit.client.1.smithi192.stdout:6/346: sync 2024-04-24T22:45:28.084 INFO:tasks.workunit.client.1.smithi192.stdout:2/460: sync 2024-04-24T22:45:28.084 INFO:tasks.workunit.client.1.smithi192.stdout:3/386: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.1.smithi192.stdout:9/422: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.1.smithi192.stdout:5/321: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.1.smithi192.stdout:7/454: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.1.smithi192.stdout:1/304: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.1.smithi192.stdout:8/352: sync 2024-04-24T22:45:28.085 INFO:tasks.workunit.client.0.smithi063.stdout:8/479: dwrite d1/d2/fe [0,4194304] 0 2024-04-24T22:45:28.088 INFO:tasks.workunit.client.1.smithi192.stdout:7/455: stat d2/db/d21/l55 0 2024-04-24T22:45:28.091 INFO:tasks.workunit.client.1.smithi192.stdout:9/423: rename d3/d59 to d3/d59/d95 22 2024-04-24T22:45:28.095 INFO:tasks.workunit.client.1.smithi192.stdout:6/347: write d0/d8/f76 [890515,50087] 0 2024-04-24T22:45:28.096 INFO:tasks.workunit.client.1.smithi192.stdout:4/299: dread d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f34 [0,4194304] 0 2024-04-24T22:45:28.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/431: dwrite d3/d27/d28/d31/f52 [0,4194304] 0 2024-04-24T22:45:28.097 INFO:tasks.workunit.client.1.smithi192.stdout:5/322: write d0/d13/d19/d64/d4f/d52/f67 [647366,31650] 0 2024-04-24T22:45:28.098 INFO:tasks.workunit.client.1.smithi192.stdout:7/456: stat d2/db/d21/d42/d5b 0 2024-04-24T22:45:28.099 INFO:tasks.workunit.client.0.smithi063.stdout:2/547: truncate d0/d2/d2d/d31/d89/d1c/d3f/f76 3066292 0 2024-04-24T22:45:28.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/548: write d0/f69 [53234,65021] 0 2024-04-24T22:45:28.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/323: readlink d0/l8 0 2024-04-24T22:45:28.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/348: readlink d0/d8/d3d/d3e/d40/l64 0 2024-04-24T22:45:28.102 INFO:tasks.workunit.client.0.smithi063.stdout:8/480: write d1/d2/d17/f3d [1124718,7011] 0 2024-04-24T22:45:28.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/300: read d2/d9/d25/f5a [386367,87514] 0 2024-04-24T22:45:28.107 INFO:tasks.workunit.client.1.smithi192.stdout:1/305: getdents d9/d15/d2d/d4a 0 2024-04-24T22:45:28.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/481: symlink d1/d11/d49/d54/l98 0 2024-04-24T22:45:28.111 INFO:tasks.workunit.client.1.smithi192.stdout:9/424: creat d3/dd/de/d17/d6a/f96 x:0 0 0 2024-04-24T22:45:28.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/432: mkdir d3/d27/d28/d2e/d4a/d5c/d80/d94 0 2024-04-24T22:45:28.113 INFO:tasks.workunit.client.1.smithi192.stdout:1/306: chown l1 1907294453 1 2024-04-24T22:45:28.115 INFO:tasks.workunit.client.0.smithi063.stdout:2/549: creat d0/d2/fa1 x:0 0 0 2024-04-24T22:45:28.116 INFO:tasks.workunit.client.0.smithi063.stdout:2/550: read - d0/dd/d62/f6e zero size 2024-04-24T22:45:28.116 INFO:tasks.workunit.client.1.smithi192.stdout:3/387: dwrite d2/d4/d12/d1a/d1d/d55/f24 [0,4194304] 0 2024-04-24T22:45:28.117 INFO:tasks.workunit.client.0.smithi063.stdout:2/551: chown d0/dd/d72/d77/f78 9395692 1 2024-04-24T22:45:28.118 INFO:tasks.workunit.client.0.smithi063.stdout:2/552: fsync d0/d2/d2d/d31/d89/f80 0 2024-04-24T22:45:28.124 INFO:tasks.workunit.client.1.smithi192.stdout:5/324: dwrite d0/d13/d19/f60 [0,4194304] 0 2024-04-24T22:45:28.124 INFO:tasks.workunit.client.1.smithi192.stdout:4/301: fdatasync d2/f17 0 2024-04-24T22:45:28.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/433: dread - d3/d27/d28/d2e/d4a/f8c zero size 2024-04-24T22:45:28.124 INFO:tasks.workunit.client.1.smithi192.stdout:8/353: dwrite db/dc/f5b [0,4194304] 0 2024-04-24T22:45:28.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/457: dread d2/db/f45 [0,4194304] 0 2024-04-24T22:45:28.126 INFO:tasks.workunit.client.1.smithi192.stdout:9/425: dread - d3/d24/f7e zero size 2024-04-24T22:45:28.128 INFO:tasks.workunit.client.0.smithi063.stdout:2/553: creat d0/d2/d2d/d31/d89/d7/d86/fa2 x:0 0 0 2024-04-24T22:45:28.135 INFO:tasks.workunit.client.0.smithi063.stdout:2/554: creat d0/dd/d72/d77/d87/fa3 x:0 0 0 2024-04-24T22:45:28.138 INFO:tasks.workunit.client.0.smithi063.stdout:2/555: unlink d0/d2/d63/l9a 0 2024-04-24T22:45:28.139 INFO:tasks.workunit.client.0.smithi063.stdout:8/482: link d1/d11/d8b/l89 d1/d11/d49/l99 0 2024-04-24T22:45:28.141 INFO:tasks.workunit.client.0.smithi063.stdout:2/556: write d0/dd/f22 [2134937,120827] 0 2024-04-24T22:45:28.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/483: symlink d1/d11/d49/d91/l9a 0 2024-04-24T22:45:28.142 INFO:tasks.workunit.client.0.smithi063.stdout:2/557: chown d0/dd/f75 442573 1 2024-04-24T22:45:28.144 INFO:tasks.workunit.client.0.smithi063.stdout:2/558: mkdir d0/d2/d2d/d31/d89/d7/d86/da4 0 2024-04-24T22:45:28.147 INFO:tasks.workunit.client.0.smithi063.stdout:2/559: mknod d0/d2/d2d/d31/d89/d7/d86/da4/ca5 0 2024-04-24T22:45:28.150 INFO:tasks.workunit.client.1.smithi192.stdout:6/349: mknod d0/d8/d29/c80 0 2024-04-24T22:45:28.150 INFO:tasks.workunit.client.1.smithi192.stdout:8/354: stat db/dc/f1c 0 2024-04-24T22:45:28.152 INFO:tasks.workunit.client.0.smithi063.stdout:8/484: write d1/d2/f39 [977617,114432] 0 2024-04-24T22:45:28.152 INFO:tasks.workunit.client.1.smithi192.stdout:0/434: mknod d3/d27/d28/c95 0 2024-04-24T22:45:28.154 INFO:tasks.workunit.client.1.smithi192.stdout:8/355: chown db/dc/f1b 17772 1 2024-04-24T22:45:28.155 INFO:tasks.workunit.client.1.smithi192.stdout:2/461: dread d4/d9/d37/f44 [0,4194304] 0 2024-04-24T22:45:28.155 INFO:tasks.workunit.client.0.smithi063.stdout:8/485: mknod d1/d11/d49/d91/c9b 0 2024-04-24T22:45:28.156 INFO:tasks.workunit.client.0.smithi063.stdout:8/486: chown d1/d11/d49 0 1 2024-04-24T22:45:28.156 INFO:tasks.workunit.client.1.smithi192.stdout:0/435: rmdir d3/d27/d3c/d6c 39 2024-04-24T22:45:28.157 INFO:tasks.workunit.client.0.smithi063.stdout:8/487: chown d1/d11/f29 1786372 1 2024-04-24T22:45:28.157 INFO:tasks.workunit.client.0.smithi063.stdout:2/560: truncate d0/d2/d2f/f6d 8026688 0 2024-04-24T22:45:28.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/302: dwrite d2/db/f4e [0,4194304] 0 2024-04-24T22:45:28.160 INFO:tasks.workunit.client.1.smithi192.stdout:1/307: unlink d9/d15/d2d/d31/l57 0 2024-04-24T22:45:28.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/308: chown d9/d15/d2d/l59 738 1 2024-04-24T22:45:28.163 INFO:tasks.workunit.client.1.smithi192.stdout:7/458: dread d2/db/d21/d42/d5b/d63/f6d [0,4194304] 0 2024-04-24T22:45:28.164 INFO:tasks.workunit.client.1.smithi192.stdout:5/325: truncate d0/d13/d3b/f55 2420699 0 2024-04-24T22:45:28.166 INFO:tasks.workunit.client.1.smithi192.stdout:4/303: dread d2/d9/d25/d2e/d31/d36/d5b/f37 [0,4194304] 0 2024-04-24T22:45:28.167 INFO:tasks.workunit.client.0.smithi063.stdout:8/488: write d1/d11/d8b/f8d [779884,89898] 0 2024-04-24T22:45:28.168 INFO:tasks.workunit.client.1.smithi192.stdout:4/304: readlink d2/l69 0 2024-04-24T22:45:28.170 INFO:tasks.workunit.client.1.smithi192.stdout:7/459: dwrite d2/db/f60 [0,4194304] 0 2024-04-24T22:45:28.171 INFO:tasks.workunit.client.1.smithi192.stdout:7/460: write d2/db/d25/f69 [851253,46057] 0 2024-04-24T22:45:28.172 INFO:tasks.workunit.client.1.smithi192.stdout:7/461: write d2/db/f20 [10507330,55602] 0 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 -- 172.21.15.63:0/2927709670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fcf10 msgr2=0x7fabfc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 --2- 172.21.15.63:0/2927709670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fcf10 0x7fabfc0fd320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fabe4004940 tx=0x7fabe4009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 -- 172.21.15.63:0/2927709670 shutdown_connections 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 --2- 172.21.15.63:0/2927709670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fe080 0x7fabfc0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 --2- 172.21.15.63:0/2927709670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fcf10 0x7fabfc0fd320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 -- 172.21.15.63:0/2927709670 >> 172.21.15.63:0/2927709670 conn(0x7fabfc0f8560 msgr2=0x7fabfc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 -- 172.21.15.63:0/2927709670 shutdown_connections 2024-04-24T22:45:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 -- 172.21.15.63:0/2927709670 wait complete. 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.193+0000 7fac0183f700 1 Processor -- start 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fac0183f700 1 -- start start 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fac0183f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fcf10 0x7fabfc10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fac0183f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fac0183f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc10e520 con 0x7fabfc0fe080 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fac0183f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc10ed70 con 0x7fabfc0fcf10 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47334/0 (socket says 172.21.15.63:47334) 2024-04-24T22:45:28.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 -- 172.21.15.63:0/391284120 learned_addr learned my addr 172.21.15.63:0/391284120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fcf10 0x7fabfc10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 -- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fcf10 msgr2=0x7fabfc10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fcf10 0x7fabfc10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 -- 172.21.15.63:0/391284120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4005040 con 0x7fabfc0fe080 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.194+0000 7fabfa7fc700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c830 tx=0x7fabec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.195+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabec0106f0 con 0x7fabfc0fe080 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.195+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabec010d20 con 0x7fabfc0fe080 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.195+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc10f050 con 0x7fabfc0fe080 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.195+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabec0113a0 con 0x7fabfc0fe080 2024-04-24T22:45:28.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.195+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc10f590 con 0x7fabfc0fe080 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.196+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc05f5c0 con 0x7fabfc0fe080 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.196+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fabec012400 con 0x7fabfc0fe080 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.197+0000 7fabf37fe700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 0x7fabe806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.197+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fabec08a1f0 con 0x7fabfc0fe080 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.197+0000 7fabfaffd700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 0x7fabe806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:28.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.197+0000 7fabfaffd700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 0x7fabe806dc40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b970 tx=0x7fabe4009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.201+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fabec057a30 con 0x7fabfc0fe080 2024-04-24T22:45:28.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/489: truncate d1/d11/d49/d54/f60 562529 0 2024-04-24T22:45:28.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/426: link d3/f31 d3/d24/f97 0 2024-04-24T22:45:28.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/490: dwrite d1/d21/f3a [0,4194304] 0 2024-04-24T22:45:28.285 INFO:tasks.workunit.client.0.smithi063.stdout:8/491: read d1/d21/f3a [2346523,1631] 0 2024-04-24T22:45:28.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/356: truncate db/dc/f17 91220 0 2024-04-24T22:45:28.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/357: chown db/dc/d22/c59 118 1 2024-04-24T22:45:28.287 INFO:tasks.workunit.client.0.smithi063.stdout:8/492: truncate d1/d2/f84 40211 0 2024-04-24T22:45:28.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/358: write db/dc/f1b [6850629,106051] 0 2024-04-24T22:45:28.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/359: chown db/dc/d22/c59 61730 1 2024-04-24T22:45:28.293 INFO:tasks.workunit.client.1.smithi192.stdout:7/462: creat d2/db/d21/d42/f8c x:0 0 0 2024-04-24T22:45:28.299 INFO:tasks.workunit.client.1.smithi192.stdout:2/462: creat d4/d9/d10/d6c/d8c/f8f x:0 0 0 2024-04-24T22:45:28.299 INFO:tasks.workunit.client.1.smithi192.stdout:7/463: dwrite d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:28.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:27 smithi063 ceph-mon[26912]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:28.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:27 smithi063 ceph-mon[26912]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:7/407: sync 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:3/498: sync 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:5/472: sync 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:1/554: sync 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/429: sync 2024-04-24T22:45:28.305 INFO:tasks.workunit.client.0.smithi063.stdout:0/413: sync 2024-04-24T22:45:28.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/490: sync 2024-04-24T22:45:28.306 INFO:tasks.workunit.client.0.smithi063.stdout:6/520: sync 2024-04-24T22:45:28.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/305: symlink d2/d9/d6b/l6d 0 2024-04-24T22:45:28.308 INFO:tasks.workunit.client.0.smithi063.stdout:7/408: fsync d0/d4/de/d17/f42 0 2024-04-24T22:45:28.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/561: creat d0/fa6 x:0 0 0 2024-04-24T22:45:28.311 INFO:tasks.workunit.client.0.smithi063.stdout:8/493: rename d1/d11/d32/d38/l5f to d1/d11/d49/d54/l9c 0 2024-04-24T22:45:28.312 INFO:tasks.workunit.client.1.smithi192.stdout:7/464: creat d2/db/d25/d82/f8d x:0 0 0 2024-04-24T22:45:28.312 INFO:tasks.workunit.client.0.smithi063.stdout:6/521: fsync db/d1d/d20/f38 0 2024-04-24T22:45:28.315 INFO:tasks.workunit.client.0.smithi063.stdout:3/499: truncate da/d18/d47/d7b/f91 1759131 0 2024-04-24T22:45:28.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/427: mkdir d3/dd/de/d17/d89/d8b/d8c/d5e/d98 0 2024-04-24T22:45:28.316 INFO:tasks.workunit.client.0.smithi063.stdout:0/414: rmdir d6/da/dc/d26 39 2024-04-24T22:45:28.316 INFO:tasks.workunit.client.0.smithi063.stdout:2/562: readlink d0/d2/d35/l5f 0 2024-04-24T22:45:28.317 INFO:tasks.workunit.client.0.smithi063.stdout:7/409: rmdir d0/d2f 39 2024-04-24T22:45:28.318 INFO:tasks.workunit.client.0.smithi063.stdout:1/555: write d3/db5/d21/d9f/daa/db7/f58 [685520,126744] 0 2024-04-24T22:45:28.319 INFO:tasks.workunit.client.0.smithi063.stdout:6/522: chown db/d1d/d20/f40 318419 1 2024-04-24T22:45:28.321 INFO:tasks.workunit.client.0.smithi063.stdout:5/473: mkdir d5/d11/d23/d2d/d62/d81/da4 0 2024-04-24T22:45:28.322 INFO:tasks.workunit.client.1.smithi192.stdout:7/465: mkdir d2/db/d3a/d6e/d77/d8e 0 2024-04-24T22:45:28.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/466: write d2/db/f20 [6820416,44361] 0 2024-04-24T22:45:28.325 INFO:tasks.workunit.client.0.smithi063.stdout:8/494: readlink d1/d11/d3c/l40 0 2024-04-24T22:45:28.327 INFO:tasks.workunit.client.0.smithi063.stdout:5/474: chown d5/d11/f3c 20 1 2024-04-24T22:45:28.327 INFO:tasks.workunit.client.0.smithi063.stdout:3/500: dread da/d18/d47/f6c [0,4194304] 0 2024-04-24T22:45:28.327 INFO:tasks.workunit.client.0.smithi063.stdout:4/430: dwrite d2/f26 [0,4194304] 0 2024-04-24T22:45:28.328 INFO:tasks.workunit.client.1.smithi192.stdout:8/360: truncate db/dc/f17 136003 0 2024-04-24T22:45:28.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/523: symlink db/d1d/d20/d84/lb2 0 2024-04-24T22:45:28.330 INFO:tasks.workunit.client.1.smithi192.stdout:2/463: creat d4/d9/f90 x:0 0 0 2024-04-24T22:45:28.331 INFO:tasks.workunit.client.1.smithi192.stdout:1/309: getdents d9/d15/d2d/d31/d2f 0 2024-04-24T22:45:28.335 INFO:tasks.workunit.client.0.smithi063.stdout:6/524: readlink db/d1d/d20/d3f/d5f/d93/l57 0 2024-04-24T22:45:28.336 INFO:tasks.workunit.client.1.smithi192.stdout:7/467: symlink d2/db/d21/d42/d5b/d63/l8f 0 2024-04-24T22:45:28.337 INFO:tasks.workunit.client.1.smithi192.stdout:7/468: readlink d2/db/d3a/l88 0 2024-04-24T22:45:28.339 INFO:tasks.workunit.client.0.smithi063.stdout:2/563: dwrite d0/d2/d35/f95 [0,4194304] 0 2024-04-24T22:45:28.339 INFO:tasks.workunit.client.1.smithi192.stdout:4/306: creat d2/d9/d25/d2e/d31/d36/d5b/f6e x:0 0 0 2024-04-24T22:45:28.341 INFO:tasks.workunit.client.0.smithi063.stdout:7/410: dwrite d0/d4/de/d17/d35/d69/f7b [0,4194304] 0 2024-04-24T22:45:28.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/564: readlink d0/d2/d39/l44 0 2024-04-24T22:45:28.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/415: dread d6/da/dc/d26/f5f [0,4194304] 0 2024-04-24T22:45:28.347 INFO:tasks.workunit.client.1.smithi192.stdout:7/469: mknod d2/db/d2a/c90 0 2024-04-24T22:45:28.347 INFO:tasks.workunit.client.0.smithi063.stdout:7/411: stat d0/d4/de/d17/c36 0 2024-04-24T22:45:28.348 INFO:tasks.workunit.client.0.smithi063.stdout:5/475: creat d5/d24/d44/fa5 x:0 0 0 2024-04-24T22:45:28.348 INFO:tasks.workunit.client.0.smithi063.stdout:3/501: dread da/f2e [0,4194304] 0 2024-04-24T22:45:28.348 INFO:tasks.workunit.client.1.smithi192.stdout:9/428: write d3/dd/d53/f7f [972446,125710] 0 2024-04-24T22:45:28.349 INFO:tasks.workunit.client.1.smithi192.stdout:9/429: chown d3/d59/d7d/f8a 1564 1 2024-04-24T22:45:28.349 INFO:tasks.workunit.client.1.smithi192.stdout:4/307: creat d2/d9/d25/d2e/d31/d36/d5b/d64/f6f x:0 0 0 2024-04-24T22:45:28.350 INFO:tasks.workunit.client.0.smithi063.stdout:0/416: write d6/da/d2b/f40 [1379672,73340] 0 2024-04-24T22:45:28.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/430: chown d3/dd/de/d17/d89/d8b/d8c/d5e/d98 482877562 1 2024-04-24T22:45:28.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/308: write d2/d9/d25/d2e/f66 [541771,91631] 0 2024-04-24T22:45:28.352 INFO:tasks.workunit.client.1.smithi192.stdout:9/431: readlink d3/dd/de/d17/d6a/l5f 0 2024-04-24T22:45:28.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/432: chown d3/dd/c29 111883391 1 2024-04-24T22:45:28.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/556: mknod d3/da/da2/cb9 0 2024-04-24T22:45:28.353 INFO:tasks.workunit.client.1.smithi192.stdout:8/361: unlink db/dc/c21 0 2024-04-24T22:45:28.355 INFO:tasks.workunit.client.0.smithi063.stdout:3/502: read da/d18/f30 [74766,110140] 0 2024-04-24T22:45:28.355 INFO:tasks.workunit.client.1.smithi192.stdout:2/464: creat d4/d9/d37/f91 x:0 0 0 2024-04-24T22:45:28.357 INFO:tasks.workunit.client.1.smithi192.stdout:1/310: unlink d9/d15/d2d/f37 0 2024-04-24T22:45:28.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/525: dwrite db/d1c/fa7 [0,4194304] 0 2024-04-24T22:45:28.360 INFO:tasks.workunit.client.0.smithi063.stdout:8/495: write d1/d11/f1b [1934213,111683] 0 2024-04-24T22:45:28.360 INFO:tasks.workunit.client.0.smithi063.stdout:5/476: dread d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:28.361 INFO:tasks.workunit.client.0.smithi063.stdout:2/565: unlink d0/l4 0 2024-04-24T22:45:28.363 INFO:tasks.workunit.client.0.smithi063.stdout:7/412: mknod d0/d4/de/d17/d35/c91 0 2024-04-24T22:45:28.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/465: truncate d4/f31 2631351 0 2024-04-24T22:45:28.364 INFO:tasks.workunit.client.0.smithi063.stdout:8/496: chown d1/d11/d3c/l52 44 1 2024-04-24T22:45:28.365 INFO:tasks.workunit.client.0.smithi063.stdout:9/491: dwrite d6/fd [0,4194304] 0 2024-04-24T22:45:28.365 INFO:tasks.workunit.client.0.smithi063.stdout:4/431: mknod d2/d8/d3c/c8e 0 2024-04-24T22:45:28.371 INFO:tasks.workunit.client.1.smithi192.stdout:4/309: write d2/d9/d25/f32 [3959921,20214] 0 2024-04-24T22:45:28.372 INFO:tasks.workunit.client.0.smithi063.stdout:9/492: chown d6/d8/d1f/d20/d2a/d4b/d5e/d9d/l9e 1934242 1 2024-04-24T22:45:28.372 INFO:tasks.workunit.client.1.smithi192.stdout:2/466: dwrite d4/f78 [0,4194304] 0 2024-04-24T22:45:28.375 INFO:tasks.workunit.client.0.smithi063.stdout:0/417: mknod d6/d64/d21/d53/c93 0 2024-04-24T22:45:28.376 INFO:tasks.workunit.client.0.smithi063.stdout:9/493: write d6/d8/d53/f99 [974114,13058] 0 2024-04-24T22:45:28.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/413: dwrite d0/d4/de/f7c [0,4194304] 0 2024-04-24T22:45:28.378 INFO:tasks.workunit.client.1.smithi192.stdout:7/470: creat d2/f91 x:0 0 0 2024-04-24T22:45:28.378 INFO:tasks.workunit.client.0.smithi063.stdout:8/497: dwrite d1/d2/d3/f82 [0,4194304] 0 2024-04-24T22:45:28.379 INFO:tasks.workunit.client.0.smithi063.stdout:3/503: chown da/f54 99 1 2024-04-24T22:45:28.379 INFO:tasks.workunit.client.1.smithi192.stdout:8/362: write db/f3d [2319968,96480] 0 2024-04-24T22:45:28.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/504: stat da/d18/d3a/l8e 0 2024-04-24T22:45:28.381 INFO:tasks.workunit.client.0.smithi063.stdout:9/494: dread - d6/d8/db/d6b/d4f/d59/f9b zero size 2024-04-24T22:45:28.381 INFO:tasks.workunit.client.1.smithi192.stdout:8/363: write db/d43/f4e [3731244,47184] 0 2024-04-24T22:45:28.386 INFO:tasks.workunit.client.0.smithi063.stdout:1/557: rename d3/da/dc to d3/da/da2/dba 0 2024-04-24T22:45:28.386 INFO:tasks.workunit.client.0.smithi063.stdout:9/495: dread d6/d8/db/d6b/d4f/d59/f76 [0,4194304] 0 2024-04-24T22:45:28.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/432: creat d2/d80/f8f x:0 0 0 2024-04-24T22:45:28.394 INFO:tasks.workunit.client.1.smithi192.stdout:7/471: symlink d2/db/d21/d42/l92 0 2024-04-24T22:45:28.396 INFO:tasks.workunit.client.1.smithi192.stdout:8/364: fsync f9 0 2024-04-24T22:45:28.397 INFO:tasks.workunit.client.1.smithi192.stdout:8/365: dread - db/f65 zero size 2024-04-24T22:45:28.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/310: write d2/d9/d25/d2e/d31/d36/d5b/f47 [2994785,47761] 0 2024-04-24T22:45:28.400 INFO:tasks.workunit.client.0.smithi063.stdout:8/498: mknod d1/d2/d3/d86/c9d 0 2024-04-24T22:45:28.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/311: getdents d9 0 2024-04-24T22:45:28.402 INFO:tasks.workunit.client.0.smithi063.stdout:7/414: chown d0/d4/de/d17/d28/c2b 65161766 1 2024-04-24T22:45:28.403 INFO:tasks.workunit.client.1.smithi192.stdout:1/312: write d9/f42 [804494,105425] 0 2024-04-24T22:45:28.404 INFO:tasks.workunit.client.0.smithi063.stdout:3/505: symlink da/d39/d94/lb2 0 2024-04-24T22:45:28.407 INFO:tasks.workunit.client.1.smithi192.stdout:8/366: symlink db/d5d/l6b 0 2024-04-24T22:45:28.411 INFO:tasks.workunit.client.0.smithi063.stdout:1/558: rename d3/db5/d21/f53 to d3/da/da2/dba/d11/d5e/d6f/fbb 0 2024-04-24T22:45:28.414 INFO:tasks.workunit.client.1.smithi192.stdout:4/311: truncate d2/d9/d25/d2e/d31/d36/d5b/d28/f2b 270130 0 2024-04-24T22:45:28.418 INFO:tasks.workunit.client.1.smithi192.stdout:7/472: fdatasync d2/f1a 0 2024-04-24T22:45:28.419 INFO:tasks.workunit.client.1.smithi192.stdout:7/473: chown d2/db/d2a/c7b 10968528 1 2024-04-24T22:45:28.420 INFO:tasks.workunit.client.0.smithi063.stdout:5/477: mkdir d5/d11/d23/d2d/d62/d81/da4/da6 0 2024-04-24T22:45:28.420 INFO:tasks.workunit.client.1.smithi192.stdout:7/474: fdatasync d2/db/d49/f6c 0 2024-04-24T22:45:28.423 INFO:tasks.workunit.client.0.smithi063.stdout:2/566: symlink d0/d2/d2d/d31/d89/d7/d86/d8e/la7 0 2024-04-24T22:45:28.425 INFO:tasks.workunit.client.1.smithi192.stdout:8/367: mknod db/dc/d22/d31/d33/c6c 0 2024-04-24T22:45:28.427 INFO:tasks.workunit.client.0.smithi063.stdout:0/418: mknod d6/d64/d21/d25/c94 0 2024-04-24T22:45:28.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:27 smithi192 ceph-mon[33991]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:28.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:27 smithi192 ceph-mon[33991]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:28.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/499: creat d1/d11/d3c/f9e x:0 0 0 2024-04-24T22:45:28.432 INFO:tasks.workunit.client.0.smithi063.stdout:8/500: fdatasync d1/d22/d33/f57 0 2024-04-24T22:45:28.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/415: mknod d0/d4/de/d17/d35/d6c/c92 0 2024-04-24T22:45:28.433 INFO:tasks.workunit.client.1.smithi192.stdout:4/312: mkdir d2/d9/d25/d2e/d31/d36/d70 0 2024-04-24T22:45:28.435 INFO:tasks.workunit.client.0.smithi063.stdout:3/506: mkdir da/d39/d94/db3 0 2024-04-24T22:45:28.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/496: write d6/d8/d1f/f28 [445228,93155] 0 2024-04-24T22:45:28.437 INFO:tasks.workunit.client.0.smithi063.stdout:0/419: dwrite d6/d64/d24/f59 [0,4194304] 0 2024-04-24T22:45:28.437 INFO:tasks.workunit.client.1.smithi192.stdout:7/475: creat d2/db/d21/d42/f93 x:0 0 0 2024-04-24T22:45:28.438 INFO:tasks.workunit.client.1.smithi192.stdout:1/313: mkdir d9/d15/d2d/d63 0 2024-04-24T22:45:28.439 INFO:tasks.workunit.client.0.smithi063.stdout:0/420: dread - d6/d64/d24/f6b zero size 2024-04-24T22:45:28.440 INFO:tasks.workunit.client.0.smithi063.stdout:1/559: rename d3/da/da2/dba/c14 to d3/da/da2/dba/d11/cbc 0 2024-04-24T22:45:28.442 INFO:tasks.workunit.client.1.smithi192.stdout:8/368: creat db/dc/d22/d31/d33/f6d x:0 0 0 2024-04-24T22:45:28.442 INFO:tasks.workunit.client.1.smithi192.stdout:7/476: read d2/db/d2a/f3c [5155126,36054] 0 2024-04-24T22:45:28.443 INFO:tasks.workunit.client.0.smithi063.stdout:1/560: rename d3/db5/d21/d9f to d3/db5/d21/d9f/daa/db7/d6e/dbd 22 2024-04-24T22:45:28.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/433: mkdir d2/d80/d81/d90 0 2024-04-24T22:45:28.446 INFO:tasks.workunit.client.0.smithi063.stdout:9/497: dwrite d6/d8/db/d2b/f49 [0,4194304] 0 2024-04-24T22:45:28.451 INFO:tasks.workunit.client.0.smithi063.stdout:1/561: read - d3/da/da2/dba/d11/d5e/d6f/fbb zero size 2024-04-24T22:45:28.455 INFO:tasks.workunit.client.0.smithi063.stdout:1/562: write d3/db5/f22 [457880,93857] 0 2024-04-24T22:45:28.457 INFO:tasks.workunit.client.0.smithi063.stdout:4/434: dwrite d2/d8/d40/f68 [0,4194304] 0 2024-04-24T22:45:28.457 INFO:tasks.workunit.client.1.smithi192.stdout:7/477: rename d2/l3f to d2/db/d2a/d57/l94 0 2024-04-24T22:45:28.457 INFO:tasks.workunit.client.0.smithi063.stdout:1/563: dread - d3/da/da2/dba/d11/f5f zero size 2024-04-24T22:45:28.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/313: write d2/f3 [4832483,62422] 0 2024-04-24T22:45:28.461 INFO:tasks.workunit.client.0.smithi063.stdout:6/526: getdents db/d1d/d20/d29/d5d 0 2024-04-24T22:45:28.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/467: write d4/f31 [3052638,80473] 0 2024-04-24T22:45:28.465 INFO:tasks.workunit.client.1.smithi192.stdout:1/314: write f3 [104555,129778] 0 2024-04-24T22:45:28.465 INFO:tasks.workunit.client.1.smithi192.stdout:1/315: read - d9/f38 zero size 2024-04-24T22:45:28.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/478: mknod d2/db/d25/d82/c95 0 2024-04-24T22:45:28.467 INFO:tasks.workunit.client.0.smithi063.stdout:4/435: dwrite d2/f2e [0,4194304] 0 2024-04-24T22:45:28.470 INFO:tasks.workunit.client.1.smithi192.stdout:8/369: creat db/dc/f6e x:0 0 0 2024-04-24T22:45:28.471 INFO:tasks.workunit.client.0.smithi063.stdout:0/421: dwrite d6/da/f3d [0,4194304] 0 2024-04-24T22:45:28.473 INFO:tasks.workunit.client.0.smithi063.stdout:0/422: fsync d6/da/d2b/f40 0 2024-04-24T22:45:28.473 INFO:tasks.workunit.client.1.smithi192.stdout:4/314: write d2/d9/d25/d2e/d31/d36/d5b/d28/f61 [594970,123967] 0 2024-04-24T22:45:28.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/315: stat d2/d9/d25/d2e/f66 0 2024-04-24T22:45:28.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/316: rename c8 to d9/d15/d2d/d31/d2f/c64 0 2024-04-24T22:45:28.481 INFO:tasks.workunit.client.1.smithi192.stdout:8/370: fdatasync f8 0 2024-04-24T22:45:28.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/564: creat d3/db5/d21/d9f/daa/db7/d6e/fbe x:0 0 0 2024-04-24T22:45:28.484 INFO:tasks.workunit.client.0.smithi063.stdout:6/527: rmdir db/d1d/d20/d29 39 2024-04-24T22:45:28.485 INFO:tasks.workunit.client.1.smithi192.stdout:1/317: dwrite d9/f13 [0,4194304] 0 2024-04-24T22:45:28.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/423: dwrite d6/d64/d21/d25/f7b [0,4194304] 0 2024-04-24T22:45:28.488 INFO:tasks.workunit.client.0.smithi063.stdout:2/567: rename d0/d2/f4b to d0/d2/fa8 0 2024-04-24T22:45:28.492 INFO:tasks.workunit.client.0.smithi063.stdout:9/498: mkdir d6/d8/db/d2b/d48/d94/da1 0 2024-04-24T22:45:28.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/318: write d9/f1a [5041613,19767] 0 2024-04-24T22:45:28.498 INFO:tasks.workunit.client.0.smithi063.stdout:4/436: dread d2/f35 [0,4194304] 0 2024-04-24T22:45:28.500 INFO:tasks.workunit.client.0.smithi063.stdout:8/501: creat d1/d11/f9f x:0 0 0 2024-04-24T22:45:28.500 INFO:tasks.workunit.client.1.smithi192.stdout:7/479: write d2/db/d36/f3d [3901770,7049] 0 2024-04-24T22:45:28.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/478: sync 2024-04-24T22:45:28.504 INFO:tasks.workunit.client.0.smithi063.stdout:3/507: creat da/fb4 x:0 0 0 2024-04-24T22:45:28.505 INFO:tasks.workunit.client.0.smithi063.stdout:3/508: readlink da/d18/l59 0 2024-04-24T22:45:28.506 INFO:tasks.workunit.client.1.smithi192.stdout:1/319: creat d9/f65 x:0 0 0 2024-04-24T22:45:28.507 INFO:tasks.workunit.client.1.smithi192.stdout:7/480: dwrite d2/db/d21/f84 [0,4194304] 0 2024-04-24T22:45:28.508 INFO:tasks.workunit.client.0.smithi063.stdout:1/565: mkdir d3/da/da2/dba/d39/dbf 0 2024-04-24T22:45:28.508 INFO:tasks.workunit.client.1.smithi192.stdout:2/468: getdents d4/d9/d10/d6c/d8c 0 2024-04-24T22:45:28.508 INFO:tasks.workunit.client.1.smithi192.stdout:7/481: chown d2/l3 95 1 2024-04-24T22:45:28.509 INFO:tasks.workunit.client.0.smithi063.stdout:5/479: dread d5/d1d/d7d/f52 [0,4194304] 0 2024-04-24T22:45:28.513 INFO:tasks.workunit.client.0.smithi063.stdout:2/568: rename d0/d2/d39/l44 to d0/d2/d2d/d31/d68/la9 0 2024-04-24T22:45:28.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/509: dwrite da/d18/d9e/da3/fab [0,4194304] 0 2024-04-24T22:45:28.517 INFO:tasks.workunit.client.1.smithi192.stdout:8/371: truncate f3 2217840 0 2024-04-24T22:45:28.518 INFO:tasks.workunit.client.0.smithi063.stdout:9/499: creat d6/d8/db/d2b/d48/d94/fa2 x:0 0 0 2024-04-24T22:45:28.520 INFO:tasks.workunit.client.0.smithi063.stdout:4/437: symlink d2/d8/d40/d51/l91 0 2024-04-24T22:45:28.521 INFO:tasks.workunit.client.1.smithi192.stdout:2/469: unlink d4/d9/f5a 0 2024-04-24T22:45:28.522 INFO:tasks.workunit.client.1.smithi192.stdout:2/470: stat d4/f78 0 2024-04-24T22:45:28.523 INFO:tasks.workunit.client.0.smithi063.stdout:3/510: dread da/d18/d47/d4d/d62/f65 [0,4194304] 0 2024-04-24T22:45:28.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/424: write d6/d64/d21/d25/d62/d75/f81 [745059,12920] 0 2024-04-24T22:45:28.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/416: getdents d0/d4/de/d6f/d5c/d8b 0 2024-04-24T22:45:28.529 INFO:tasks.workunit.client.0.smithi063.stdout:6/528: symlink db/d1d/d20/d3f/d5f/lb3 0 2024-04-24T22:45:28.533 INFO:tasks.workunit.client.1.smithi192.stdout:8/372: symlink db/dc/d22/d31/l6f 0 2024-04-24T22:45:28.534 INFO:tasks.workunit.client.1.smithi192.stdout:7/482: creat d2/db/d25/f96 x:0 0 0 2024-04-24T22:45:28.536 INFO:tasks.workunit.client.0.smithi063.stdout:2/569: chown d0/d2/d2f/f6d 498988 1 2024-04-24T22:45:28.538 INFO:tasks.workunit.client.1.smithi192.stdout:3/388: sync 2024-04-24T22:45:28.539 INFO:tasks.workunit.client.1.smithi192.stdout:7/483: rename d2/l3 to d2/db/d2a/l97 0 2024-04-24T22:45:28.540 INFO:tasks.workunit.client.1.smithi192.stdout:7/484: stat d2/db/d21/d42/d5b/d63/d80 0 2024-04-24T22:45:28.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/511: dread da/d18/d47/f6a [0,4194304] 0 2024-04-24T22:45:28.542 INFO:tasks.workunit.client.0.smithi063.stdout:3/512: stat da/d18/d3a/l8e 0 2024-04-24T22:45:28.542 INFO:tasks.workunit.client.0.smithi063.stdout:3/513: readlink l5 0 2024-04-24T22:45:28.542 INFO:tasks.workunit.client.0.smithi063.stdout:7/417: fsync d0/d4/de/f1a 0 2024-04-24T22:45:28.543 INFO:tasks.workunit.client.1.smithi192.stdout:2/471: getdents d4/d9/d37 0 2024-04-24T22:45:28.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/566: truncate d3/f3c 3872709 0 2024-04-24T22:45:28.545 INFO:tasks.workunit.client.1.smithi192.stdout:3/389: rename d2/d27/f73 to d2/d62/f7d 0 2024-04-24T22:45:28.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/567: truncate d3/da/d34/f93 1133017 0 2024-04-24T22:45:28.547 INFO:tasks.workunit.client.1.smithi192.stdout:3/390: write d2/d4/d12/d65/f66 [755426,83332] 0 2024-04-24T22:45:28.548 INFO:tasks.workunit.client.1.smithi192.stdout:2/472: dread d4/d9/d37/f55 [0,4194304] 0 2024-04-24T22:45:28.548 INFO:tasks.workunit.client.0.smithi063.stdout:5/480: creat d5/d24/d36/d65/fa7 x:0 0 0 2024-04-24T22:45:28.552 INFO:tasks.workunit.client.0.smithi063.stdout:9/500: symlink d6/d8/db/d2b/d36/d7d/la3 0 2024-04-24T22:45:28.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/501: chown d6/d8/db/d6b/d4f/d59/l84 26263466 1 2024-04-24T22:45:28.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/568: dwrite d3/da/da2/dba/f8c [0,4194304] 0 2024-04-24T22:45:28.555 INFO:tasks.workunit.client.1.smithi192.stdout:3/391: unlink d2/d27/d28/d38/f72 0 2024-04-24T22:45:28.556 INFO:tasks.workunit.client.0.smithi063.stdout:9/502: fsync d6/d8/d1f/f28 0 2024-04-24T22:45:28.556 INFO:tasks.workunit.client.0.smithi063.stdout:8/502: link d1/f58 d1/d11/d32/d38/fa0 0 2024-04-24T22:45:28.557 INFO:tasks.workunit.client.1.smithi192.stdout:1/320: write d9/d15/d2d/d31/d2f/f3a [4215498,108161] 0 2024-04-24T22:45:28.559 INFO:tasks.workunit.client.1.smithi192.stdout:7/485: link d2/db/d2a/c90 d2/db/d3a/d6e/c98 0 2024-04-24T22:45:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.558+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabfc10f970 con 0x7fabe806b790 2024-04-24T22:45:28.560 INFO:tasks.workunit.client.1.smithi192.stdout:7/486: dread - d2/db/d25/f7d zero size 2024-04-24T22:45:28.561 INFO:tasks.workunit.client.0.smithi063.stdout:7/418: symlink d0/d4/de/d6f/d5c/d63/l93 0 2024-04-24T22:45:28.562 INFO:tasks.workunit.client.0.smithi063.stdout:4/438: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:28.562 INFO:tasks.workunit.client.1.smithi192.stdout:7/487: fsync d2/db/f50 0 2024-04-24T22:45:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.560+0000 7fabf37fe700 1 -- 172.21.15.63:0/391284120 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fabfc10f970 con 0x7fabe806b790 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:45:28.563 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:45:28.565 INFO:tasks.workunit.client.1.smithi192.stdout:3/392: dwrite d2/d4/d12/d1a/d1d/d55/d5f/f68 [0,4194304] 0 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 msgr2=0x7fabe806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 0x7fabe806dc40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b970 tx=0x7fabe4009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 msgr2=0x7fabfc10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c830 tx=0x7fabec004470 comp rx=0 tx=0).stop 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 shutdown_connections 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7fabe806b790 0x7fabe806dc40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabfc0fcf10 0x7fabfc10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 --2- 172.21.15.63:0/391284120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabfc0fe080 0x7fabfc10df20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 >> 172.21.15.63:0/391284120 conn(0x7fabfc0f8560 msgr2=0x7fabfc0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 shutdown_connections 2024-04-24T22:45:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:28.564+0000 7fac0183f700 1 -- 172.21.15.63:0/391284120 wait complete. 2024-04-24T22:45:28.568 INFO:tasks.workunit.client.0.smithi063.stdout:3/514: rename da/d40/c49 to da/d18/d47/cb5 0 2024-04-24T22:45:28.570 INFO:tasks.workunit.client.0.smithi063.stdout:3/515: fsync da/fa0 0 2024-04-24T22:45:28.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/373: dwrite db/dc/f24 [0,4194304] 0 2024-04-24T22:45:28.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/529: creat db/d1d/d2d/d69/d82/d96/fb4 x:0 0 0 2024-04-24T22:45:28.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/488: stat d2/db/d3a/d6e/d77/l7c 0 2024-04-24T22:45:28.579 INFO:tasks.workunit.client.0.smithi063.stdout:2/570: symlink d0/d2/d2d/laa 0 2024-04-24T22:45:28.581 INFO:tasks.workunit.client.1.smithi192.stdout:2/473: creat d4/d9/f92 x:0 0 0 2024-04-24T22:45:28.582 INFO:tasks.workunit.client.0.smithi063.stdout:9/503: creat d6/d2f/fa4 x:0 0 0 2024-04-24T22:45:28.583 INFO:tasks.workunit.client.1.smithi192.stdout:2/474: chown d4/d29/f34 380 1 2024-04-24T22:45:28.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/489: dread d2/f1a [4194304,4194304] 0 2024-04-24T22:45:28.584 INFO:tasks.workunit.client.1.smithi192.stdout:8/374: fsync db/dc/d22/d31/f4d 0 2024-04-24T22:45:28.586 INFO:tasks.workunit.client.0.smithi063.stdout:5/481: rename d5/d11/d18/f20 to d5/d24/d49/d3b/fa8 0 2024-04-24T22:45:28.587 INFO:tasks.workunit.client.1.smithi192.stdout:3/393: dwrite d2/d4/d7/f7c [0,4194304] 0 2024-04-24T22:45:28.589 INFO:tasks.workunit.client.1.smithi192.stdout:1/321: symlink d9/d51/l66 0 2024-04-24T22:45:28.590 INFO:tasks.workunit.client.0.smithi063.stdout:2/571: read d0/d2/d2d/d31/d68/f74 [424939,128510] 0 2024-04-24T22:45:28.591 INFO:tasks.workunit.client.0.smithi063.stdout:7/419: dread d0/fa [0,4194304] 0 2024-04-24T22:45:28.592 INFO:tasks.workunit.client.0.smithi063.stdout:2/572: chown d0/d2/d27/c83 226409938 1 2024-04-24T22:45:28.594 INFO:tasks.workunit.client.1.smithi192.stdout:2/475: creat d4/d9/d10/d6c/d8c/f93 x:0 0 0 2024-04-24T22:45:28.594 INFO:tasks.workunit.client.1.smithi192.stdout:8/375: dwrite db/f34 [4194304,4194304] 0 2024-04-24T22:45:28.597 INFO:tasks.workunit.client.1.smithi192.stdout:8/376: read - db/dc/d22/d31/d33/f6d zero size 2024-04-24T22:45:28.597 INFO:tasks.workunit.client.0.smithi063.stdout:9/504: truncate d6/d8/d1f/d20/d2a/f66 1037422 0 2024-04-24T22:45:28.601 INFO:tasks.workunit.client.1.smithi192.stdout:3/394: rename d2/d4/d12/d1a/d1d/c67 to d2/d4/d4e/c7e 0 2024-04-24T22:45:28.606 INFO:tasks.workunit.client.0.smithi063.stdout:3/516: unlink da/d18/d3a/d63/c67 0 2024-04-24T22:45:28.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/377: mkdir db/dc/d29/d70 0 2024-04-24T22:45:28.609 INFO:tasks.workunit.client.0.smithi063.stdout:6/530: truncate db/d1d/d20/f38 3730036 0 2024-04-24T22:45:28.611 INFO:tasks.workunit.client.1.smithi192.stdout:2/476: rename d4/d29/f86 to d4/d9/d37/f94 0 2024-04-24T22:45:28.613 INFO:tasks.workunit.client.1.smithi192.stdout:3/395: unlink d2/d4/d12/d1a/d1d/d55/c18 0 2024-04-24T22:45:28.614 INFO:tasks.workunit.client.0.smithi063.stdout:1/569: dwrite d3/db5/d2b/f43 [4194304,4194304] 0 2024-04-24T22:45:28.616 INFO:tasks.workunit.client.1.smithi192.stdout:1/322: mkdir d9/d15/d5e/d67 0 2024-04-24T22:45:28.617 INFO:tasks.workunit.client.0.smithi063.stdout:7/420: truncate d0/d4/de/d17/d28/f64 287889 0 2024-04-24T22:45:28.622 INFO:tasks.workunit.client.1.smithi192.stdout:8/378: rename db/dc/d29/d70 to db/d43/d71 0 2024-04-24T22:45:28.625 INFO:tasks.workunit.client.1.smithi192.stdout:2/477: mkdir d4/d9/d10/d6c/d95 0 2024-04-24T22:45:28.625 INFO:tasks.workunit.client.0.smithi063.stdout:9/505: dread d6/f7 [0,4194304] 0 2024-04-24T22:45:28.632 INFO:tasks.workunit.client.1.smithi192.stdout:7/490: dread d2/db/d2a/f3c [4194304,4194304] 0 2024-04-24T22:45:28.635 INFO:tasks.workunit.client.0.smithi063.stdout:6/531: dread db/d1d/d20/f71 [0,4194304] 0 2024-04-24T22:45:28.635 INFO:tasks.workunit.client.1.smithi192.stdout:8/379: symlink db/dc/d22/l72 0 2024-04-24T22:45:28.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/439: getdents d2/d80/d8d 0 2024-04-24T22:45:28.637 INFO:tasks.workunit.client.0.smithi063.stdout:4/440: write d2/d8/d40/d51/f64 [702029,4577] 0 2024-04-24T22:45:28.637 INFO:tasks.workunit.client.1.smithi192.stdout:3/396: dread d2/d4/d7/f47 [0,4194304] 0 2024-04-24T22:45:28.639 INFO:tasks.workunit.client.1.smithi192.stdout:1/323: mkdir d9/d15/d2d/d63/d68 0 2024-04-24T22:45:28.640 INFO:tasks.workunit.client.0.smithi063.stdout:1/570: dread - d3/da/da2/dba/d11/d5e/d6f/faf zero size 2024-04-24T22:45:28.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/491: creat d2/db/d3a/f99 x:0 0 0 2024-04-24T22:45:28.641 INFO:tasks.workunit.client.0.smithi063.stdout:1/571: chown d3/da/da2/dba/d11/d5e/d6f/c72 896 1 2024-04-24T22:45:28.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/492: dread - d2/db/d3a/f99 zero size 2024-04-24T22:45:28.642 INFO:tasks.workunit.client.0.smithi063.stdout:1/572: chown d3/db5/f22 1 1 2024-04-24T22:45:28.642 INFO:tasks.workunit.client.0.smithi063.stdout:7/421: mkdir d0/d4/de/d17/d28/d31/d94 0 2024-04-24T22:45:28.644 INFO:tasks.workunit.client.1.smithi192.stdout:8/380: unlink db/dc/d22/d31/d33/f58 0 2024-04-24T22:45:28.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/478: unlink d4/d9/f70 0 2024-04-24T22:45:28.648 INFO:tasks.workunit.client.1.smithi192.stdout:2/479: read d4/f41 [2987715,113663] 0 2024-04-24T22:45:28.649 INFO:tasks.workunit.client.1.smithi192.stdout:2/480: dread - d4/d9/d37/f3e zero size 2024-04-24T22:45:28.650 INFO:tasks.workunit.client.0.smithi063.stdout:3/517: mkdir da/db6 0 2024-04-24T22:45:28.653 INFO:tasks.workunit.client.1.smithi192.stdout:8/381: mkdir db/d5d/d73 0 2024-04-24T22:45:28.653 INFO:tasks.workunit.client.1.smithi192.stdout:1/324: dread d9/dd/f19 [0,4194304] 0 2024-04-24T22:45:28.653 INFO:tasks.workunit.client.0.smithi063.stdout:6/532: mkdir db/d85/db5 0 2024-04-24T22:45:28.654 INFO:tasks.workunit.client.1.smithi192.stdout:1/325: truncate d9/dd/f29 4697027 0 2024-04-24T22:45:28.654 INFO:tasks.workunit.client.1.smithi192.stdout:3/397: getdents d2/d4/d12/d1a/d1d/d7b 0 2024-04-24T22:45:28.656 INFO:tasks.workunit.client.0.smithi063.stdout:2/573: rename d0/d2/d35/f95 to d0/d2/d2d/d31/fab 0 2024-04-24T22:45:28.657 INFO:tasks.workunit.client.1.smithi192.stdout:2/481: symlink d4/d9/d10/d6c/d6f/l96 0 2024-04-24T22:45:28.657 INFO:tasks.workunit.client.1.smithi192.stdout:2/482: fdatasync d4/d9/f8b 0 2024-04-24T22:45:28.658 INFO:tasks.workunit.client.0.smithi063.stdout:1/573: mknod d3/db5/d21/d9f/cc0 0 2024-04-24T22:45:28.658 INFO:tasks.workunit.client.0.smithi063.stdout:2/574: read d0/d2/d39/f65 [1353478,3444] 0 2024-04-24T22:45:28.659 INFO:tasks.workunit.client.0.smithi063.stdout:5/482: link d5/d24/d2c/l34 d5/d1d/la9 0 2024-04-24T22:45:28.662 INFO:tasks.workunit.client.1.smithi192.stdout:7/493: creat d2/db/d21/d42/d5b/f9a x:0 0 0 2024-04-24T22:45:28.662 INFO:tasks.workunit.client.0.smithi063.stdout:6/533: dwrite db/d1d/d2d/d69/d82/d96/fb4 [0,4194304] 0 2024-04-24T22:45:28.662 INFO:tasks.workunit.client.0.smithi063.stdout:5/483: fsync d5/d24/d49/d3b/f48 0 2024-04-24T22:45:28.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/518: write da/d18/d3a/f3f [2595272,83947] 0 2024-04-24T22:45:28.664 INFO:tasks.workunit.client.0.smithi063.stdout:4/441: creat d2/d80/d81/d90/f92 x:0 0 0 2024-04-24T22:45:28.664 INFO:tasks.workunit.client.0.smithi063.stdout:6/534: fsync db/d1d/d20/d3f/d7f/d68/d70/fad 0 2024-04-24T22:45:28.665 INFO:tasks.workunit.client.1.smithi192.stdout:1/326: rename d9/d15/d2d/l23 to d9/d15/d2d/d4a/l69 0 2024-04-24T22:45:28.666 INFO:tasks.workunit.client.0.smithi063.stdout:3/519: chown da/d18/d47/d4d/d62/f80 233958 1 2024-04-24T22:45:28.667 INFO:tasks.workunit.client.0.smithi063.stdout:7/422: mkdir d0/d4/de/d95 0 2024-04-24T22:45:28.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/327: dwrite d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:28.675 INFO:tasks.workunit.client.0.smithi063.stdout:2/575: dread d0/f69 [0,4194304] 0 2024-04-24T22:45:28.675 INFO:tasks.workunit.client.0.smithi063.stdout:1/574: rmdir d3/da 39 2024-04-24T22:45:28.676 INFO:tasks.workunit.client.1.smithi192.stdout:6/350: sync 2024-04-24T22:45:28.676 INFO:tasks.workunit.client.1.smithi192.stdout:0/436: sync 2024-04-24T22:45:28.676 INFO:tasks.workunit.client.1.smithi192.stdout:5/326: sync 2024-04-24T22:45:28.677 INFO:tasks.workunit.client.1.smithi192.stdout:9/433: sync 2024-04-24T22:45:28.677 INFO:tasks.workunit.client.0.smithi063.stdout:0/425: sync 2024-04-24T22:45:28.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/398: link d2/d4/d12/d39/c5c d2/d4/d7/d4b/c7f 0 2024-04-24T22:45:28.681 INFO:tasks.workunit.client.1.smithi192.stdout:3/399: write d2/d4/d7/f2e [779062,69319] 0 2024-04-24T22:45:28.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/506: link d6/d8/d1f/d20/c8d d6/d8/db/d2b/d36/d7d/ca5 0 2024-04-24T22:45:28.688 INFO:tasks.workunit.client.0.smithi063.stdout:2/576: dwrite d0/d2/d2d/d31/f5b [0,4194304] 0 2024-04-24T22:45:28.689 INFO:tasks.workunit.client.0.smithi063.stdout:7/423: dread d0/d4/de/d6f/f53 [0,4194304] 0 2024-04-24T22:45:28.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/434: chown d3/d5d/f61 105277002 1 2024-04-24T22:45:28.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/435: chown d3/d24/f7e 1242293 1 2024-04-24T22:45:28.691 INFO:tasks.workunit.client.1.smithi192.stdout:9/436: fdatasync d3/dd/de/d17/d6a/f96 0 2024-04-24T22:45:28.692 INFO:tasks.workunit.client.1.smithi192.stdout:9/437: dread - d3/d59/d7d/f8a zero size 2024-04-24T22:45:28.693 INFO:tasks.workunit.client.0.smithi063.stdout:0/426: dwrite d6/d64/d24/f73 [0,4194304] 0 2024-04-24T22:45:28.693 INFO:tasks.workunit.client.1.smithi192.stdout:6/351: unlink d0/d8/c59 0 2024-04-24T22:45:28.693 INFO:tasks.workunit.client.1.smithi192.stdout:9/438: readlink d3/dd/de/l1e 0 2024-04-24T22:45:28.696 INFO:tasks.workunit.client.1.smithi192.stdout:5/327: write d0/d1b/f24 [1505640,37945] 0 2024-04-24T22:45:28.698 INFO:tasks.workunit.client.0.smithi063.stdout:9/507: dwrite d6/d8/db/d6b/f8f [0,4194304] 0 2024-04-24T22:45:28.704 INFO:tasks.workunit.client.1.smithi192.stdout:8/382: dwrite db/dc/f17 [0,4194304] 0 2024-04-24T22:45:28.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/575: creat d3/db5/d2b/d79/fc1 x:0 0 0 2024-04-24T22:45:28.706 INFO:tasks.workunit.client.0.smithi063.stdout:8/503: sync 2024-04-24T22:45:28.707 INFO:tasks.workunit.client.1.smithi192.stdout:3/400: creat d2/d4/d12/d1a/d1d/d55/d5f/f80 x:0 0 0 2024-04-24T22:45:28.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/383: read f6 [2238874,75141] 0 2024-04-24T22:45:28.709 INFO:tasks.workunit.client.1.smithi192.stdout:8/384: read db/dc/d22/f5e [385575,373] 0 2024-04-24T22:45:28.711 INFO:tasks.workunit.client.0.smithi063.stdout:5/484: fdatasync d5/d11/d18/f3a 0 2024-04-24T22:45:28.711 INFO:tasks.workunit.client.1.smithi192.stdout:2/483: getdents d4/d29 0 2024-04-24T22:45:28.713 INFO:tasks.workunit.client.0.smithi063.stdout:8/504: read d1/f5 [2912312,124877] 0 2024-04-24T22:45:28.715 INFO:tasks.workunit.client.1.smithi192.stdout:7/494: rmdir d2/db/d2a/d86 0 2024-04-24T22:45:28.718 INFO:tasks.workunit.client.0.smithi063.stdout:4/442: fdatasync d2/d8/f31 0 2024-04-24T22:45:28.719 INFO:tasks.workunit.client.1.smithi192.stdout:9/439: chown d3/f43 63074636 1 2024-04-24T22:45:28.719 INFO:tasks.workunit.client.1.smithi192.stdout:4/316: sync 2024-04-24T22:45:28.720 INFO:tasks.workunit.client.0.smithi063.stdout:6/535: rename db/d1d/d20/d3f/d7f/f88 to db/d1d/d20/d29/d5d/fb6 0 2024-04-24T22:45:28.721 INFO:tasks.workunit.client.1.smithi192.stdout:0/437: dwrite d3/d27/d28/d2e/f53 [0,4194304] 0 2024-04-24T22:45:28.721 INFO:tasks.workunit.client.1.smithi192.stdout:6/352: readlink d0/d8/d18/l2f 0 2024-04-24T22:45:28.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/505: dwrite d1/d2/fd [8388608,4194304] 0 2024-04-24T22:45:28.723 INFO:tasks.workunit.client.1.smithi192.stdout:0/438: write d3/d6d/d79/f8b [1635225,36754] 0 2024-04-24T22:45:28.727 INFO:tasks.workunit.client.0.smithi063.stdout:2/577: mkdir d0/d2/d2d/d31/d89/d7/dac 0 2024-04-24T22:45:28.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/427: chown d6/d3a/c8d 12 1 2024-04-24T22:45:28.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/401: creat d2/d4/d12/d1a/f81 x:0 0 0 2024-04-24T22:45:28.734 INFO:tasks.workunit.client.0.smithi063.stdout:9/508: creat d6/d8/db/d2b/d48/d94/fa6 x:0 0 0 2024-04-24T22:45:28.735 INFO:tasks.workunit.client.1.smithi192.stdout:8/385: truncate f9 3150883 0 2024-04-24T22:45:28.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/576: write d3/da/da2/dba/d11/f46 [2002434,110559] 0 2024-04-24T22:45:28.739 INFO:tasks.workunit.client.1.smithi192.stdout:7/495: rename d2/db/d3a to d2/db/d21/d42/d67/d9b 0 2024-04-24T22:45:28.739 INFO:tasks.workunit.client.1.smithi192.stdout:7/496: chown d2/db/d25/f69 15093777 1 2024-04-24T22:45:28.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/328: creat d9/f6a x:0 0 0 2024-04-24T22:45:28.741 INFO:tasks.workunit.client.1.smithi192.stdout:7/497: chown d2/db/d21/d42/d67/d9b/d6e 10548 1 2024-04-24T22:45:28.746 INFO:tasks.workunit.client.1.smithi192.stdout:9/440: creat d3/dd/de/d17/d89/d8b/f99 x:0 0 0 2024-04-24T22:45:28.749 INFO:tasks.workunit.client.0.smithi063.stdout:7/424: rename d0/d4/de/d6f/d2e/l78 to d0/d4/de/d17/d35/l96 0 2024-04-24T22:45:28.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/443: dread d2/d8/f23 [0,4194304] 0 2024-04-24T22:45:28.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/444: chown d2 845 1 2024-04-24T22:45:28.752 INFO:tasks.workunit.client.0.smithi063.stdout:4/445: stat d2/d8/d40/f68 0 2024-04-24T22:45:28.752 INFO:tasks.workunit.client.1.smithi192.stdout:0/439: mknod d3/d27/d3c/d64/c96 0 2024-04-24T22:45:28.753 INFO:tasks.workunit.client.0.smithi063.stdout:4/446: chown d2/d8/d40/c6a 3175882 1 2024-04-24T22:45:28.754 INFO:tasks.workunit.client.0.smithi063.stdout:3/520: creat da/d18/d3a/fb7 x:0 0 0 2024-04-24T22:45:28.756 INFO:tasks.workunit.client.1.smithi192.stdout:3/402: symlink d2/d4/d12/d39/d3a/l82 0 2024-04-24T22:45:28.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/521: dwrite da/d18/d47/d4d/f72 [0,4194304] 0 2024-04-24T22:45:28.763 INFO:tasks.workunit.client.1.smithi192.stdout:2/484: dread d4/d9/f2c [0,4194304] 0 2024-04-24T22:45:28.766 INFO:tasks.workunit.client.1.smithi192.stdout:1/329: truncate d9/d15/d2d/d4a/f5b 28543 0 2024-04-24T22:45:28.767 INFO:tasks.workunit.client.1.smithi192.stdout:7/498: write f0 [4058697,33719] 0 2024-04-24T22:45:28.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/577: chown d3/c4 276363377 1 2024-04-24T22:45:28.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/441: unlink d3/dd/de/l1e 0 2024-04-24T22:45:28.773 INFO:tasks.workunit.client.0.smithi063.stdout:5/485: link d5/d24/d36/f5d d5/d1d/faa 0 2024-04-24T22:45:28.774 INFO:tasks.workunit.client.0.smithi063.stdout:0/428: dwrite d6/dd/f4d [0,4194304] 0 2024-04-24T22:45:28.775 INFO:tasks.workunit.client.1.smithi192.stdout:6/353: creat d0/d8/d29/d5a/d63/f81 x:0 0 0 2024-04-24T22:45:28.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/328: creat d0/d1b/f69 x:0 0 0 2024-04-24T22:45:28.781 INFO:tasks.workunit.client.0.smithi063.stdout:6/536: link db/d1d/d2d/f42 db/fb7 0 2024-04-24T22:45:28.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/429: dwrite d6/d64/d24/f72 [0,4194304] 0 2024-04-24T22:45:28.783 INFO:tasks.workunit.client.0.smithi063.stdout:4/447: symlink d2/d80/l93 0 2024-04-24T22:45:28.784 INFO:tasks.workunit.client.0.smithi063.stdout:4/448: read d2/f7c [1505169,98768] 0 2024-04-24T22:45:28.786 INFO:tasks.workunit.client.1.smithi192.stdout:5/329: dwrite d0/d13/d19/d64/d4f/d52/f67 [0,4194304] 0 2024-04-24T22:45:28.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/330: dread - d0/d13/d19/d64/d4f/d52/f61 zero size 2024-04-24T22:45:28.790 INFO:tasks.workunit.client.0.smithi063.stdout:8/506: symlink d1/la1 0 2024-04-24T22:45:28.790 INFO:tasks.workunit.client.1.smithi192.stdout:2/485: rmdir d4/d9/d10/d6c/d8c 39 2024-04-24T22:45:28.791 INFO:tasks.workunit.client.1.smithi192.stdout:2/486: chown d4/d9/d10/d6c/d6f/l7e 6499331 1 2024-04-24T22:45:28.795 INFO:tasks.workunit.client.1.smithi192.stdout:7/499: unlink d2/db/d2a/f2b 0 2024-04-24T22:45:28.797 INFO:tasks.workunit.client.1.smithi192.stdout:4/317: creat d2/d9/d25/f71 x:0 0 0 2024-04-24T22:45:28.799 INFO:tasks.workunit.client.0.smithi063.stdout:9/509: mkdir d6/d8/db/d2b/d48/d94/da1/da7 0 2024-04-24T22:45:28.801 INFO:tasks.workunit.client.0.smithi063.stdout:5/486: mknod d5/d1d/d7d/d67/cab 0 2024-04-24T22:45:28.803 INFO:tasks.workunit.client.1.smithi192.stdout:5/331: creat d0/d13/d19/d64/d4f/d52/f6a x:0 0 0 2024-04-24T22:45:28.804 INFO:tasks.workunit.client.1.smithi192.stdout:2/487: truncate d4/d9/d37/f62 33715 0 2024-04-24T22:45:28.805 INFO:tasks.workunit.client.0.smithi063.stdout:7/425: mknod d0/d4/de/d95/c97 0 2024-04-24T22:45:28.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/537: unlink db/d1d/d20/d3f/c44 0 2024-04-24T22:45:28.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/442: symlink d3/dd/d62/d6d/d91/l9a 0 2024-04-24T22:45:28.812 INFO:tasks.workunit.client.1.smithi192.stdout:4/318: unlink d2/f50 0 2024-04-24T22:45:28.812 INFO:tasks.workunit.client.0.smithi063.stdout:2/578: truncate d0/d2/d2d/f4d 886281 0 2024-04-24T22:45:28.813 INFO:tasks.workunit.client.1.smithi192.stdout:4/319: read d2/d9/d25/f68 [43253,59690] 0 2024-04-24T22:45:28.814 INFO:tasks.workunit.client.0.smithi063.stdout:2/579: chown d0/d2/d2d/d31/f5b 0 1 2024-04-24T22:45:28.814 INFO:tasks.workunit.client.0.smithi063.stdout:4/449: mknod d2/d19/d82/c94 0 2024-04-24T22:45:28.815 INFO:tasks.workunit.client.0.smithi063.stdout:2/580: truncate d0/d2/d27/f7f 666680 0 2024-04-24T22:45:28.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/522: write da/d18/d47/d4d/f72 [4929214,47567] 0 2024-04-24T22:45:28.816 INFO:tasks.workunit.client.0.smithi063.stdout:3/523: readlink da/d18/d47/l75 0 2024-04-24T22:45:28.817 INFO:tasks.workunit.client.1.smithi192.stdout:0/440: rmdir d3/d27/d28/d81 0 2024-04-24T22:45:28.818 INFO:tasks.workunit.client.0.smithi063.stdout:3/524: write da/fa2 [129699,77300] 0 2024-04-24T22:45:28.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/578: symlink d3/da/da2/dba/lc2 0 2024-04-24T22:45:28.819 INFO:tasks.workunit.client.1.smithi192.stdout:3/403: write d2/d4/d12/f1c [1998969,65835] 0 2024-04-24T22:45:28.820 INFO:tasks.workunit.client.1.smithi192.stdout:3/404: dread - d2/d4/d7/d4b/d54/f6a zero size 2024-04-24T22:45:28.820 INFO:tasks.workunit.client.0.smithi063.stdout:3/525: read da/d18/f8d [374182,71597] 0 2024-04-24T22:45:28.823 INFO:tasks.workunit.client.0.smithi063.stdout:5/487: rename d5/d24/d44/d71/fa2 to d5/d11/d23/d2d/d62/d73/d95/fac 0 2024-04-24T22:45:28.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/332: symlink d0/d1b/d50/l6b 0 2024-04-24T22:45:28.829 INFO:tasks.workunit.client.0.smithi063.stdout:5/488: dwrite f0 [0,4194304] 0 2024-04-24T22:45:28.830 INFO:tasks.workunit.client.1.smithi192.stdout:2/488: fdatasync d4/d9/f52 0 2024-04-24T22:45:28.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/489: write d5/d1d/f93 [299267,83284] 0 2024-04-24T22:45:28.831 INFO:tasks.workunit.client.0.smithi063.stdout:5/490: chown d5/f1c 25 1 2024-04-24T22:45:28.832 INFO:tasks.workunit.client.0.smithi063.stdout:0/430: getdents d6/d64/d21/d80 0 2024-04-24T22:45:28.834 INFO:tasks.workunit.client.0.smithi063.stdout:7/426: rmdir d0/d4/de 39 2024-04-24T22:45:28.835 INFO:tasks.workunit.client.1.smithi192.stdout:9/443: mkdir d3/d59/d7d/d9b 0 2024-04-24T22:45:28.836 INFO:tasks.workunit.client.1.smithi192.stdout:8/386: dread f9 [0,4194304] 0 2024-04-24T22:45:28.846 INFO:tasks.workunit.client.1.smithi192.stdout:6/354: link d0/d8/f14 d0/f82 0 2024-04-24T22:45:28.846 INFO:tasks.workunit.client.0.smithi063.stdout:9/510: dwrite d6/d8/d1f/d20/d22/f6d [4194304,4194304] 0 2024-04-24T22:45:28.847 INFO:tasks.workunit.client.0.smithi063.stdout:9/511: dread - d6/d2f/d42/d81/f95 zero size 2024-04-24T22:45:28.849 INFO:tasks.workunit.client.1.smithi192.stdout:0/441: symlink d3/d27/d28/d2e/d4a/d5c/d8a/l97 0 2024-04-24T22:45:28.850 INFO:tasks.workunit.client.1.smithi192.stdout:4/320: dwrite d2/d9/d25/d2e/d31/d36/d5b/f47 [0,4194304] 0 2024-04-24T22:45:28.851 INFO:tasks.workunit.client.1.smithi192.stdout:3/405: readlink d2/d4/d12/d1a/d1d/l4d 0 2024-04-24T22:45:28.853 INFO:tasks.workunit.client.1.smithi192.stdout:3/406: write d2/d4/d12/d39/d3a/f78 [334307,126026] 0 2024-04-24T22:45:28.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/407: readlink d2/d4/d7/d4b/d54/l5b 0 2024-04-24T22:45:28.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/431: mkdir d6/d64/d21/d25/d62/d63/d89/d95 0 2024-04-24T22:45:28.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/489: mknod d4/d29/d60/d75/c97 0 2024-04-24T22:45:28.865 INFO:tasks.workunit.client.0.smithi063.stdout:7/427: dread - d0/d4/de/d6f/d2e/f85 zero size 2024-04-24T22:45:28.867 INFO:tasks.workunit.client.1.smithi192.stdout:3/408: dwrite d2/d4/d7/d4f/f7a [0,4194304] 0 2024-04-24T22:45:28.868 INFO:tasks.workunit.client.1.smithi192.stdout:3/409: write d2/d4/f3e [2386559,75598] 0 2024-04-24T22:45:28.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/444: unlink d3/f43 0 2024-04-24T22:45:28.870 INFO:tasks.workunit.client.1.smithi192.stdout:9/445: fsync d3/d24/f7e 0 2024-04-24T22:45:28.872 INFO:tasks.workunit.client.1.smithi192.stdout:9/446: chown d3/dd/de/d17/l72 3 1 2024-04-24T22:45:28.876 INFO:tasks.workunit.client.1.smithi192.stdout:3/410: dwrite d2/d62/f75 [0,4194304] 0 2024-04-24T22:45:28.876 INFO:tasks.workunit.client.0.smithi063.stdout:9/512: rmdir d6/d8/d1f/d20/d2a/d4b/d5e/d9d 39 2024-04-24T22:45:28.876 INFO:tasks.workunit.client.0.smithi063.stdout:3/526: dwrite da/d18/f27 [0,4194304] 0 2024-04-24T22:45:28.878 INFO:tasks.workunit.client.1.smithi192.stdout:5/333: fdatasync d0/d13/d19/f57 0 2024-04-24T22:45:28.879 INFO:tasks.workunit.client.0.smithi063.stdout:3/527: dread - da/d18/d3a/f96 zero size 2024-04-24T22:45:28.880 INFO:tasks.workunit.client.1.smithi192.stdout:5/334: chown d0/d13/d19/d64/d5f/d2c/d3c 18518 1 2024-04-24T22:45:28.880 INFO:tasks.workunit.client.1.smithi192.stdout:5/335: read - d0/d1b/f65 zero size 2024-04-24T22:45:28.882 INFO:tasks.workunit.client.0.smithi063.stdout:7/428: creat d0/d4/de/d17/d28/d31/f98 x:0 0 0 2024-04-24T22:45:28.882 INFO:tasks.workunit.client.0.smithi063.stdout:3/528: fdatasync da/d39/fa4 0 2024-04-24T22:45:28.882 INFO:tasks.workunit.client.0.smithi063.stdout:4/450: link d2/d80/l93 d2/d80/l95 0 2024-04-24T22:45:28.886 INFO:tasks.workunit.client.1.smithi192.stdout:5/336: dread d0/d1b/f24 [0,4194304] 0 2024-04-24T22:45:28.887 INFO:tasks.workunit.client.0.smithi063.stdout:6/538: link db/d1d/d20/d3f/d5f/d93/c77 db/d1d/d20/d3f/d7f/d68/d70/cb8 0 2024-04-24T22:45:28.889 INFO:tasks.workunit.client.0.smithi063.stdout:7/429: mkdir d0/d4/de/d17/d28/d31/d99 0 2024-04-24T22:45:28.891 INFO:tasks.workunit.client.0.smithi063.stdout:2/581: getdents d0/d2/d2d/d31/d68 0 2024-04-24T22:45:28.893 INFO:tasks.workunit.client.0.smithi063.stdout:4/451: mkdir d2/d19/d82/d96 0 2024-04-24T22:45:28.894 INFO:tasks.workunit.client.0.smithi063.stdout:4/452: dread - d2/d8/d3c/f62 zero size 2024-04-24T22:45:28.895 INFO:tasks.workunit.client.0.smithi063.stdout:4/453: dread - d2/d8/d40/d51/f66 zero size 2024-04-24T22:45:28.898 INFO:tasks.workunit.client.0.smithi063.stdout:1/579: getdents d3/da/da2/dba/d11/d5e 0 2024-04-24T22:45:28.900 INFO:tasks.workunit.client.1.smithi192.stdout:0/442: write d3/d27/d28/d2e/d4a/f4d [4407287,96848] 0 2024-04-24T22:45:28.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/443: readlink d3/d27/d28/l21 0 2024-04-24T22:45:28.902 INFO:tasks.workunit.client.0.smithi063.stdout:6/539: mknod db/d1d/d20/d29/dab/cb9 0 2024-04-24T22:45:28.903 INFO:tasks.workunit.client.0.smithi063.stdout:8/507: sync 2024-04-24T22:45:28.904 INFO:tasks.workunit.client.0.smithi063.stdout:7/430: truncate d0/d4/f19 1857726 0 2024-04-24T22:45:28.905 INFO:tasks.workunit.client.1.smithi192.stdout:2/490: fsync d4/d9/d37/f94 0 2024-04-24T22:45:28.907 INFO:tasks.workunit.client.0.smithi063.stdout:1/580: fsync d3/da/da2/dba/d11/d5e/d6f/faf 0 2024-04-24T22:45:28.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/330: link d9/dd/le d9/d15/d2d/d31/l6b 0 2024-04-24T22:45:28.909 INFO:tasks.workunit.client.0.smithi063.stdout:6/540: dread db/d1d/d20/d3e/f43 [0,4194304] 0 2024-04-24T22:45:28.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/447: symlink d3/dd/de/d17/d6a/l9c 0 2024-04-24T22:45:28.916 INFO:tasks.workunit.client.0.smithi063.stdout:9/513: getdents d6/d2f/d42 0 2024-04-24T22:45:28.916 INFO:tasks.workunit.client.1.smithi192.stdout:4/321: symlink d2/db/l72 0 2024-04-24T22:45:28.921 INFO:tasks.workunit.client.1.smithi192.stdout:3/411: dwrite d2/d62/f7d [0,4194304] 0 2024-04-24T22:45:28.922 INFO:tasks.workunit.client.1.smithi192.stdout:2/491: mknod d4/d29/d60/d75/c98 0 2024-04-24T22:45:28.924 INFO:tasks.workunit.client.1.smithi192.stdout:1/331: chown d9/c11 59 1 2024-04-24T22:45:28.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/514: creat d6/d2f/d42/d81/fa8 x:0 0 0 2024-04-24T22:45:28.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/529: truncate da/d18/d3a/d63/f6e 86126 0 2024-04-24T22:45:28.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/541: symlink db/da0/lba 0 2024-04-24T22:45:28.930 INFO:tasks.workunit.client.1.smithi192.stdout:4/322: fdatasync d2/db/d10/f4f 0 2024-04-24T22:45:28.933 INFO:tasks.workunit.client.0.smithi063.stdout:1/581: truncate d3/da/da2/dba/d11/f35 3511055 0 2024-04-24T22:45:28.938 INFO:tasks.workunit.client.0.smithi063.stdout:8/508: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:28.938 INFO:tasks.workunit.client.1.smithi192.stdout:3/412: fdatasync d2/d27/d28/f63 0 2024-04-24T22:45:28.941 INFO:tasks.workunit.client.0.smithi063.stdout:7/431: write d0/d4/de/d17/d28/f64 [91237,83760] 0 2024-04-24T22:45:28.944 INFO:tasks.workunit.client.0.smithi063.stdout:9/515: rename d6/d8/db/d2b/f49 to d6/d2f/d42/d81/fa9 0 2024-04-24T22:45:28.946 INFO:tasks.workunit.client.1.smithi192.stdout:9/448: fsync d3/f4 0 2024-04-24T22:45:28.947 INFO:tasks.workunit.client.1.smithi192.stdout:9/449: truncate d3/dd/d53/d60/f74 283463 0 2024-04-24T22:45:28.950 INFO:tasks.workunit.client.0.smithi063.stdout:6/542: rmdir db/d1d/d2d 39 2024-04-24T22:45:28.951 INFO:tasks.workunit.client.1.smithi192.stdout:6/355: getdents d0/d8/d3d/d3e/d40/d44 0 2024-04-24T22:45:28.952 INFO:tasks.workunit.client.1.smithi192.stdout:6/356: read - d0/d8/d29/d4c/f6d zero size 2024-04-24T22:45:28.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/357: truncate d0/d8/d3d/f49 1069928 0 2024-04-24T22:45:28.954 INFO:tasks.workunit.client.1.smithi192.stdout:4/323: symlink d2/d9/d25/d2e/d31/l73 0 2024-04-24T22:45:28.956 INFO:tasks.workunit.client.0.smithi063.stdout:1/582: chown d3/da/da2/dba/d39/d5a/lad 247 1 2024-04-24T22:45:28.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/583: chown d3/da/da2/dba/d11/d5e/la4 80723758 1 2024-04-24T22:45:28.957 INFO:tasks.workunit.client.0.smithi063.stdout:5/491: sync 2024-04-24T22:45:28.958 INFO:tasks.workunit.client.0.smithi063.stdout:4/454: link d2/d8/f10 d2/f97 0 2024-04-24T22:45:28.962 INFO:tasks.workunit.client.1.smithi192.stdout:3/413: rename d2/d4/d12/d65/c79 to d2/d4/d7/d4b/c83 0 2024-04-24T22:45:28.963 INFO:tasks.workunit.client.1.smithi192.stdout:3/414: truncate d2/d4/d7/f60 748948 0 2024-04-24T22:45:28.969 INFO:tasks.workunit.client.0.smithi063.stdout:3/530: dwrite da/d39/f98 [0,4194304] 0 2024-04-24T22:45:28.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/415: dwrite d2/d4/f3e [0,4194304] 0 2024-04-24T22:45:28.971 INFO:tasks.workunit.client.0.smithi063.stdout:3/531: chown da/d18/d47/l51 51 1 2024-04-24T22:45:28.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/324: symlink d2/db/d10/l74 0 2024-04-24T22:45:28.976 INFO:tasks.workunit.client.0.smithi063.stdout:3/532: read da/d18/f8d [952385,113428] 0 2024-04-24T22:45:28.976 INFO:tasks.workunit.client.0.smithi063.stdout:1/584: symlink d3/db5/d2b/lc3 0 2024-04-24T22:45:28.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/416: dread d2/d62/f75 [0,4194304] 0 2024-04-24T22:45:28.981 INFO:tasks.workunit.client.0.smithi063.stdout:7/432: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:28.981 INFO:tasks.workunit.client.1.smithi192.stdout:3/417: dread d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:28.982 INFO:tasks.workunit.client.0.smithi063.stdout:4/455: symlink d2/d8/d5c/l98 0 2024-04-24T22:45:28.984 INFO:tasks.workunit.client.0.smithi063.stdout:7/433: write d0/d4/de/d17/d35/d69/f7b [2272677,118460] 0 2024-04-24T22:45:28.988 INFO:tasks.workunit.client.1.smithi192.stdout:2/492: dwrite d4/d9/f5b [0,4194304] 0 2024-04-24T22:45:28.989 INFO:tasks.workunit.client.1.smithi192.stdout:1/332: truncate d9/dd/f2b 2533117 0 2024-04-24T22:45:28.991 INFO:tasks.workunit.client.1.smithi192.stdout:3/418: symlink d2/d4/d4e/l84 0 2024-04-24T22:45:28.991 INFO:tasks.workunit.client.1.smithi192.stdout:1/333: read d9/d15/d2d/d31/d2f/f3a [3729949,15203] 0 2024-04-24T22:45:28.993 INFO:tasks.workunit.client.0.smithi063.stdout:9/516: mkdir d6/d8/db/d1c/d78/d9a/daa 0 2024-04-24T22:45:28.994 INFO:tasks.workunit.client.0.smithi063.stdout:9/517: write d6/d8/d1f/f28 [563906,79433] 0 2024-04-24T22:45:28.998 INFO:tasks.workunit.client.0.smithi063.stdout:5/492: write d5/d1d/faa [1749072,62293] 0 2024-04-24T22:45:28.999 INFO:tasks.workunit.client.1.smithi192.stdout:9/450: dwrite d3/f31 [0,4194304] 0 2024-04-24T22:45:29.001 INFO:tasks.workunit.client.1.smithi192.stdout:9/451: write d3/d59/f5b [958049,80183] 0 2024-04-24T22:45:29.001 INFO:tasks.workunit.client.1.smithi192.stdout:2/493: mknod d4/d9/d10/d6c/d95/c99 0 2024-04-24T22:45:29.004 INFO:tasks.workunit.client.0.smithi063.stdout:8/509: rename d1/d11/d49/f72 to d1/d11/d49/fa2 0 2024-04-24T22:45:29.005 INFO:tasks.workunit.client.0.smithi063.stdout:8/510: dread - d1/d11/d3c/f9e zero size 2024-04-24T22:45:29.008 INFO:tasks.workunit.client.1.smithi192.stdout:4/325: dwrite d2/d9/f3f [0,4194304] 0 2024-04-24T22:45:29.009 INFO:tasks.workunit.client.0.smithi063.stdout:5/493: dwrite d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:29.013 INFO:tasks.workunit.client.0.smithi063.stdout:7/434: rename d0/cc to d0/d4/de/d6f/d2e/d55/c9a 0 2024-04-24T22:45:29.016 INFO:tasks.workunit.client.0.smithi063.stdout:6/543: mknod db/d1d/d20/d29/cbb 0 2024-04-24T22:45:29.018 INFO:tasks.workunit.client.0.smithi063.stdout:7/435: write d0/d4/de/d17/d35/d69/f6b [291625,6059] 0 2024-04-24T22:45:29.018 INFO:tasks.workunit.client.1.smithi192.stdout:1/334: creat d9/d15/d2d/f6c x:0 0 0 2024-04-24T22:45:29.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/436: chown d0/l39 7113426 1 2024-04-24T22:45:29.020 INFO:tasks.workunit.client.1.smithi192.stdout:4/326: unlink d2/d9/d25/d2e/d31/d36/d5b/d64/f6f 0 2024-04-24T22:45:29.022 INFO:tasks.workunit.client.1.smithi192.stdout:2/494: unlink d4/d9/d10/l38 0 2024-04-24T22:45:29.023 INFO:tasks.workunit.client.0.smithi063.stdout:2/582: sync 2024-04-24T22:45:29.023 INFO:tasks.workunit.client.0.smithi063.stdout:0/432: sync 2024-04-24T22:45:29.024 INFO:tasks.workunit.client.1.smithi192.stdout:9/452: mknod d3/d5d/c9d 0 2024-04-24T22:45:29.024 INFO:tasks.workunit.client.0.smithi063.stdout:0/433: chown d6/d64/d21/f86 478921 1 2024-04-24T22:45:29.025 INFO:tasks.workunit.client.0.smithi063.stdout:2/583: write d0/d2/d27/f34 [903866,100369] 0 2024-04-24T22:45:29.030 INFO:tasks.workunit.client.0.smithi063.stdout:5/494: truncate d5/d24/d49/d3b/f58 414174 0 2024-04-24T22:45:29.032 INFO:tasks.workunit.client.0.smithi063.stdout:0/434: dwrite d6/d3a/f7a [0,4194304] 0 2024-04-24T22:45:29.032 INFO:tasks.workunit.client.0.smithi063.stdout:9/518: rename d6/d8/db/d2b/f3d to d6/d8/db/d1c/d78/d9a/daa/fab 0 2024-04-24T22:45:29.034 INFO:tasks.workunit.client.1.smithi192.stdout:9/453: creat d3/dd/d62/f9e x:0 0 0 2024-04-24T22:45:29.035 INFO:tasks.workunit.client.0.smithi063.stdout:3/533: link da/d18/d3a/l8f da/d18/d9e/da3/lb8 0 2024-04-24T22:45:29.038 INFO:tasks.workunit.client.1.smithi192.stdout:4/327: mkdir d2/db/d75 0 2024-04-24T22:45:29.039 INFO:tasks.workunit.client.1.smithi192.stdout:2/495: dwrite d4/d9/d10/d6c/d8c/f93 [0,4194304] 0 2024-04-24T22:45:29.047 INFO:tasks.workunit.client.1.smithi192.stdout:4/328: dwrite d2/d9/d25/d2e/d31/d36/d5b/d28/f61 [0,4194304] 0 2024-04-24T22:45:29.052 INFO:tasks.workunit.client.0.smithi063.stdout:4/456: link d2/d19/f37 d2/f99 0 2024-04-24T22:45:29.057 INFO:tasks.workunit.client.0.smithi063.stdout:2/584: mknod d0/d2/d2d/d31/d89/d7/d86/cad 0 2024-04-24T22:45:29.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/335: link d9/c11 d9/d15/d2d/c6d 0 2024-04-24T22:45:29.058 INFO:tasks.workunit.client.1.smithi192.stdout:1/336: truncate d9/f65 283420 0 2024-04-24T22:45:29.061 INFO:tasks.workunit.client.0.smithi063.stdout:8/511: dwrite d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:29.064 INFO:tasks.workunit.client.1.smithi192.stdout:1/337: mknod d9/d15/d2d/d4a/c6e 0 2024-04-24T22:45:29.066 INFO:tasks.workunit.client.0.smithi063.stdout:1/585: getdents d3/da/da2/dba/d11/d5e/d6f 0 2024-04-24T22:45:29.067 INFO:tasks.workunit.client.1.smithi192.stdout:1/338: mkdir d9/d6f 0 2024-04-24T22:45:29.067 INFO:tasks.workunit.client.0.smithi063.stdout:7/437: dwrite d0/d4/de/d6f/f53 [4194304,4194304] 0 2024-04-24T22:45:29.068 INFO:tasks.workunit.client.1.smithi192.stdout:1/339: truncate d9/d15/d2d/f6c 474429 0 2024-04-24T22:45:29.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/586: write d3/db5/d21/d9f/daa/db7/f73 [3984644,47529] 0 2024-04-24T22:45:29.070 INFO:tasks.workunit.client.0.smithi063.stdout:9/519: creat d6/d8/db/d93/da0/fac x:0 0 0 2024-04-24T22:45:29.071 INFO:tasks.workunit.client.0.smithi063.stdout:3/534: rename da/l37 to da/d40/lb9 0 2024-04-24T22:45:29.077 INFO:tasks.workunit.client.0.smithi063.stdout:9/520: dread d6/d8/db/d6b/f8f [0,4194304] 0 2024-04-24T22:45:29.078 INFO:tasks.workunit.client.0.smithi063.stdout:3/535: chown da/c11 26444650 1 2024-04-24T22:45:29.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/521: fdatasync d6/d8/db/d1c/f97 0 2024-04-24T22:45:29.080 INFO:tasks.workunit.client.0.smithi063.stdout:2/585: getdents d0/dd/d72/d77/d87 0 2024-04-24T22:45:29.080 INFO:tasks.workunit.client.0.smithi063.stdout:9/522: chown d6/l18 2 1 2024-04-24T22:45:29.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/457: rename d2/d8/f11 to d2/d8/d36/d3a/f9a 0 2024-04-24T22:45:29.083 INFO:tasks.workunit.client.0.smithi063.stdout:7/438: creat d0/d4/de/d17/f9b x:0 0 0 2024-04-24T22:45:29.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/458: fsync d2/f35 0 2024-04-24T22:45:29.086 INFO:tasks.workunit.client.1.smithi192.stdout:9/454: dread d3/dd/de/d17/f48 [0,4194304] 0 2024-04-24T22:45:29.087 INFO:tasks.workunit.client.0.smithi063.stdout:3/536: mknod da/cba 0 2024-04-24T22:45:29.088 INFO:tasks.workunit.client.0.smithi063.stdout:2/586: fsync d0/dd/f49 0 2024-04-24T22:45:29.089 INFO:tasks.workunit.client.1.smithi192.stdout:9/455: chown d3/d24/f4e 3961851 1 2024-04-24T22:45:29.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/537: read - da/d40/f9c zero size 2024-04-24T22:45:29.091 INFO:tasks.workunit.client.0.smithi063.stdout:5/495: stat d5/d24/d49/d3b/f58 0 2024-04-24T22:45:29.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/523: dwrite d6/d8/f71 [0,4194304] 0 2024-04-24T22:45:29.092 INFO:tasks.workunit.client.0.smithi063.stdout:7/439: stat d0/d4/de/d17/d35/l96 0 2024-04-24T22:45:29.092 INFO:tasks.workunit.client.1.smithi192.stdout:2/496: write d4/d9/f4c [212419,77519] 0 2024-04-24T22:45:29.093 INFO:tasks.workunit.client.0.smithi063.stdout:3/538: chown da/d18/d3a/d63 139488 1 2024-04-24T22:45:29.093 INFO:tasks.workunit.client.1.smithi192.stdout:9/456: mknod d3/dd/de/d17/d89/d8b/d8c/c9f 0 2024-04-24T22:45:29.094 INFO:tasks.workunit.client.0.smithi063.stdout:2/587: chown d0/d2/d2d/d31/f5b 8628 1 2024-04-24T22:45:29.094 INFO:tasks.workunit.client.1.smithi192.stdout:4/329: truncate d2/d9/d25/f68 223354 0 2024-04-24T22:45:29.095 INFO:tasks.workunit.client.0.smithi063.stdout:5/496: write d5/d1d/f93 [175301,55902] 0 2024-04-24T22:45:29.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/524: fdatasync d6/d8/d1f/d20/f21 0 2024-04-24T22:45:29.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/459: fdatasync d2/d8/d36/d3a/d50/f5d 0 2024-04-24T22:45:29.096 INFO:tasks.workunit.client.1.smithi192.stdout:4/330: write d2/f3 [988509,98135] 0 2024-04-24T22:45:29.098 INFO:tasks.workunit.client.0.smithi063.stdout:2/588: write d0/d2/d2d/d31/d89/d7/f5c [1497498,66716] 0 2024-04-24T22:45:29.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/525: chown d6/d8/f17 36 1 2024-04-24T22:45:29.100 INFO:tasks.workunit.client.0.smithi063.stdout:8/512: rename d1/d11/c81 to d1/d22/ca3 0 2024-04-24T22:45:29.100 INFO:tasks.workunit.client.0.smithi063.stdout:6/544: truncate db/f37 1763301 0 2024-04-24T22:45:29.102 INFO:tasks.workunit.client.1.smithi192.stdout:2/497: rename d4/d29/d60/d75 to d4/d29/d60/d9a 0 2024-04-24T22:45:29.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/526: dread - d6/d8/db/d6b/d4f/d59/f9b zero size 2024-04-24T22:45:29.103 INFO:tasks.workunit.client.0.smithi063.stdout:6/545: stat db/d1d/d20/d29/c2f 0 2024-04-24T22:45:29.103 INFO:tasks.workunit.client.0.smithi063.stdout:2/589: write d0/d2/d2d/d31/d89/f80 [293512,68219] 0 2024-04-24T22:45:29.105 INFO:tasks.workunit.client.0.smithi063.stdout:1/587: dread d3/da/da2/dba/d11/d5e/d6f/f90 [0,4194304] 0 2024-04-24T22:45:29.108 INFO:tasks.workunit.client.0.smithi063.stdout:1/588: stat d3/da/f6a 0 2024-04-24T22:45:29.108 INFO:tasks.workunit.client.1.smithi192.stdout:2/498: readlink d4/d29/d60/d9a/l7c 0 2024-04-24T22:45:29.109 INFO:tasks.workunit.client.1.smithi192.stdout:2/499: dread - d4/d9/f8b zero size 2024-04-24T22:45:29.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/539: chown da/f15 127 1 2024-04-24T22:45:29.111 INFO:tasks.workunit.client.0.smithi063.stdout:7/440: mkdir d0/d4/de/d6f/d5c/d9c 0 2024-04-24T22:45:29.112 INFO:tasks.workunit.client.1.smithi192.stdout:1/340: dwrite d9/dd/f41 [0,4194304] 0 2024-04-24T22:45:29.113 INFO:tasks.workunit.client.1.smithi192.stdout:1/341: dread - d9/d15/f49 zero size 2024-04-24T22:45:29.114 INFO:tasks.workunit.client.0.smithi063.stdout:3/540: chown da/d18/d3a/f69 1192035963 1 2024-04-24T22:45:29.115 INFO:tasks.workunit.client.0.smithi063.stdout:0/435: dwrite d6/d64/f3b [0,4194304] 0 2024-04-24T22:45:29.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/441: read d0/d4/de/f7c [1063389,111757] 0 2024-04-24T22:45:29.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/589: write d3/da/da2/dba/d11/f89 [3316880,66951] 0 2024-04-24T22:45:29.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/331: creat d2/f76 x:0 0 0 2024-04-24T22:45:29.120 INFO:tasks.workunit.client.0.smithi063.stdout:1/590: truncate d3/da/da2/dba/f8c 4294307 0 2024-04-24T22:45:29.120 INFO:tasks.workunit.client.1.smithi192.stdout:2/500: rename d4/d9/d37/l7d to d4/d9/d10/d6c/d8c/l9b 0 2024-04-24T22:45:29.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/591: dread - d3/da/d34/d6d/fa0 zero size 2024-04-24T22:45:29.124 INFO:tasks.workunit.client.1.smithi192.stdout:9/457: dread d3/dd/de/d17/f1d [0,4194304] 0 2024-04-24T22:45:29.125 INFO:tasks.workunit.client.0.smithi063.stdout:9/527: rename d6/d8/db/d2b to d6/d8/db/d6b/d4f/dad 0 2024-04-24T22:45:29.127 INFO:tasks.workunit.client.0.smithi063.stdout:0/436: dwrite d6/d64/d24/f59 [0,4194304] 0 2024-04-24T22:45:29.136 INFO:tasks.workunit.client.0.smithi063.stdout:5/497: dread d5/d11/d23/d2d/d62/d73/f75 [0,4194304] 0 2024-04-24T22:45:29.137 INFO:tasks.workunit.client.0.smithi063.stdout:9/528: dwrite d6/d8/db/d6b/f1e [0,4194304] 0 2024-04-24T22:45:29.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/332: mkdir d2/db/d5d/d77 0 2024-04-24T22:45:29.141 INFO:tasks.workunit.client.0.smithi063.stdout:0/437: dwrite d6/d64/d21/d25/d62/d63/f82 [0,4194304] 0 2024-04-24T22:45:29.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/442: symlink d0/d4/de/d6f/d2e/d55/l9d 0 2024-04-24T22:45:29.145 INFO:tasks.workunit.client.1.smithi192.stdout:1/342: mkdir d9/d51/d70 0 2024-04-24T22:45:29.148 INFO:tasks.workunit.client.1.smithi192.stdout:9/458: symlink d3/dd/de/d17/d89/d8b/d8c/d5e/d98/la0 0 2024-04-24T22:45:29.154 INFO:tasks.workunit.client.1.smithi192.stdout:2/501: write d4/d29/f2b [1536403,49430] 0 2024-04-24T22:45:29.161 INFO:tasks.workunit.client.0.smithi063.stdout:3/541: fdatasync da/d48/f5d 0 2024-04-24T22:45:29.162 INFO:tasks.workunit.client.0.smithi063.stdout:9/529: write d6/d8/f17 [2683772,122921] 0 2024-04-24T22:45:29.162 INFO:tasks.workunit.client.0.smithi063.stdout:5/498: unlink d5/d1d/f82 0 2024-04-24T22:45:29.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/502: creat d4/d9/d37/f9c x:0 0 0 2024-04-24T22:45:29.165 INFO:tasks.workunit.client.1.smithi192.stdout:4/333: rmdir d2/d5c/d5f 0 2024-04-24T22:45:29.167 INFO:tasks.workunit.client.0.smithi063.stdout:7/443: creat d0/d4/de/d17/d35/d6c/f9e x:0 0 0 2024-04-24T22:45:29.168 INFO:tasks.workunit.client.1.smithi192.stdout:2/503: stat d4/d9/f18 0 2024-04-24T22:45:29.173 INFO:tasks.workunit.client.0.smithi063.stdout:8/513: dwrite d1/f58 [4194304,4194304] 0 2024-04-24T22:45:29.173 INFO:tasks.workunit.client.1.smithi192.stdout:4/334: symlink d2/d9/d25/d2e/d31/d53/l78 0 2024-04-24T22:45:29.174 INFO:tasks.workunit.client.0.smithi063.stdout:7/444: dread d0/fa [0,4194304] 0 2024-04-24T22:45:29.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/504: symlink d4/d9/l9d 0 2024-04-24T22:45:29.176 INFO:tasks.workunit.client.1.smithi192.stdout:2/505: fdatasync d4/d9/d10/d6c/d8c/f93 0 2024-04-24T22:45:29.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/343: dread d9/dd/f2b [0,4194304] 0 2024-04-24T22:45:29.179 INFO:tasks.workunit.client.1.smithi192.stdout:9/459: dwrite d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:29.180 INFO:tasks.workunit.client.0.smithi063.stdout:8/514: dread d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:29.181 INFO:tasks.workunit.client.0.smithi063.stdout:5/499: creat d5/d11/d23/d2d/d62/fad x:0 0 0 2024-04-24T22:45:29.184 INFO:tasks.workunit.client.0.smithi063.stdout:0/438: creat d6/da/dc/d26/d33/d87/f96 x:0 0 0 2024-04-24T22:45:29.185 INFO:tasks.workunit.client.0.smithi063.stdout:9/530: creat d6/d8/db/d93/da0/fae x:0 0 0 2024-04-24T22:45:29.185 INFO:tasks.workunit.client.0.smithi063.stdout:1/592: creat d3/da/da2/dba/fc4 x:0 0 0 2024-04-24T22:45:29.186 INFO:tasks.workunit.client.0.smithi063.stdout:2/590: getdents d0/d2/d2d 0 2024-04-24T22:45:29.186 INFO:tasks.workunit.client.0.smithi063.stdout:9/531: chown d6/d8/d1f/d20/d2a/c92 131849850 1 2024-04-24T22:45:29.188 INFO:tasks.workunit.client.0.smithi063.stdout:3/542: symlink da/d39/d94/db3/lbb 0 2024-04-24T22:45:29.190 INFO:tasks.workunit.client.0.smithi063.stdout:5/500: read d5/d1d/faa [885723,40106] 0 2024-04-24T22:45:29.191 INFO:tasks.workunit.client.0.smithi063.stdout:1/593: write d3/f7 [2951001,75771] 0 2024-04-24T22:45:29.192 INFO:tasks.workunit.client.0.smithi063.stdout:2/591: mknod d0/dd/d62/d7c/cae 0 2024-04-24T22:45:29.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/532: creat d6/d8/db/d6b/d4f/d59/faf x:0 0 0 2024-04-24T22:45:29.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/335: rmdir d2/d9/d25 39 2024-04-24T22:45:29.199 INFO:tasks.workunit.client.1.smithi192.stdout:1/344: rename d9/f65 to d9/d15/d5e/d67/f71 0 2024-04-24T22:45:29.199 INFO:tasks.workunit.client.1.smithi192.stdout:9/460: dread - d3/dd/de/d17/d89/d8b/f99 zero size 2024-04-24T22:45:29.200 INFO:tasks.workunit.client.0.smithi063.stdout:3/543: dread da/f31 [0,4194304] 0 2024-04-24T22:45:29.200 INFO:tasks.workunit.client.0.smithi063.stdout:4/460: sync 2024-04-24T22:45:29.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/594: rename d3/da/l1a to d3/db5/d21/d9f/daa/db7/d6e/lc5 0 2024-04-24T22:45:29.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/445: dread d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:29.202 INFO:tasks.workunit.client.0.smithi063.stdout:4/461: dread - d2/d8/d40/d51/f69 zero size 2024-04-24T22:45:29.203 INFO:tasks.workunit.client.1.smithi192.stdout:2/506: dwrite d4/d29/d60/f89 [0,4194304] 0 2024-04-24T22:45:29.205 INFO:tasks.workunit.client.0.smithi063.stdout:1/595: chown d3/da/da2/dba 96339298 1 2024-04-24T22:45:29.206 INFO:tasks.workunit.client.0.smithi063.stdout:1/596: fdatasync d3/db5/f4d 0 2024-04-24T22:45:29.206 INFO:tasks.workunit.client.0.smithi063.stdout:5/501: dwrite d5/d1d/faa [0,4194304] 0 2024-04-24T22:45:29.207 INFO:tasks.workunit.client.0.smithi063.stdout:2/592: getdents d0/d2/d2d/d31/d89/d7/dac 0 2024-04-24T22:45:29.212 INFO:tasks.workunit.client.1.smithi192.stdout:1/345: dwrite d9/d15/d5e/d67/f71 [0,4194304] 0 2024-04-24T22:45:29.214 INFO:tasks.workunit.client.0.smithi063.stdout:4/462: unlink d2/d8/d36/l46 0 2024-04-24T22:45:29.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/515: write d1/d11/d49/d54/f60 [1533537,60583] 0 2024-04-24T22:45:29.216 INFO:tasks.workunit.client.1.smithi192.stdout:9/461: rename d3/dd/de/d17/d89/d8b/d8c to d3/d5d/da1 0 2024-04-24T22:45:29.217 INFO:tasks.workunit.client.0.smithi063.stdout:1/597: dread - d3/da/da2/dba/d11/d5e/d6f/faf zero size 2024-04-24T22:45:29.217 INFO:tasks.workunit.client.0.smithi063.stdout:4/463: stat d2/d8/d40/d51/f69 0 2024-04-24T22:45:29.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/507: symlink d4/d29/l9e 0 2024-04-24T22:45:29.219 INFO:tasks.workunit.client.0.smithi063.stdout:1/598: readlink d3/da/l85 0 2024-04-24T22:45:29.219 INFO:tasks.workunit.client.0.smithi063.stdout:5/502: truncate d5/d24/d49/d3b/f48 5748986 0 2024-04-24T22:45:29.220 INFO:tasks.workunit.client.0.smithi063.stdout:4/464: chown d2/d8/d40/d51/f6c 460 1 2024-04-24T22:45:29.222 INFO:tasks.workunit.client.0.smithi063.stdout:0/439: dwrite d6/da/dc/d26/f5f [0,4194304] 0 2024-04-24T22:45:29.223 INFO:tasks.workunit.client.1.smithi192.stdout:4/336: write d2/f1f [2137376,68922] 0 2024-04-24T22:45:29.224 INFO:tasks.workunit.client.1.smithi192.stdout:1/346: symlink d9/d15/d5e/l72 0 2024-04-24T22:45:29.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/508: truncate d4/d9/d10/f5d 416674 0 2024-04-24T22:45:29.228 INFO:tasks.workunit.client.1.smithi192.stdout:4/337: mkdir d2/d9/d6b/d79 0 2024-04-24T22:45:29.229 INFO:tasks.workunit.client.1.smithi192.stdout:4/338: fdatasync d2/f3 0 2024-04-24T22:45:29.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/347: dwrite d9/dd/f29 [4194304,4194304] 0 2024-04-24T22:45:29.232 INFO:tasks.workunit.client.0.smithi063.stdout:9/533: dwrite d6/d8/d1f/d20/d22/d31/f33 [0,4194304] 0 2024-04-24T22:45:29.233 INFO:tasks.workunit.client.1.smithi192.stdout:1/348: write d9/d15/f1f [4071921,37776] 0 2024-04-24T22:45:29.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/440: dwrite d6/d64/d24/f8b [0,4194304] 0 2024-04-24T22:45:29.236 INFO:tasks.workunit.client.1.smithi192.stdout:2/509: unlink d4/d29/d60/f63 0 2024-04-24T22:45:29.240 INFO:tasks.workunit.client.1.smithi192.stdout:4/339: unlink d2/db/c55 0 2024-04-24T22:45:29.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/462: link d3/dd/de/c1c d3/dd/d53/ca2 0 2024-04-24T22:45:29.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/463: dread - d3/dd/d62/f9e zero size 2024-04-24T22:45:29.246 INFO:tasks.workunit.client.0.smithi063.stdout:2/593: unlink d0/dd/d72/d77/c9f 0 2024-04-24T22:45:29.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/516: symlink d1/d11/d49/d54/d7b/la4 0 2024-04-24T22:45:29.248 INFO:tasks.workunit.client.0.smithi063.stdout:7/446: mkdir d0/d4/de/d6f/d2e/d76/d9f 0 2024-04-24T22:45:29.249 INFO:tasks.workunit.client.1.smithi192.stdout:9/464: fsync d3/f8 0 2024-04-24T22:45:29.249 INFO:tasks.workunit.client.0.smithi063.stdout:5/503: mknod d5/d24/d49/d3b/cae 0 2024-04-24T22:45:29.249 INFO:tasks.workunit.client.0.smithi063.stdout:7/447: truncate d0/d4/de/d6f/f53 8815896 0 2024-04-24T22:45:29.250 INFO:tasks.workunit.client.1.smithi192.stdout:9/465: stat d3/dd/de/c3d 0 2024-04-24T22:45:29.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/599: mkdir d3/da/da2/dba/d11/dc6 0 2024-04-24T22:45:29.252 INFO:tasks.workunit.client.1.smithi192.stdout:9/466: truncate d3/f8d 641753 0 2024-04-24T22:45:29.252 INFO:tasks.workunit.client.1.smithi192.stdout:2/510: dwrite d4/d9/d37/f55 [0,4194304] 0 2024-04-24T22:45:29.253 INFO:tasks.workunit.client.0.smithi063.stdout:4/465: creat d2/d8/d36/d3a/f9b x:0 0 0 2024-04-24T22:45:29.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/340: write d2/d9/d25/f32 [3811014,56961] 0 2024-04-24T22:45:29.253 INFO:tasks.workunit.client.0.smithi063.stdout:4/466: chown d2/d80/f8a 3367 1 2024-04-24T22:45:29.254 INFO:tasks.workunit.client.1.smithi192.stdout:2/511: read d4/d9/d37/f44 [507322,43256] 0 2024-04-24T22:45:29.254 INFO:tasks.workunit.client.0.smithi063.stdout:4/467: chown d2/d8/d3c/f62 810144113 1 2024-04-24T22:45:29.255 INFO:tasks.workunit.client.1.smithi192.stdout:2/512: truncate d4/d9/d37/f8a 800235 0 2024-04-24T22:45:29.256 INFO:tasks.workunit.client.0.smithi063.stdout:9/534: creat d6/d2f/d42/fb0 x:0 0 0 2024-04-24T22:45:29.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/468: write d2/d80/d8d/f76 [1360199,115011] 0 2024-04-24T22:45:29.257 INFO:tasks.workunit.client.1.smithi192.stdout:9/467: mknod d3/dd/de/d17/d89/ca3 0 2024-04-24T22:45:29.257 INFO:tasks.workunit.client.0.smithi063.stdout:5/504: dwrite d5/d1d/f64 [0,4194304] 0 2024-04-24T22:45:29.258 INFO:tasks.workunit.client.0.smithi063.stdout:4/469: chown d2/f35 236945256 1 2024-04-24T22:45:29.259 INFO:tasks.workunit.client.1.smithi192.stdout:2/513: truncate d4/d9/d37/f4a 769993 0 2024-04-24T22:45:29.259 INFO:tasks.workunit.client.1.smithi192.stdout:9/468: read - d3/dd/de/d17/d89/d8b/f82 zero size 2024-04-24T22:45:29.260 INFO:tasks.workunit.client.0.smithi063.stdout:2/594: stat d0/d2/d2d/d31/d89/f25 0 2024-04-24T22:45:29.261 INFO:tasks.workunit.client.0.smithi063.stdout:8/517: creat d1/d11/d32/fa5 x:0 0 0 2024-04-24T22:45:29.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/595: chown d0/l2c 7439603 1 2024-04-24T22:45:29.264 INFO:tasks.workunit.client.0.smithi063.stdout:7/448: mkdir d0/d4/de/d17/d35/d69/da0 0 2024-04-24T22:45:29.265 INFO:tasks.workunit.client.0.smithi063.stdout:1/600: mknod d3/da/da2/cc7 0 2024-04-24T22:45:29.265 INFO:tasks.workunit.client.0.smithi063.stdout:7/449: dread - d0/d4/de/d6f/d2e/f85 zero size 2024-04-24T22:45:29.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/514: dwrite d4/d9/f4c [0,4194304] 0 2024-04-24T22:45:29.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/450: chown d0/d4/de 104461480 1 2024-04-24T22:45:29.267 INFO:tasks.workunit.client.0.smithi063.stdout:1/601: chown d3/da/f5c 476 1 2024-04-24T22:45:29.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/535: unlink d6/d8/db/d1c/l43 0 2024-04-24T22:45:29.268 INFO:tasks.workunit.client.0.smithi063.stdout:0/441: creat d6/d74/f97 x:0 0 0 2024-04-24T22:45:29.269 INFO:tasks.workunit.client.1.smithi192.stdout:9/469: creat d3/dd/fa4 x:0 0 0 2024-04-24T22:45:29.271 INFO:tasks.workunit.client.0.smithi063.stdout:4/470: symlink d2/d19/d82/l9c 0 2024-04-24T22:45:29.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/596: creat d0/d2/d6c/faf x:0 0 0 2024-04-24T22:45:29.272 INFO:tasks.workunit.client.0.smithi063.stdout:4/471: chown d2/d80/f8f 0 1 2024-04-24T22:45:29.274 INFO:tasks.workunit.client.1.smithi192.stdout:1/349: dread d9/dd/f29 [0,4194304] 0 2024-04-24T22:45:29.274 INFO:tasks.workunit.client.1.smithi192.stdout:9/470: mknod d3/dd/de/d17/d89/d8b/ca5 0 2024-04-24T22:45:29.279 INFO:tasks.workunit.client.1.smithi192.stdout:1/350: creat d9/d15/d2d/d31/d2f/f73 x:0 0 0 2024-04-24T22:45:29.279 INFO:tasks.workunit.client.0.smithi063.stdout:5/505: mkdir d5/d11/d23/daf 0 2024-04-24T22:45:29.280 INFO:tasks.workunit.client.1.smithi192.stdout:1/351: write d9/d15/d2d/f6c [362521,108811] 0 2024-04-24T22:45:29.281 INFO:tasks.workunit.client.1.smithi192.stdout:1/352: write d9/d15/f47 [419129,38274] 0 2024-04-24T22:45:29.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/597: creat d0/d2/d63/fb0 x:0 0 0 2024-04-24T22:45:29.283 INFO:tasks.workunit.client.0.smithi063.stdout:2/598: chown d0/d2/d2d/d31/d89/d7/d86/da4/ca5 201 1 2024-04-24T22:45:29.284 INFO:tasks.workunit.client.0.smithi063.stdout:4/472: mkdir d2/d19/d33/d9d 0 2024-04-24T22:45:29.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/599: fsync d0/d2/d2d/d31/f92 0 2024-04-24T22:45:29.285 INFO:tasks.workunit.client.0.smithi063.stdout:0/442: fdatasync d6/da/f10 0 2024-04-24T22:45:29.288 INFO:tasks.workunit.client.0.smithi063.stdout:5/506: mknod d5/d24/d49/d3b/cb0 0 2024-04-24T22:45:29.292 INFO:tasks.workunit.client.1.smithi192.stdout:9/471: link d3/dd/de/d17/d6a/l5f d3/dd/de/la6 0 2024-04-24T22:45:29.295 INFO:tasks.workunit.client.0.smithi063.stdout:5/507: symlink d5/d24/d49/d3b/lb1 0 2024-04-24T22:45:29.297 INFO:tasks.workunit.client.1.smithi192.stdout:1/353: dwrite d9/f3c [0,4194304] 0 2024-04-24T22:45:29.297 INFO:tasks.workunit.client.0.smithi063.stdout:8/518: rename d1/d11/d49 to d1/d22/d33/da6 0 2024-04-24T22:45:29.298 INFO:tasks.workunit.client.0.smithi063.stdout:5/508: mknod d5/d11/d23/d2d/d62/cb2 0 2024-04-24T22:45:29.299 INFO:tasks.workunit.client.1.smithi192.stdout:9/472: mkdir d3/dd/de/da7 0 2024-04-24T22:45:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:28 smithi063 ceph-mon[26912]: pgmap v12: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 38 MiB/s rd, 223 MiB/s wr, 416 op/s 2024-04-24T22:45:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:28 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3875402426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:45:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:28 smithi063 ceph-mon[26912]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:28 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:28 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:29.306 INFO:tasks.workunit.client.0.smithi063.stdout:6/546: sync 2024-04-24T22:45:29.306 INFO:tasks.workunit.client.0.smithi063.stdout:8/519: symlink d1/d21/la7 0 2024-04-24T22:45:29.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/473: rmdir d2/d8/d36/d3a 39 2024-04-24T22:45:29.309 INFO:tasks.workunit.client.1.smithi192.stdout:1/354: mkdir d9/dd/d74 0 2024-04-24T22:45:29.310 INFO:tasks.workunit.client.1.smithi192.stdout:2/515: dread d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:29.311 INFO:tasks.workunit.client.0.smithi063.stdout:5/509: fsync d5/d11/d76/f9a 0 2024-04-24T22:45:29.312 INFO:tasks.workunit.client.1.smithi192.stdout:9/473: rmdir d3/dd/de/d17/d89 39 2024-04-24T22:45:29.313 INFO:tasks.workunit.client.1.smithi192.stdout:9/474: dread - d3/d24/f65 zero size 2024-04-24T22:45:29.314 INFO:tasks.workunit.client.1.smithi192.stdout:9/475: chown d3/dd/de/d17/f1d 222 1 2024-04-24T22:45:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:8/520: dwrite d1/d2/fe [0,4194304] 0 2024-04-24T22:45:29.316 INFO:tasks.workunit.client.0.smithi063.stdout:5/510: read d5/d11/f15 [140701,517] 0 2024-04-24T22:45:29.319 INFO:tasks.workunit.client.0.smithi063.stdout:4/474: fsync d2/f6 0 2024-04-24T22:45:29.319 INFO:tasks.workunit.client.1.smithi192.stdout:2/516: dread d4/d9/d10/f58 [0,4194304] 0 2024-04-24T22:45:29.320 INFO:tasks.workunit.client.1.smithi192.stdout:9/476: rmdir d3/dd/d62/d6d 39 2024-04-24T22:45:29.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/517: dwrite d4/f5e [0,4194304] 0 2024-04-24T22:45:29.330 INFO:tasks.workunit.client.0.smithi063.stdout:0/443: rename d6/d64/d21/d25/d62/d63/d91 to d6/da/dc/d26/d98 0 2024-04-24T22:45:29.333 INFO:tasks.workunit.client.0.smithi063.stdout:2/600: fsync d0/d2/d6c/faf 0 2024-04-24T22:45:29.334 INFO:tasks.workunit.client.0.smithi063.stdout:6/547: dread db/fb7 [0,4194304] 0 2024-04-24T22:45:29.334 INFO:tasks.workunit.client.0.smithi063.stdout:9/536: write d6/d8/d1f/d20/d2a/f6e [1763783,7326] 0 2024-04-24T22:45:29.336 INFO:tasks.workunit.client.0.smithi063.stdout:6/548: fdatasync db/d1d/d20/fa1 0 2024-04-24T22:45:29.337 INFO:tasks.workunit.client.0.smithi063.stdout:5/511: unlink d5/d1d/l56 0 2024-04-24T22:45:29.338 INFO:tasks.workunit.client.0.smithi063.stdout:0/444: dread d6/d64/d21/d25/d62/d75/f81 [0,4194304] 0 2024-04-24T22:45:29.342 INFO:tasks.workunit.client.1.smithi192.stdout:2/518: fsync d4/f13 0 2024-04-24T22:45:29.344 INFO:tasks.workunit.client.1.smithi192.stdout:9/477: mkdir d3/dd/de/d17/d89/d8b/da8 0 2024-04-24T22:45:29.345 INFO:tasks.workunit.client.0.smithi063.stdout:8/521: dread d1/d2/d17/f3d [0,4194304] 0 2024-04-24T22:45:29.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/602: truncate d3/db5/d2b/f32 2861339 0 2024-04-24T22:45:29.348 INFO:tasks.workunit.client.0.smithi063.stdout:1/603: read - d3/da/d34/fa1 zero size 2024-04-24T22:45:29.348 INFO:tasks.workunit.client.0.smithi063.stdout:1/604: dread - d3/db5/d21/f91 zero size 2024-04-24T22:45:29.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/512: dread d5/d11/d23/f69 [0,4194304] 0 2024-04-24T22:45:29.350 INFO:tasks.workunit.client.1.smithi192.stdout:1/355: rename c5 to d9/dd/c75 0 2024-04-24T22:45:29.352 INFO:tasks.workunit.client.0.smithi063.stdout:5/513: write d5/d24/f91 [409122,94058] 0 2024-04-24T22:45:29.354 INFO:tasks.workunit.client.0.smithi063.stdout:2/601: fdatasync d0/dd/f49 0 2024-04-24T22:45:29.356 INFO:tasks.workunit.client.0.smithi063.stdout:5/514: write d5/d11/d23/d2d/d62/d73/f74 [941666,92556] 0 2024-04-24T22:45:29.357 INFO:tasks.workunit.client.0.smithi063.stdout:9/537: readlink d6/d8/l41 0 2024-04-24T22:45:29.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/605: dwrite d3/da/da2/dba/d11/d5e/fa6 [0,4194304] 0 2024-04-24T22:45:29.361 INFO:tasks.workunit.client.1.smithi192.stdout:9/478: rename d3/dd/de/f7a to d3/d5d/fa9 0 2024-04-24T22:45:29.363 INFO:tasks.workunit.client.1.smithi192.stdout:1/356: creat d9/d15/d2d/d63/d68/f76 x:0 0 0 2024-04-24T22:45:29.365 INFO:tasks.workunit.client.0.smithi063.stdout:0/445: creat d6/da/dc/d26/d33/f99 x:0 0 0 2024-04-24T22:45:29.365 INFO:tasks.workunit.client.0.smithi063.stdout:8/522: dread d1/f2c [0,4194304] 0 2024-04-24T22:45:29.367 INFO:tasks.workunit.client.0.smithi063.stdout:8/523: truncate d1/d2/f84 832128 0 2024-04-24T22:45:29.370 INFO:tasks.workunit.client.1.smithi192.stdout:2/519: getdents d4 0 2024-04-24T22:45:29.371 INFO:tasks.workunit.client.1.smithi192.stdout:9/479: dread d3/dd/f30 [0,4194304] 0 2024-04-24T22:45:29.372 INFO:tasks.workunit.client.1.smithi192.stdout:9/480: fsync d3/d59/f5a 0 2024-04-24T22:45:29.372 INFO:tasks.workunit.client.1.smithi192.stdout:1/357: rename d9/f13 to d9/d6f/f77 0 2024-04-24T22:45:29.378 INFO:tasks.workunit.client.0.smithi063.stdout:5/515: symlink d5/d24/d44/d71/lb3 0 2024-04-24T22:45:29.379 INFO:tasks.workunit.client.0.smithi063.stdout:5/516: stat d5/d11/d23/d2d/d62/d81/d8f 0 2024-04-24T22:45:29.382 INFO:tasks.workunit.client.0.smithi063.stdout:4/475: write d2/d8/f2c [2405106,53470] 0 2024-04-24T22:45:29.382 INFO:tasks.workunit.client.1.smithi192.stdout:2/520: mkdir d4/d9/d9f 0 2024-04-24T22:45:29.385 INFO:tasks.workunit.client.1.smithi192.stdout:9/481: getdents d3/d59/d7d/d88 0 2024-04-24T22:45:29.385 INFO:tasks.workunit.client.0.smithi063.stdout:6/549: link db/l31 db/d1d/d20/d3f/d7f/d68/d70/lbc 0 2024-04-24T22:45:29.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/358: truncate d9/fa 4967252 0 2024-04-24T22:45:29.388 INFO:tasks.workunit.client.1.smithi192.stdout:2/521: dwrite d4/d9/f4c [0,4194304] 0 2024-04-24T22:45:29.389 INFO:tasks.workunit.client.0.smithi063.stdout:1/606: symlink d3/da/da2/dba/d39/dbf/lc8 0 2024-04-24T22:45:29.389 INFO:tasks.workunit.client.1.smithi192.stdout:2/522: readlink d4/d9/l9d 0 2024-04-24T22:45:29.390 INFO:tasks.workunit.client.1.smithi192.stdout:2/523: readlink d4/d9/d10/d6c/d6f/l7e 0 2024-04-24T22:45:29.391 INFO:tasks.workunit.client.1.smithi192.stdout:9/482: rename d3/f8 to d3/dd/d53/d60/faa 0 2024-04-24T22:45:29.394 INFO:tasks.workunit.client.0.smithi063.stdout:8/524: symlink d1/la8 0 2024-04-24T22:45:29.394 INFO:tasks.workunit.client.0.smithi063.stdout:8/525: stat d1/d22/l64 0 2024-04-24T22:45:29.397 INFO:tasks.workunit.client.0.smithi063.stdout:8/526: write d1/d22/d33/da6/d54/f60 [864096,85367] 0 2024-04-24T22:45:29.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/483: mkdir d3/dd/d53/dab 0 2024-04-24T22:45:29.398 INFO:tasks.workunit.client.0.smithi063.stdout:8/527: write d1/f5 [5192106,115788] 0 2024-04-24T22:45:29.398 INFO:tasks.workunit.client.1.smithi192.stdout:9/484: write d3/dd/d53/f56 [546692,98008] 0 2024-04-24T22:45:29.399 INFO:tasks.workunit.client.0.smithi063.stdout:3/544: sync 2024-04-24T22:45:29.399 INFO:tasks.workunit.client.1.smithi192.stdout:1/359: mkdir d9/d15/d2d/d78 0 2024-04-24T22:45:29.401 INFO:tasks.workunit.client.0.smithi063.stdout:8/528: fsync d1/d22/d33/da6/d54/d7b/f7f 0 2024-04-24T22:45:29.402 INFO:tasks.workunit.client.1.smithi192.stdout:1/360: write d9/f1a [3045360,98005] 0 2024-04-24T22:45:29.403 INFO:tasks.workunit.client.1.smithi192.stdout:9/485: creat d3/d59/d7d/d9b/fac x:0 0 0 2024-04-24T22:45:29.406 INFO:tasks.workunit.client.0.smithi063.stdout:0/446: rename d6/d64/d21/d25/d62/c68 to d6/da/dc/d26/c9a 0 2024-04-24T22:45:29.406 INFO:tasks.workunit.client.1.smithi192.stdout:1/361: creat d9/d15/d2d/d63/f79 x:0 0 0 2024-04-24T22:45:29.408 INFO:tasks.workunit.client.0.smithi063.stdout:9/538: link d6/l18 d6/d8/d1f/d20/d2a/d4b/d5e/d9d/lb1 0 2024-04-24T22:45:29.410 INFO:tasks.workunit.client.1.smithi192.stdout:9/486: symlink d3/dd/d53/dab/lad 0 2024-04-24T22:45:29.411 INFO:tasks.workunit.client.0.smithi063.stdout:1/607: symlink d3/da/da2/dba/d11/dc6/lc9 0 2024-04-24T22:45:29.412 INFO:tasks.workunit.client.0.smithi063.stdout:1/608: write d3/f7 [935702,41283] 0 2024-04-24T22:45:29.416 INFO:tasks.workunit.client.0.smithi063.stdout:4/476: write d2/d8/f3e [38834,59975] 0 2024-04-24T22:45:29.417 INFO:tasks.workunit.client.0.smithi063.stdout:5/517: creat d5/d24/d49/fb4 x:0 0 0 2024-04-24T22:45:29.418 INFO:tasks.workunit.client.1.smithi192.stdout:9/487: mkdir d3/dd/de/da7/dae 0 2024-04-24T22:45:29.420 INFO:tasks.workunit.client.1.smithi192.stdout:2/524: dread d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:29.425 INFO:tasks.workunit.client.0.smithi063.stdout:1/609: rename d3/db5/d21/d9f/daa/db7/l54 to d3/da/d34/lca 0 2024-04-24T22:45:29.425 INFO:tasks.workunit.client.0.smithi063.stdout:4/477: truncate d2/d8/d3c/f4e 28415 0 2024-04-24T22:45:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:28 smithi192 ceph-mon[33991]: pgmap v12: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 38 MiB/s rd, 223 MiB/s wr, 416 op/s 2024-04-24T22:45:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:28 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3875402426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:45:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:28 smithi192 ceph-mon[33991]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:45:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:28 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:28 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:29.428 INFO:tasks.workunit.client.0.smithi063.stdout:4/478: read - d2/d80/d81/d90/f92 zero size 2024-04-24T22:45:29.429 INFO:tasks.workunit.client.0.smithi063.stdout:8/529: link d1/d11/d32/d38/f3f d1/d11/d8b/fa9 0 2024-04-24T22:45:29.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/610: stat d3/db5/d21/d49/f9e 0 2024-04-24T22:45:29.430 INFO:tasks.workunit.client.0.smithi063.stdout:9/539: creat d6/d8/d1f/d20/d22/fb2 x:0 0 0 2024-04-24T22:45:29.430 INFO:tasks.workunit.client.0.smithi063.stdout:5/518: link d5/d1d/c35 d5/d11/d76/cb5 0 2024-04-24T22:45:29.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/479: unlink d2/d19/c56 0 2024-04-24T22:45:29.431 INFO:tasks.workunit.client.0.smithi063.stdout:3/545: dwrite da/d18/f2a [4194304,4194304] 0 2024-04-24T22:45:29.434 INFO:tasks.workunit.client.0.smithi063.stdout:8/530: creat d1/d21/faa x:0 0 0 2024-04-24T22:45:29.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/540: write d6/d8/db/d6b/d4f/d59/f9b [86449,14221] 0 2024-04-24T22:45:29.437 INFO:tasks.workunit.client.0.smithi063.stdout:0/447: truncate d6/da/f3d 3958183 0 2024-04-24T22:45:29.439 INFO:tasks.workunit.client.0.smithi063.stdout:3/546: truncate da/d18/d47/f6a 4821355 0 2024-04-24T22:45:29.442 INFO:tasks.workunit.client.0.smithi063.stdout:8/531: rmdir d1/d22/d33/da6/d54/d7b 39 2024-04-24T22:45:29.443 INFO:tasks.workunit.client.0.smithi063.stdout:0/448: stat d6/d64/d21/d25/d62/d63/d89/d95 0 2024-04-24T22:45:29.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/611: dwrite d3/da/d34/fa1 [0,4194304] 0 2024-04-24T22:45:29.451 INFO:tasks.workunit.client.0.smithi063.stdout:5/519: truncate d5/d11/d23/f79 1177813 0 2024-04-24T22:45:29.451 INFO:tasks.workunit.client.0.smithi063.stdout:4/480: link d2/d8/d36/d3a/f9a d2/d8/d36/d3a/d50/f9e 0 2024-04-24T22:45:29.452 INFO:tasks.workunit.client.0.smithi063.stdout:0/449: rmdir d6/d64/d21/d25/d62/d63/d89 39 2024-04-24T22:45:29.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/541: dwrite d6/d8/d1f/d20/f21 [0,4194304] 0 2024-04-24T22:45:29.454 INFO:tasks.workunit.client.0.smithi063.stdout:5/520: write d5/d24/d36/f8c [3817858,36927] 0 2024-04-24T22:45:29.454 INFO:tasks.workunit.client.0.smithi063.stdout:8/532: rename d1/d22/d33/da6/d54/f60 to d1/d22/fab 0 2024-04-24T22:45:29.455 INFO:tasks.workunit.client.0.smithi063.stdout:0/450: chown d6/d64/d24/f8b 781 1 2024-04-24T22:45:29.457 INFO:tasks.workunit.client.0.smithi063.stdout:4/481: mkdir d2/d9f 0 2024-04-24T22:45:29.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/521: write d5/d1d/faa [1023686,90646] 0 2024-04-24T22:45:29.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/542: mkdir d6/d8/db/d1c/db3 0 2024-04-24T22:45:29.460 INFO:tasks.workunit.client.0.smithi063.stdout:0/451: fdatasync d6/d64/d24/f8b 0 2024-04-24T22:45:29.461 INFO:tasks.workunit.client.0.smithi063.stdout:1/612: dwrite d3/da/da2/dba/d39/d5a/f8f [0,4194304] 0 2024-04-24T22:45:29.467 INFO:tasks.workunit.client.0.smithi063.stdout:9/543: unlink d6/d8/db/d6b/d4f/dad/c91 0 2024-04-24T22:45:29.472 INFO:tasks.workunit.client.0.smithi063.stdout:0/452: write d6/da/d2b/f40 [2068307,80626] 0 2024-04-24T22:45:29.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/522: mkdir d5/d24/db6 0 2024-04-24T22:45:29.482 INFO:tasks.workunit.client.0.smithi063.stdout:8/533: dread d1/d22/d33/da6/d54/f71 [0,4194304] 0 2024-04-24T22:45:29.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/602: sync 2024-04-24T22:45:29.482 INFO:tasks.workunit.client.0.smithi063.stdout:7/451: sync 2024-04-24T22:45:29.482 INFO:tasks.workunit.client.0.smithi063.stdout:4/482: dwrite d2/d80/d8d/f89 [0,4194304] 0 2024-04-24T22:45:29.482 INFO:tasks.workunit.client.0.smithi063.stdout:1/613: dwrite d3/db5/d21/d9f/daa/db7/f58 [4194304,4194304] 0 2024-04-24T22:45:29.487 INFO:tasks.workunit.client.0.smithi063.stdout:7/452: write d0/d4/de/d17/d35/d6c/f9e [242802,7749] 0 2024-04-24T22:45:29.489 INFO:tasks.workunit.client.0.smithi063.stdout:8/534: chown d1/d2/d17/l97 1 1 2024-04-24T22:45:29.490 INFO:tasks.workunit.client.0.smithi063.stdout:1/614: truncate d3/f23 3856338 0 2024-04-24T22:45:29.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/535: chown d1/d11/d3c/l52 3 1 2024-04-24T22:45:29.494 INFO:tasks.workunit.client.1.smithi192.stdout:2/525: mknod d4/ca0 0 2024-04-24T22:45:29.497 INFO:tasks.workunit.client.0.smithi063.stdout:4/483: dwrite d2/d8/f1a [0,4194304] 0 2024-04-24T22:45:29.498 INFO:tasks.workunit.client.0.smithi063.stdout:7/453: dread d0/d4/de/f7c [0,4194304] 0 2024-04-24T22:45:29.498 INFO:tasks.workunit.client.1.smithi192.stdout:9/488: write d3/f71 [188536,79370] 0 2024-04-24T22:45:29.501 INFO:tasks.workunit.client.1.smithi192.stdout:1/362: dwrite d9/dd/f2b [0,4194304] 0 2024-04-24T22:45:29.502 INFO:tasks.workunit.client.0.smithi063.stdout:4/484: chown d2/d8/l6e 13 1 2024-04-24T22:45:29.502 INFO:tasks.workunit.client.0.smithi063.stdout:0/453: dread d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:29.503 INFO:tasks.workunit.client.0.smithi063.stdout:7/454: fsync d0/d4/de/d17/d35/f54 0 2024-04-24T22:45:29.506 INFO:tasks.workunit.client.1.smithi192.stdout:9/489: dread d3/dd/de/d17/f48 [0,4194304] 0 2024-04-24T22:45:29.509 INFO:tasks.workunit.client.0.smithi063.stdout:4/485: mkdir d2/d8/d40/da0 0 2024-04-24T22:45:29.511 INFO:tasks.workunit.client.0.smithi063.stdout:4/486: truncate d2/f99 356354 0 2024-04-24T22:45:29.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/603: dread d0/d2/d2d/f81 [0,4194304] 0 2024-04-24T22:45:29.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/615: truncate d3/db5/d21/d9f/daa/db7/f73 2835636 0 2024-04-24T22:45:29.517 INFO:tasks.workunit.client.1.smithi192.stdout:1/363: dread d9/d51/f54 [0,4194304] 0 2024-04-24T22:45:29.517 INFO:tasks.workunit.client.0.smithi063.stdout:2/604: readlink d0/dd/d72/d77/l90 0 2024-04-24T22:45:29.518 INFO:tasks.workunit.client.0.smithi063.stdout:7/455: write d0/f7 [742721,85467] 0 2024-04-24T22:45:29.519 INFO:tasks.workunit.client.0.smithi063.stdout:0/454: getdents d6/d64/d21/d25/d62/d63/d7c 0 2024-04-24T22:45:29.520 INFO:tasks.workunit.client.0.smithi063.stdout:0/455: stat d6/l88 0 2024-04-24T22:45:29.520 INFO:tasks.workunit.client.0.smithi063.stdout:2/605: truncate d0/fa6 294118 0 2024-04-24T22:45:29.521 INFO:tasks.workunit.client.1.smithi192.stdout:9/490: link d3/dd/de/d17/d89/d8b/ca5 d3/dd/de/caf 0 2024-04-24T22:45:29.523 INFO:tasks.workunit.client.0.smithi063.stdout:7/456: stat d0/d4/de/d6f/d2e/l65 0 2024-04-24T22:45:29.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/456: readlink d6/da/d2b/l77 0 2024-04-24T22:45:29.523 INFO:tasks.workunit.client.1.smithi192.stdout:1/364: creat d9/dd/d74/f7a x:0 0 0 2024-04-24T22:45:29.524 INFO:tasks.workunit.client.0.smithi063.stdout:1/616: dwrite d3/f6c [4194304,4194304] 0 2024-04-24T22:45:29.524 INFO:tasks.workunit.client.0.smithi063.stdout:2/606: fdatasync d0/dd/f99 0 2024-04-24T22:45:29.526 INFO:tasks.workunit.client.1.smithi192.stdout:9/491: rmdir d3/dd/de/d17/d89/d8b 39 2024-04-24T22:45:29.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/457: chown d6/dd/f3f 23777130 1 2024-04-24T22:45:29.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/457: stat d0/d4/de/d6f/d2e/d76/d9f 0 2024-04-24T22:45:29.530 INFO:tasks.workunit.client.1.smithi192.stdout:9/492: stat d3/dd/d53/ca2 0 2024-04-24T22:45:29.531 INFO:tasks.workunit.client.0.smithi063.stdout:4/487: dwrite d2/f99 [0,4194304] 0 2024-04-24T22:45:29.534 INFO:tasks.workunit.client.0.smithi063.stdout:9/544: dread d6/d2f/f35 [0,4194304] 0 2024-04-24T22:45:29.534 INFO:tasks.workunit.client.0.smithi063.stdout:0/458: truncate d6/da/dc/d26/d33/d87/f96 15775 0 2024-04-24T22:45:29.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/365: write d9/dd/f19 [916128,124772] 0 2024-04-24T22:45:29.537 INFO:tasks.workunit.client.0.smithi063.stdout:0/459: write d6/d64/d24/f8b [4853608,122574] 0 2024-04-24T22:45:29.540 INFO:tasks.workunit.client.1.smithi192.stdout:9/493: rename d3/dd/de/d17/d6a to d3/db0 0 2024-04-24T22:45:29.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/366: mknod d9/d15/d2d/d63/d68/c7b 0 2024-04-24T22:45:29.544 INFO:tasks.workunit.client.1.smithi192.stdout:9/494: symlink d3/d59/d7d/d9b/lb1 0 2024-04-24T22:45:29.544 INFO:tasks.workunit.client.1.smithi192.stdout:1/367: read d9/f14 [1197808,115881] 0 2024-04-24T22:45:29.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/495: creat d3/d5d/fb2 x:0 0 0 2024-04-24T22:45:29.547 INFO:tasks.workunit.client.1.smithi192.stdout:1/368: chown d9/d15/c5d 9 1 2024-04-24T22:45:29.548 INFO:tasks.workunit.client.1.smithi192.stdout:9/496: dread - d3/f58 zero size 2024-04-24T22:45:29.549 INFO:tasks.workunit.client.0.smithi063.stdout:9/545: creat d6/d8/d1f/d20/d2a/d4b/d5e/fb4 x:0 0 0 2024-04-24T22:45:29.550 INFO:tasks.workunit.client.0.smithi063.stdout:0/460: creat d6/da/dc/d26/f9b x:0 0 0 2024-04-24T22:45:29.550 INFO:tasks.workunit.client.1.smithi192.stdout:1/369: chown d9/c11 7284574 1 2024-04-24T22:45:29.553 INFO:tasks.workunit.client.0.smithi063.stdout:7/458: rename d0/d4/de/d6f/d2e/d55/c9a to d0/d4/de/d17/ca1 0 2024-04-24T22:45:29.554 INFO:tasks.workunit.client.1.smithi192.stdout:9/497: rename d3/d24/l90 to d3/d5d/da1/d85/lb3 0 2024-04-24T22:45:29.554 INFO:tasks.workunit.client.0.smithi063.stdout:7/459: fdatasync d0/d4/de/d17/f22 0 2024-04-24T22:45:29.556 INFO:tasks.workunit.client.1.smithi192.stdout:1/370: mkdir d9/d15/d5e/d7c 0 2024-04-24T22:45:29.556 INFO:tasks.workunit.client.0.smithi063.stdout:7/460: readlink d0/d4/de/d17/d28/l2d 0 2024-04-24T22:45:29.556 INFO:tasks.workunit.client.1.smithi192.stdout:1/371: stat d9/d15/f49 0 2024-04-24T22:45:29.557 INFO:tasks.workunit.client.0.smithi063.stdout:9/546: dwrite d6/d8/d1f/d20/d2a/f6e [0,4194304] 0 2024-04-24T22:45:29.557 INFO:tasks.workunit.client.0.smithi063.stdout:0/461: mkdir d6/d64/d21/d9c 0 2024-04-24T22:45:29.558 INFO:tasks.workunit.client.1.smithi192.stdout:9/498: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:29.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/372: chown d9/d15/d2d/d31/c27 2967643 1 2024-04-24T22:45:29.561 INFO:tasks.workunit.client.1.smithi192.stdout:9/499: mkdir d3/dd/de/d17/d89/db4 0 2024-04-24T22:45:29.561 INFO:tasks.workunit.client.1.smithi192.stdout:9/500: chown d3/dd/d62/f92 633 1 2024-04-24T22:45:29.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/373: symlink d9/d15/l7d 0 2024-04-24T22:45:29.562 INFO:tasks.workunit.client.1.smithi192.stdout:9/501: chown d3/dd/de/d17/c76 283 1 2024-04-24T22:45:29.564 INFO:tasks.workunit.client.0.smithi063.stdout:4/488: dwrite d2/d8/d36/f44 [0,4194304] 0 2024-04-24T22:45:29.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/547: rename d6/d8/db/d1c/d78/c9c to d6/d8/db/d1c/d78/d9a/cb5 0 2024-04-24T22:45:29.566 INFO:tasks.workunit.client.0.smithi063.stdout:1/617: dread d3/f19 [0,4194304] 0 2024-04-24T22:45:29.566 INFO:tasks.workunit.client.0.smithi063.stdout:3/547: sync 2024-04-24T22:45:29.566 INFO:tasks.workunit.client.0.smithi063.stdout:6/550: sync 2024-04-24T22:45:29.568 INFO:tasks.workunit.client.1.smithi192.stdout:1/374: fdatasync d9/fa 0 2024-04-24T22:45:29.568 INFO:tasks.workunit.client.0.smithi063.stdout:6/551: fsync db/d1d/d20/d3f/d7f/d68/d70/f67 0 2024-04-24T22:45:29.569 INFO:tasks.workunit.client.0.smithi063.stdout:6/552: readlink db/d1d/la3 0 2024-04-24T22:45:29.569 INFO:tasks.workunit.client.1.smithi192.stdout:9/502: dwrite d3/d24/f44 [0,4194304] 0 2024-04-24T22:45:29.570 INFO:tasks.workunit.client.0.smithi063.stdout:0/462: link d6/d64/d21/d25/f7b d6/d64/d21/d9c/f9d 0 2024-04-24T22:45:29.572 INFO:tasks.workunit.client.0.smithi063.stdout:4/489: rmdir d2/d8/d5c 39 2024-04-24T22:45:29.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/490: write d2/d80/d81/d90/f92 [975738,7046] 0 2024-04-24T22:45:29.578 INFO:tasks.workunit.client.0.smithi063.stdout:3/548: readlink da/d18/l2f 0 2024-04-24T22:45:29.586 INFO:tasks.workunit.client.0.smithi063.stdout:1/618: dread d3/db5/f71 [0,4194304] 0 2024-04-24T22:45:29.601 INFO:tasks.workunit.client.0.smithi063.stdout:7/461: link d0/d2f/f37 d0/fa2 0 2024-04-24T22:45:29.605 INFO:tasks.workunit.client.1.smithi192.stdout:7/500: sync 2024-04-24T22:45:29.605 INFO:tasks.workunit.client.1.smithi192.stdout:5/337: sync 2024-04-24T22:45:29.605 INFO:tasks.workunit.client.1.smithi192.stdout:8/387: sync 2024-04-24T22:45:29.606 INFO:tasks.workunit.client.1.smithi192.stdout:7/501: write d2/db/d49/f70 [8206744,11424] 0 2024-04-24T22:45:29.625 INFO:tasks.workunit.client.0.smithi063.stdout:5/523: sync 2024-04-24T22:45:29.625 INFO:tasks.workunit.client.0.smithi063.stdout:8/536: sync 2024-04-24T22:45:29.627 INFO:tasks.workunit.client.0.smithi063.stdout:8/537: read d1/d11/d32/d38/fa0 [8006994,22940] 0 2024-04-24T22:45:29.629 INFO:tasks.workunit.client.0.smithi063.stdout:6/553: mkdir db/da0/dbd 0 2024-04-24T22:45:29.632 INFO:tasks.workunit.client.0.smithi063.stdout:0/463: rmdir d6/d64/d24 39 2024-04-24T22:45:29.633 INFO:tasks.workunit.client.1.smithi192.stdout:5/338: mkdir d0/d13/d19/d64/d4f/d6c 0 2024-04-24T22:45:29.633 INFO:tasks.workunit.client.0.smithi063.stdout:0/464: chown d6/d64/d21/d25/d62/d75/l7d 87834370 1 2024-04-24T22:45:29.635 INFO:tasks.workunit.client.0.smithi063.stdout:4/491: truncate d2/d8/f23 137084 0 2024-04-24T22:45:29.638 INFO:tasks.workunit.client.0.smithi063.stdout:4/492: read d2/d19/f4d [1113,98770] 0 2024-04-24T22:45:29.639 INFO:tasks.workunit.client.0.smithi063.stdout:9/548: getdents d6/d8/db/d1c/db3 0 2024-04-24T22:45:29.644 INFO:tasks.workunit.client.0.smithi063.stdout:7/462: fdatasync d0/fa 0 2024-04-24T22:45:29.645 INFO:tasks.workunit.client.1.smithi192.stdout:7/502: link d2/db/d21/d42/d67/c78 d2/db/d21/d42/d67/d9b/d6e/d77/c9c 0 2024-04-24T22:45:29.646 INFO:tasks.workunit.client.1.smithi192.stdout:7/503: truncate d2/db/f50 65479 0 2024-04-24T22:45:29.647 INFO:tasks.workunit.client.1.smithi192.stdout:7/504: fdatasync d2/db/d36/f3d 0 2024-04-24T22:45:29.648 INFO:tasks.workunit.client.1.smithi192.stdout:8/388: getdents db/d43 0 2024-04-24T22:45:29.649 INFO:tasks.workunit.client.1.smithi192.stdout:8/389: stat db/f34 0 2024-04-24T22:45:29.651 INFO:tasks.workunit.client.1.smithi192.stdout:7/505: chown d2/db/d25/d82/c95 70746595 1 2024-04-24T22:45:29.651 INFO:tasks.workunit.client.0.smithi063.stdout:4/493: dwrite d2/f74 [0,4194304] 0 2024-04-24T22:45:29.652 INFO:tasks.workunit.client.1.smithi192.stdout:1/375: dwrite d9/d15/d2d/d31/d2f/f3f [0,4194304] 0 2024-04-24T22:45:29.653 INFO:tasks.workunit.client.0.smithi063.stdout:9/549: dwrite d6/d2f/f72 [0,4194304] 0 2024-04-24T22:45:29.653 INFO:tasks.workunit.client.1.smithi192.stdout:5/339: creat d0/d1b/f6d x:0 0 0 2024-04-24T22:45:29.654 INFO:tasks.workunit.client.0.smithi063.stdout:6/554: symlink db/d1d/d20/d29/d5d/da6/lbe 0 2024-04-24T22:45:29.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/494: fdatasync d2/d8/d36/d3a/d50/f77 0 2024-04-24T22:45:29.659 INFO:tasks.workunit.client.1.smithi192.stdout:8/390: mkdir db/d5d/d74 0 2024-04-24T22:45:29.662 INFO:tasks.workunit.client.1.smithi192.stdout:7/506: unlink d2/db/d21/d42/d67/d9b/l88 0 2024-04-24T22:45:29.662 INFO:tasks.workunit.client.0.smithi063.stdout:2/607: sync 2024-04-24T22:45:29.664 INFO:tasks.workunit.client.0.smithi063.stdout:0/465: dread f5 [0,4194304] 0 2024-04-24T22:45:29.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/340: symlink d0/d13/d19/d64/d5f/d66/l6e 0 2024-04-24T22:45:29.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/463: chown d0/d4/de/d17/d35/c71 232 1 2024-04-24T22:45:29.670 INFO:tasks.workunit.client.0.smithi063.stdout:7/464: chown d0/d4/de/d6f/d5c/d63 7 1 2024-04-24T22:45:29.671 INFO:tasks.workunit.client.0.smithi063.stdout:3/549: dread da/f3d [0,4194304] 0 2024-04-24T22:45:29.671 INFO:tasks.workunit.client.0.smithi063.stdout:2/608: read d0/d2/d27/f29 [3151055,58434] 0 2024-04-24T22:45:29.672 INFO:tasks.workunit.client.0.smithi063.stdout:9/550: dread d6/d2f/d42/f7b [0,4194304] 0 2024-04-24T22:45:29.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/609: readlink d0/d2/d6c/l97 0 2024-04-24T22:45:29.675 INFO:tasks.workunit.client.1.smithi192.stdout:1/376: mknod d9/dd/c7e 0 2024-04-24T22:45:29.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/550: fsync da/d18/d47/d4d/f72 0 2024-04-24T22:45:29.679 INFO:tasks.workunit.client.1.smithi192.stdout:8/391: truncate f6 6052796 0 2024-04-24T22:45:29.682 INFO:tasks.workunit.client.1.smithi192.stdout:1/377: dread - d9/f4f zero size 2024-04-24T22:45:29.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/392: dwrite db/f65 [0,4194304] 0 2024-04-24T22:45:29.690 INFO:tasks.workunit.client.0.smithi063.stdout:9/551: readlink d6/d8/l2e 0 2024-04-24T22:45:29.758 INFO:tasks.workunit.client.1.smithi192.stdout:1/378: link d9/d15/f2e d9/d51/d70/f7f 0 2024-04-24T22:45:29.759 INFO:tasks.workunit.client.0.smithi063.stdout:4/495: symlink d2/d9f/la1 0 2024-04-24T22:45:29.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/619: rename d3/db5/d21/d9f/daa/db7/d6e to d3/da/da2/dba/d11/d5e/d9c/dcb 0 2024-04-24T22:45:29.761 INFO:tasks.workunit.client.1.smithi192.stdout:1/379: dread d9/d15/d2d/f6c [0,4194304] 0 2024-04-24T22:45:29.764 INFO:tasks.workunit.client.0.smithi063.stdout:0/466: creat d6/d64/d21/d80/f9e x:0 0 0 2024-04-24T22:45:29.765 INFO:tasks.workunit.client.0.smithi063.stdout:0/467: chown d6/da/d2b/l37 71750830 1 2024-04-24T22:45:29.765 INFO:tasks.workunit.client.1.smithi192.stdout:8/393: getdents db/dc/d22 0 2024-04-24T22:45:29.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/468: dread - d6/d64/d21/d80/f9e zero size 2024-04-24T22:45:29.767 INFO:tasks.workunit.client.0.smithi063.stdout:3/551: sync 2024-04-24T22:45:29.767 INFO:tasks.workunit.client.0.smithi063.stdout:8/538: getdents d1/d21 0 2024-04-24T22:45:29.769 INFO:tasks.workunit.client.0.smithi063.stdout:8/539: read d1/d11/f47 [2723859,12372] 0 2024-04-24T22:45:29.771 INFO:tasks.workunit.client.0.smithi063.stdout:7/465: mkdir d0/d4/de/d6f/d2e/d76/d9f/da3 0 2024-04-24T22:45:29.772 INFO:tasks.workunit.client.0.smithi063.stdout:7/466: write d0/d4/de/d17/d28/d31/f6a [412508,122799] 0 2024-04-24T22:45:29.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/469: creat d6/d74/f9f x:0 0 0 2024-04-24T22:45:29.781 INFO:tasks.workunit.client.1.smithi192.stdout:1/380: dread d9/d15/f2e [0,4194304] 0 2024-04-24T22:45:29.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/394: rename db/d5d/d73 to db/d75 0 2024-04-24T22:45:29.788 INFO:tasks.workunit.client.0.smithi063.stdout:9/552: creat d6/d8/db/d1c/db3/fb6 x:0 0 0 2024-04-24T22:45:29.789 INFO:tasks.workunit.client.1.smithi192.stdout:8/395: mkdir db/d43/d76 0 2024-04-24T22:45:29.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/396: truncate db/dc/d22/d31/f50 115769 0 2024-04-24T22:45:29.790 INFO:tasks.workunit.client.0.smithi063.stdout:3/552: dwrite da/f60 [0,4194304] 0 2024-04-24T22:45:29.791 INFO:tasks.workunit.client.0.smithi063.stdout:8/540: write d1/d21/f6f [1642200,4702] 0 2024-04-24T22:45:29.792 INFO:tasks.workunit.client.0.smithi063.stdout:3/553: stat da/d18/d47/d4d/d62/f80 0 2024-04-24T22:45:29.794 INFO:tasks.workunit.client.1.smithi192.stdout:8/397: mkdir db/dc/d22/d31/d33/d77 0 2024-04-24T22:45:29.798 INFO:tasks.workunit.client.0.smithi063.stdout:7/467: dwrite d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:29.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/398: dwrite db/dc/f41 [0,4194304] 0 2024-04-24T22:45:29.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/541: mkdir d1/d2/d3/dac 0 2024-04-24T22:45:29.807 INFO:tasks.workunit.client.1.smithi192.stdout:8/399: symlink db/d43/d71/l78 0 2024-04-24T22:45:29.809 INFO:tasks.workunit.client.0.smithi063.stdout:3/554: mkdir da/d39/d94/dbc 0 2024-04-24T22:45:29.809 INFO:tasks.workunit.client.0.smithi063.stdout:7/468: read d0/d4/de/d17/f22 [2404854,104644] 0 2024-04-24T22:45:29.812 INFO:tasks.workunit.client.0.smithi063.stdout:7/469: chown d0/f73 175 1 2024-04-24T22:45:29.814 INFO:tasks.workunit.client.0.smithi063.stdout:3/555: rename da/d18/d9e/fa5 to da/d18/d3a/da1/fbd 0 2024-04-24T22:45:29.818 INFO:tasks.workunit.client.0.smithi063.stdout:3/556: mkdir da/d39/d94/db3/dbe 0 2024-04-24T22:45:29.821 INFO:tasks.workunit.client.0.smithi063.stdout:8/542: link d1/d2/d3/f82 d1/d22/d33/da6/d54/fad 0 2024-04-24T22:45:29.826 INFO:tasks.workunit.client.0.smithi063.stdout:3/557: unlink da/c77 0 2024-04-24T22:45:29.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/543: mkdir d1/d22/d33/da6/dae 0 2024-04-24T22:45:29.828 INFO:tasks.workunit.client.1.smithi192.stdout:8/400: mknod db/dc/d29/c79 0 2024-04-24T22:45:29.830 INFO:tasks.workunit.client.0.smithi063.stdout:3/558: symlink da/d18/d47/d7b/lbf 0 2024-04-24T22:45:29.830 INFO:tasks.workunit.client.0.smithi063.stdout:8/544: creat d1/d11/d3c/faf x:0 0 0 2024-04-24T22:45:29.835 INFO:tasks.workunit.client.0.smithi063.stdout:3/559: mknod da/d48/cc0 0 2024-04-24T22:45:29.838 INFO:tasks.workunit.client.0.smithi063.stdout:3/560: mkdir da/d18/d9e/da3/dc1 0 2024-04-24T22:45:29.838 INFO:tasks.workunit.client.0.smithi063.stdout:7/470: dread d0/d4/de/f8d [0,4194304] 0 2024-04-24T22:45:29.839 INFO:tasks.workunit.client.0.smithi063.stdout:7/471: fdatasync d0/d4/de/d17/f22 0 2024-04-24T22:45:29.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/545: dread d1/f1f [0,4194304] 0 2024-04-24T22:45:29.842 INFO:tasks.workunit.client.0.smithi063.stdout:7/472: chown d0/d4/de/d6f/l3e 423 1 2024-04-24T22:45:29.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/473: stat d0/l39 0 2024-04-24T22:45:29.843 INFO:tasks.workunit.client.0.smithi063.stdout:8/546: creat d1/d11/d32/d38/fb0 x:0 0 0 2024-04-24T22:45:29.847 INFO:tasks.workunit.client.0.smithi063.stdout:7/474: truncate d0/d4/de/d6f/f47 402938 0 2024-04-24T22:45:29.849 INFO:tasks.workunit.client.0.smithi063.stdout:7/475: mknod d0/d4/de/d17/d35/ca4 0 2024-04-24T22:45:29.849 INFO:tasks.workunit.client.0.smithi063.stdout:7/476: fsync d0/d4/de/d17/f22 0 2024-04-24T22:45:29.852 INFO:tasks.workunit.client.0.smithi063.stdout:7/477: creat d0/d4/de/d17/d35/d6c/d7a/fa5 x:0 0 0 2024-04-24T22:45:29.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/478: dwrite d0/d4/de/d17/f9b [0,4194304] 0 2024-04-24T22:45:29.863 INFO:tasks.workunit.client.0.smithi063.stdout:6/555: dwrite db/d1d/d20/f38 [0,4194304] 0 2024-04-24T22:45:29.864 INFO:tasks.workunit.client.0.smithi063.stdout:7/479: dread d0/d4/de/f10 [0,4194304] 0 2024-04-24T22:45:29.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/556: chown db/d1d/d20/d3f/d5f/lb3 13 1 2024-04-24T22:45:29.869 INFO:tasks.workunit.client.0.smithi063.stdout:6/557: creat db/d1c/fbf x:0 0 0 2024-04-24T22:45:29.869 INFO:tasks.workunit.client.0.smithi063.stdout:6/558: dread - db/d1c/fbf zero size 2024-04-24T22:45:29.870 INFO:tasks.workunit.client.0.smithi063.stdout:2/610: unlink d0/f40 0 2024-04-24T22:45:29.877 INFO:tasks.workunit.client.0.smithi063.stdout:2/611: rename d0/d2/d39/l4c to d0/d2/d2d/d31/d89/d1c/d3f/lb1 0 2024-04-24T22:45:29.879 INFO:tasks.workunit.client.0.smithi063.stdout:7/480: symlink d0/d4/de/d17/d28/d31/d94/la6 0 2024-04-24T22:45:29.882 INFO:tasks.workunit.client.0.smithi063.stdout:4/496: unlink d2/d19/l65 0 2024-04-24T22:45:29.885 INFO:tasks.workunit.client.0.smithi063.stdout:2/612: symlink d0/lb2 0 2024-04-24T22:45:29.886 INFO:tasks.workunit.client.0.smithi063.stdout:6/559: dread db/d1d/d20/d84/fae [0,4194304] 0 2024-04-24T22:45:29.886 INFO:tasks.workunit.client.0.smithi063.stdout:7/481: dread d0/d4/de/f61 [0,4194304] 0 2024-04-24T22:45:29.887 INFO:tasks.workunit.client.0.smithi063.stdout:4/497: unlink d2/f7e 0 2024-04-24T22:45:29.888 INFO:tasks.workunit.client.0.smithi063.stdout:1/620: write d3/da/da2/dba/d11/f35 [1825482,125778] 0 2024-04-24T22:45:29.890 INFO:tasks.workunit.client.0.smithi063.stdout:0/470: write d6/d64/d21/d9c/f9d [4228241,87981] 0 2024-04-24T22:45:29.892 INFO:tasks.workunit.client.1.smithi192.stdout:6/358: sync 2024-04-24T22:45:29.892 INFO:tasks.workunit.client.1.smithi192.stdout:0/444: sync 2024-04-24T22:45:29.893 INFO:tasks.workunit.client.1.smithi192.stdout:6/359: fdatasync d0/d8/fe 0 2024-04-24T22:45:29.893 INFO:tasks.workunit.client.0.smithi063.stdout:9/553: write d6/d2f/d42/f7b [2111106,76535] 0 2024-04-24T22:45:29.894 INFO:tasks.workunit.client.0.smithi063.stdout:7/482: truncate d0/d4/de/d17/d35/d6c/f9e 484115 0 2024-04-24T22:45:29.894 INFO:tasks.workunit.client.1.smithi192.stdout:6/360: write d0/d8/d29/d5a/d63/f81 [796422,31167] 0 2024-04-24T22:45:29.897 INFO:tasks.workunit.client.0.smithi063.stdout:5/524: mknod d5/d11/d23/d2d/cb7 0 2024-04-24T22:45:29.898 INFO:tasks.workunit.client.0.smithi063.stdout:2/613: dwrite d0/d2/d2d/d31/f5b [0,4194304] 0 2024-04-24T22:45:29.899 INFO:tasks.workunit.client.0.smithi063.stdout:6/560: dwrite db/d1c/f90 [0,4194304] 0 2024-04-24T22:45:29.901 INFO:tasks.workunit.client.1.smithi192.stdout:6/361: dwrite d0/d8/f2a [0,4194304] 0 2024-04-24T22:45:29.901 INFO:tasks.workunit.client.0.smithi063.stdout:9/554: fdatasync d6/f30 0 2024-04-24T22:45:29.902 INFO:tasks.workunit.client.1.smithi192.stdout:7/507: symlink d2/db/d21/l9d 0 2024-04-24T22:45:29.903 INFO:tasks.workunit.client.0.smithi063.stdout:1/621: dwrite d3/da/d34/f9a [0,4194304] 0 2024-04-24T22:45:29.911 INFO:tasks.workunit.client.0.smithi063.stdout:9/555: dwrite d6/d8/db/d93/da0/fac [0,4194304] 0 2024-04-24T22:45:29.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/362: dwrite d0/d8/d3d/f55 [0,4194304] 0 2024-04-24T22:45:29.925 INFO:tasks.workunit.client.0.smithi063.stdout:6/561: dread - db/d1d/d2d/f76 zero size 2024-04-24T22:45:29.926 INFO:tasks.workunit.client.0.smithi063.stdout:7/483: rename d0/d4/de/d17/c90 to d0/d4/de/d6f/d2e/d76/d9f/ca7 0 2024-04-24T22:45:29.927 INFO:tasks.workunit.client.0.smithi063.stdout:6/562: write db/d1d/d20/d29/d5d/f6b [1729475,66582] 0 2024-04-24T22:45:29.927 INFO:tasks.workunit.client.1.smithi192.stdout:7/508: creat d2/db/d21/d42/d67/d9b/d6e/f9e x:0 0 0 2024-04-24T22:45:29.928 INFO:tasks.workunit.client.0.smithi063.stdout:6/563: stat db/d1d/d20/l9c 0 2024-04-24T22:45:29.930 INFO:tasks.workunit.client.1.smithi192.stdout:3/419: sync 2024-04-24T22:45:29.930 INFO:tasks.workunit.client.1.smithi192.stdout:4/341: sync 2024-04-24T22:45:29.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/420: chown d2/d27/d28/d38/l51 122 1 2024-04-24T22:45:29.932 INFO:tasks.workunit.client.1.smithi192.stdout:8/401: write db/dc/d22/f5e [81762,38918] 0 2024-04-24T22:45:29.932 INFO:tasks.workunit.client.0.smithi063.stdout:4/498: getdents d2/d19/d82 0 2024-04-24T22:45:29.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/342: dread d2/db/f3d [0,4194304] 0 2024-04-24T22:45:29.936 INFO:tasks.workunit.client.1.smithi192.stdout:0/445: symlink d3/d27/d28/d2e/d4a/d5c/d80/d83/l98 0 2024-04-24T22:45:29.942 INFO:tasks.workunit.client.1.smithi192.stdout:8/402: dwrite db/dc/d22/f23 [0,4194304] 0 2024-04-24T22:45:29.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/421: dread d2/d4/d7/d4f/f52 [4194304,4194304] 0 2024-04-24T22:45:29.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/403: read - db/dc/d22/f66 zero size 2024-04-24T22:45:29.946 INFO:tasks.workunit.client.0.smithi063.stdout:8/547: write d1/d11/d8b/fa9 [1845559,78939] 0 2024-04-24T22:45:29.946 INFO:tasks.workunit.client.1.smithi192.stdout:4/343: read d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f3c [1286456,15515] 0 2024-04-24T22:45:29.949 INFO:tasks.workunit.client.1.smithi192.stdout:0/446: mknod d3/d27/d3c/d5d/c99 0 2024-04-24T22:45:29.950 INFO:tasks.workunit.client.0.smithi063.stdout:3/561: dwrite da/d18/f30 [0,4194304] 0 2024-04-24T22:45:29.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/344: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:29.960 INFO:tasks.workunit.client.0.smithi063.stdout:0/471: getdents d6/d64/d21/d25 0 2024-04-24T22:45:29.960 INFO:tasks.workunit.client.0.smithi063.stdout:9/556: mknod d6/d8/cb7 0 2024-04-24T22:45:29.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/557: dread - d6/d2f/d42/d81/f95 zero size 2024-04-24T22:45:29.966 INFO:tasks.workunit.client.1.smithi192.stdout:0/447: creat d3/d27/d28/d2e/d4a/d5c/f9a x:0 0 0 2024-04-24T22:45:29.966 INFO:tasks.workunit.client.0.smithi063.stdout:7/484: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:29.967 INFO:tasks.workunit.client.0.smithi063.stdout:5/525: dread d5/d11/d18/f3a [0,4194304] 0 2024-04-24T22:45:29.967 INFO:tasks.workunit.client.1.smithi192.stdout:0/448: truncate d3/d27/d28/d2e/d4a/d5c/f9a 225123 0 2024-04-24T22:45:29.968 INFO:tasks.workunit.client.0.smithi063.stdout:1/622: link d3/db5/d2b/f32 d3/da/da2/dba/d11/d5e/d9c/dcb/fcc 0 2024-04-24T22:45:29.975 INFO:tasks.workunit.client.1.smithi192.stdout:8/404: symlink db/dc/d22/d31/l7a 0 2024-04-24T22:45:29.982 INFO:tasks.workunit.client.1.smithi192.stdout:8/405: dwrite db/dc/d29/f2b [0,4194304] 0 2024-04-24T22:45:29.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/499: link d2/f5 d2/d19/fa2 0 2024-04-24T22:45:29.986 INFO:tasks.workunit.client.0.smithi063.stdout:5/526: mknod d5/d11/d23/d2d/d62/d81/cb8 0 2024-04-24T22:45:29.988 INFO:tasks.workunit.client.1.smithi192.stdout:8/406: readlink db/dc/l3a 0 2024-04-24T22:45:29.989 INFO:tasks.workunit.client.0.smithi063.stdout:9/558: dread d6/d8/db/d6b/f46 [0,4194304] 0 2024-04-24T22:45:29.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/422: dread d2/d4/d12/f1c [0,4194304] 0 2024-04-24T22:45:29.990 INFO:tasks.workunit.client.0.smithi063.stdout:1/623: symlink d3/da/da2/lcd 0 2024-04-24T22:45:29.990 INFO:tasks.workunit.client.0.smithi063.stdout:9/559: write d6/d8/db/d6b/d4f/dad/d48/d94/fa6 [78514,87875] 0 2024-04-24T22:45:29.991 INFO:tasks.workunit.client.0.smithi063.stdout:1/624: fsync d3/db5/d2b/d79/fac 0 2024-04-24T22:45:29.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/423: dread d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:29.994 INFO:tasks.workunit.client.0.smithi063.stdout:7/485: symlink d0/d4/de/d17/d35/d69/da0/la8 0 2024-04-24T22:45:29.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/625: truncate d3/da/da2/dba/d11/f5f 511693 0 2024-04-24T22:45:29.995 INFO:tasks.workunit.client.1.smithi192.stdout:3/424: read d2/f29 [126576,119300] 0 2024-04-24T22:45:29.998 INFO:tasks.workunit.client.0.smithi063.stdout:0/472: rename d6/da/dc/d26 to d6/d64/da0 0 2024-04-24T22:45:30.003 INFO:tasks.workunit.client.0.smithi063.stdout:5/527: symlink d5/d24/lb9 0 2024-04-24T22:45:30.005 INFO:tasks.workunit.client.0.smithi063.stdout:5/528: read d5/d24/f86 [1351277,22152] 0 2024-04-24T22:45:30.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/473: creat d6/da/d2b/d60/fa1 x:0 0 0 2024-04-24T22:45:30.011 INFO:tasks.workunit.client.1.smithi192.stdout:8/407: dread db/dc/f5b [0,4194304] 0 2024-04-24T22:45:30.012 INFO:tasks.workunit.client.0.smithi063.stdout:0/474: symlink d6/d74/la2 0 2024-04-24T22:45:30.013 INFO:tasks.workunit.client.1.smithi192.stdout:8/408: fsync db/dc/d22/d31/f45 0 2024-04-24T22:45:30.014 INFO:tasks.workunit.client.1.smithi192.stdout:8/409: stat db/dc/d22/f5e 0 2024-04-24T22:45:30.016 INFO:tasks.workunit.client.1.smithi192.stdout:8/410: read - db/dc/d22/f35 zero size 2024-04-24T22:45:30.018 INFO:tasks.workunit.client.0.smithi063.stdout:2/614: sync 2024-04-24T22:45:30.018 INFO:tasks.workunit.client.1.smithi192.stdout:8/411: readlink db/dc/ld 0 2024-04-24T22:45:30.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/509: truncate d2/db/d36/f64 265600 0 2024-04-24T22:45:30.019 INFO:tasks.workunit.client.1.smithi192.stdout:8/412: mknod db/d5d/d74/c7b 0 2024-04-24T22:45:30.022 INFO:tasks.workunit.client.1.smithi192.stdout:8/413: rmdir db/d5d 39 2024-04-24T22:45:30.022 INFO:tasks.workunit.client.0.smithi063.stdout:2/615: symlink d0/d2/d2d/d31/d89/d7/lb3 0 2024-04-24T22:45:30.023 INFO:tasks.workunit.client.0.smithi063.stdout:0/475: dread d6/d64/d21/d25/d31/f35 [0,4194304] 0 2024-04-24T22:45:30.026 INFO:tasks.workunit.client.1.smithi192.stdout:7/510: link d2/db/d2a/d57/l5f d2/db/d49/l9f 0 2024-04-24T22:45:30.027 INFO:tasks.workunit.client.1.smithi192.stdout:8/414: dwrite db/d43/f4e [0,4194304] 0 2024-04-24T22:45:30.028 INFO:tasks.workunit.client.0.smithi063.stdout:2/616: dread d0/d2/d2d/d31/d89/d7/f5c [0,4194304] 0 2024-04-24T22:45:30.029 INFO:tasks.workunit.client.0.smithi063.stdout:2/617: write d0/d2/d2d/d31/f5b [2065124,122576] 0 2024-04-24T22:45:30.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/511: mknod d2/db/d21/d42/d5b/ca0 0 2024-04-24T22:45:30.032 INFO:tasks.workunit.client.0.smithi063.stdout:8/548: dwrite d1/d22/d33/da6/d54/fad [0,4194304] 0 2024-04-24T22:45:30.034 INFO:tasks.workunit.client.0.smithi063.stdout:8/549: chown d1/d2/d3/f14 710546 1 2024-04-24T22:45:30.035 INFO:tasks.workunit.client.1.smithi192.stdout:7/512: mknod d2/db/d21/d42/d67/ca1 0 2024-04-24T22:45:30.037 INFO:tasks.workunit.client.1.smithi192.stdout:7/513: read d2/db/f6f [277328,14141] 0 2024-04-24T22:45:30.037 INFO:tasks.workunit.client.0.smithi063.stdout:8/550: fdatasync d1/d22/d33/da6/fa2 0 2024-04-24T22:45:30.037 INFO:tasks.workunit.client.0.smithi063.stdout:2/618: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:30.038 INFO:tasks.workunit.client.1.smithi192.stdout:7/514: write d2/db/f50 [816449,120867] 0 2024-04-24T22:45:30.039 INFO:tasks.workunit.client.0.smithi063.stdout:3/562: truncate da/d18/f27 798298 0 2024-04-24T22:45:30.040 INFO:tasks.workunit.client.1.smithi192.stdout:4/345: truncate d2/d9/d25/d2e/d31/d36/d5b/f47 480389 0 2024-04-24T22:45:30.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/346: chown d2/d9/d6b/l6d 89 1 2024-04-24T22:45:30.043 INFO:tasks.workunit.client.0.smithi063.stdout:8/551: mkdir d1/d11/d32/d38/db1 0 2024-04-24T22:45:30.046 INFO:tasks.workunit.client.0.smithi063.stdout:2/619: dwrite d0/d2/d2d/d31/f5b [4194304,4194304] 0 2024-04-24T22:45:30.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/515: mkdir d2/db/d21/d42/d67/d9b/d6e/da2 0 2024-04-24T22:45:30.048 INFO:tasks.workunit.client.1.smithi192.stdout:7/516: write d2/db/d21/d42/f8c [666132,64962] 0 2024-04-24T22:45:30.049 INFO:tasks.workunit.client.0.smithi063.stdout:3/563: mkdir da/d18/d47/d4d/dc2 0 2024-04-24T22:45:30.049 INFO:tasks.workunit.client.0.smithi063.stdout:4/500: write d2/d8/d3c/f75 [402218,97889] 0 2024-04-24T22:45:30.050 INFO:tasks.workunit.client.1.smithi192.stdout:7/517: write d2/db/d2a/f61 [4085685,41249] 0 2024-04-24T22:45:30.052 INFO:tasks.workunit.client.0.smithi063.stdout:2/620: unlink d0/d2/d2d/d31/d89/d7/lb3 0 2024-04-24T22:45:30.057 INFO:tasks.workunit.client.0.smithi063.stdout:7/486: truncate d0/d4/de/f8d 3245487 0 2024-04-24T22:45:30.058 INFO:tasks.workunit.client.0.smithi063.stdout:9/560: dwrite d6/d8/d1f/f52 [0,4194304] 0 2024-04-24T22:45:30.058 INFO:tasks.workunit.client.0.smithi063.stdout:4/501: unlink d2/d8/c52 0 2024-04-24T22:45:30.059 INFO:tasks.workunit.client.1.smithi192.stdout:3/425: dwrite d2/d4/d12/f42 [0,4194304] 0 2024-04-24T22:45:30.062 INFO:tasks.workunit.client.0.smithi063.stdout:2/621: unlink d0/d2/d2d/d31/d68/la9 0 2024-04-24T22:45:30.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/626: truncate d3/da/f6a 1612222 0 2024-04-24T22:45:30.065 INFO:tasks.workunit.client.0.smithi063.stdout:3/564: rename da/fc to da/d39/d94/db3/fc3 0 2024-04-24T22:45:30.067 INFO:tasks.workunit.client.1.smithi192.stdout:4/347: creat d2/db/f7a x:0 0 0 2024-04-24T22:45:30.067 INFO:tasks.workunit.client.0.smithi063.stdout:3/565: chown da/d18/d47/f6c 111366272 1 2024-04-24T22:45:30.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/487: creat d0/d4/de/d95/fa9 x:0 0 0 2024-04-24T22:45:30.069 INFO:tasks.workunit.client.1.smithi192.stdout:3/426: dread - d2/d4/f69 zero size 2024-04-24T22:45:30.072 INFO:tasks.workunit.client.0.smithi063.stdout:5/529: dwrite d5/d24/d36/f63 [0,4194304] 0 2024-04-24T22:45:30.076 INFO:tasks.workunit.client.0.smithi063.stdout:2/622: mknod d0/d2/d2d/d31/d89/d7/d86/da4/cb4 0 2024-04-24T22:45:30.076 INFO:tasks.workunit.client.1.smithi192.stdout:4/348: dwrite d2/d9/d25/d2e/d31/d36/d5b/f6e [0,4194304] 0 2024-04-24T22:45:30.083 INFO:tasks.workunit.client.0.smithi063.stdout:1/627: dwrite d3/da/da2/dba/d11/d5e/d6f/f90 [4194304,4194304] 0 2024-04-24T22:45:30.084 INFO:tasks.workunit.client.1.smithi192.stdout:3/427: creat d2/d4/d12/d1a/d1d/d71/f85 x:0 0 0 2024-04-24T22:45:30.085 INFO:tasks.workunit.client.0.smithi063.stdout:9/561: creat d6/d8/d8a/fb8 x:0 0 0 2024-04-24T22:45:30.087 INFO:tasks.workunit.client.1.smithi192.stdout:8/415: write f6 [3941349,12851] 0 2024-04-24T22:45:30.087 INFO:tasks.workunit.client.0.smithi063.stdout:0/476: write d6/da/f10 [171027,62126] 0 2024-04-24T22:45:30.088 INFO:tasks.workunit.client.0.smithi063.stdout:7/488: dread d0/d4/de/d17/d28/d31/f6a [0,4194304] 0 2024-04-24T22:45:30.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/502: mkdir d2/d8/d36/da3 0 2024-04-24T22:45:30.091 INFO:tasks.workunit.client.1.smithi192.stdout:3/428: mkdir d2/d4/d12/d39/d86 0 2024-04-24T22:45:30.092 INFO:tasks.workunit.client.0.smithi063.stdout:1/628: write d3/da/da2/dba/d11/d5e/d6f/f90 [5889561,115932] 0 2024-04-24T22:45:30.094 INFO:tasks.workunit.client.1.smithi192.stdout:8/416: mkdir db/d43/d7c 0 2024-04-24T22:45:30.095 INFO:tasks.workunit.client.0.smithi063.stdout:2/623: mknod d0/dd/d62/d7c/cb5 0 2024-04-24T22:45:30.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/417: fdatasync db/dc/f44 0 2024-04-24T22:45:30.100 INFO:tasks.workunit.client.1.smithi192.stdout:3/429: dwrite d2/d4/d12/d39/d3a/f78 [0,4194304] 0 2024-04-24T22:45:30.101 INFO:tasks.workunit.client.1.smithi192.stdout:3/430: chown d2/d4/d4e/l84 208310 1 2024-04-24T22:45:30.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/562: dwrite d6/d2f/d42/f6f [0,4194304] 0 2024-04-24T22:45:30.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/418: fdatasync db/f5a 0 2024-04-24T22:45:30.103 INFO:tasks.workunit.client.0.smithi063.stdout:9/563: readlink d6/d2f/l85 0 2024-04-24T22:45:30.106 INFO:tasks.workunit.client.0.smithi063.stdout:0/477: dwrite d6/d64/d21/f86 [0,4194304] 0 2024-04-24T22:45:30.106 INFO:tasks.workunit.client.0.smithi063.stdout:7/489: creat d0/d4/de/d6f/d2e/d55/faa x:0 0 0 2024-04-24T22:45:30.106 INFO:tasks.workunit.client.0.smithi063.stdout:9/564: read d6/d8/db/d6b/f46 [830753,32949] 0 2024-04-24T22:45:30.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/419: rmdir db/dc 39 2024-04-24T22:45:30.116 INFO:tasks.workunit.client.0.smithi063.stdout:2/624: creat d0/d2/d2d/fb6 x:0 0 0 2024-04-24T22:45:30.117 INFO:tasks.workunit.client.0.smithi063.stdout:9/565: dwrite d6/d2f/d42/d81/f95 [0,4194304] 0 2024-04-24T22:45:30.124 INFO:tasks.workunit.client.1.smithi192.stdout:3/431: creat d2/d4/d7/f87 x:0 0 0 2024-04-24T22:45:30.132 INFO:tasks.workunit.client.0.smithi063.stdout:0/478: mkdir d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:30.132 INFO:tasks.workunit.client.0.smithi063.stdout:0/479: fsync d6/da/f10 0 2024-04-24T22:45:30.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/503: mknod d2/d8/d3c/ca4 0 2024-04-24T22:45:30.135 INFO:tasks.workunit.client.0.smithi063.stdout:6/564: sync 2024-04-24T22:45:30.136 INFO:tasks.workunit.client.0.smithi063.stdout:6/565: fsync db/d1d/d20/d29/f2e 0 2024-04-24T22:45:30.137 INFO:tasks.workunit.client.1.smithi192.stdout:4/349: truncate d2/db/f4e 3915202 0 2024-04-24T22:45:30.138 INFO:tasks.workunit.client.0.smithi063.stdout:7/490: symlink d0/d4/de/lab 0 2024-04-24T22:45:30.140 INFO:tasks.workunit.client.0.smithi063.stdout:5/530: getdents d5/d24/d2c 0 2024-04-24T22:45:30.141 INFO:tasks.workunit.client.1.smithi192.stdout:4/350: rmdir d2/d9/d25/d2e/d31/d36 39 2024-04-24T22:45:30.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/381: sync 2024-04-24T22:45:30.142 INFO:tasks.workunit.client.1.smithi192.stdout:2/526: sync 2024-04-24T22:45:30.142 INFO:tasks.workunit.client.1.smithi192.stdout:9/503: sync 2024-04-24T22:45:30.142 INFO:tasks.workunit.client.1.smithi192.stdout:5/341: sync 2024-04-24T22:45:30.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/351: chown d2/d9/d25/d2e/d31/c58 21862923 1 2024-04-24T22:45:30.143 INFO:tasks.workunit.client.1.smithi192.stdout:1/382: chown d9/d6f/f77 3768199 1 2024-04-24T22:45:30.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/491: dwrite d0/d4/de/d6f/f5b [0,4194304] 0 2024-04-24T22:45:30.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/566: truncate da/d18/d9e/da3/fab 4044668 0 2024-04-24T22:45:30.147 INFO:tasks.workunit.client.1.smithi192.stdout:9/504: read d3/db0/f2e [4034360,102876] 0 2024-04-24T22:45:30.148 INFO:tasks.workunit.client.1.smithi192.stdout:9/505: truncate d3/d5d/da1/f79 880485 0 2024-04-24T22:45:30.154 INFO:tasks.workunit.client.1.smithi192.stdout:1/383: fdatasync d9/fa 0 2024-04-24T22:45:30.157 INFO:tasks.workunit.client.1.smithi192.stdout:2/527: readlink d4/d9/d10/l4f 0 2024-04-24T22:45:30.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/528: chown d4/d9/d10/d6c/d95 47320 1 2024-04-24T22:45:30.159 INFO:tasks.workunit.client.0.smithi063.stdout:9/566: dread d6/fd [0,4194304] 0 2024-04-24T22:45:30.159 INFO:tasks.workunit.client.1.smithi192.stdout:1/384: dread d9/d15/d2d/d31/d2f/f3f [0,4194304] 0 2024-04-24T22:45:30.160 INFO:tasks.workunit.client.0.smithi063.stdout:8/552: sync 2024-04-24T22:45:30.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/385: readlink d9/d15/d2d/d31/d2f/l61 0 2024-04-24T22:45:30.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/386: chown d9/d51/d70 3 1 2024-04-24T22:45:30.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/480: rename d6/d64/d21/d53 to d6/d64/d21/d25/d62/d63/d89/d95/da4 0 2024-04-24T22:45:30.163 INFO:tasks.workunit.client.0.smithi063.stdout:0/481: chown d6/da/dc/f13 25868611 1 2024-04-24T22:45:30.165 INFO:tasks.workunit.client.0.smithi063.stdout:3/567: readlink da/lf 0 2024-04-24T22:45:30.166 INFO:tasks.workunit.client.1.smithi192.stdout:8/420: write db/dc/d22/d31/f4d [279378,71487] 0 2024-04-24T22:45:30.166 INFO:tasks.workunit.client.0.smithi063.stdout:3/568: chown da/d39/f7a 45 1 2024-04-24T22:45:30.168 INFO:tasks.workunit.client.0.smithi063.stdout:1/629: dwrite d3/db5/f77 [0,4194304] 0 2024-04-24T22:45:30.169 INFO:tasks.workunit.client.0.smithi063.stdout:3/569: truncate da/d48/f95 391324 0 2024-04-24T22:45:30.170 INFO:tasks.workunit.client.0.smithi063.stdout:3/570: readlink da/d39/l9d 0 2024-04-24T22:45:30.170 INFO:tasks.workunit.client.0.smithi063.stdout:5/531: link d5/d11/d76/d92/f9d d5/d1d/fba 0 2024-04-24T22:45:30.177 INFO:tasks.workunit.client.1.smithi192.stdout:2/529: rename d4/d9/d10/c50 to d4/d9/d9f/ca1 0 2024-04-24T22:45:30.179 INFO:tasks.workunit.client.1.smithi192.stdout:1/387: symlink d9/d15/d5e/l80 0 2024-04-24T22:45:30.181 INFO:tasks.workunit.client.0.smithi063.stdout:2/625: dwrite d0/d2/d2d/d31/d89/d7/f13 [0,4194304] 0 2024-04-24T22:45:30.183 INFO:tasks.workunit.client.0.smithi063.stdout:8/553: creat d1/d11/d32/fb2 x:0 0 0 2024-04-24T22:45:30.185 INFO:tasks.workunit.client.0.smithi063.stdout:6/566: dwrite db/d1d/d2d/f42 [0,4194304] 0 2024-04-24T22:45:30.186 INFO:tasks.workunit.client.0.smithi063.stdout:4/504: write d2/d8/d36/d3a/f9a [1489524,123815] 0 2024-04-24T22:45:30.186 INFO:tasks.workunit.client.1.smithi192.stdout:4/352: symlink d2/d9/d25/d2e/d31/d36/d5b/l7b 0 2024-04-24T22:45:30.189 INFO:tasks.workunit.client.1.smithi192.stdout:5/342: getdents d0/d13 0 2024-04-24T22:45:30.192 INFO:tasks.workunit.client.0.smithi063.stdout:1/630: creat d3/db5/d21/d9f/daa/db7/fce x:0 0 0 2024-04-24T22:45:30.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/571: rmdir da/d18/d3a/da1 39 2024-04-24T22:45:30.194 INFO:tasks.workunit.client.1.smithi192.stdout:8/421: mknod db/c7d 0 2024-04-24T22:45:30.196 INFO:tasks.workunit.client.0.smithi063.stdout:0/482: dwrite d6/d64/d24/f6b [0,4194304] 0 2024-04-24T22:45:30.197 INFO:tasks.workunit.client.0.smithi063.stdout:7/492: dwrite d0/d4/de/d17/d35/f54 [0,4194304] 0 2024-04-24T22:45:30.198 INFO:tasks.workunit.client.0.smithi063.stdout:0/483: dread - d6/d64/f92 zero size 2024-04-24T22:45:30.198 INFO:tasks.workunit.client.0.smithi063.stdout:5/532: rename d5/d24/d2c to d5/d24/d44/d71/dbb 0 2024-04-24T22:45:30.200 INFO:tasks.workunit.client.1.smithi192.stdout:4/353: mknod d2/d9/d25/d2e/d31/d36/d5b/d28/c7c 0 2024-04-24T22:45:30.200 INFO:tasks.workunit.client.1.smithi192.stdout:4/354: chown d2/db/l4d 243724 1 2024-04-24T22:45:30.201 INFO:tasks.workunit.client.0.smithi063.stdout:9/567: mkdir d6/d8/d1f/d20/d2a/d4b/d5e/db9 0 2024-04-24T22:45:30.202 INFO:tasks.workunit.client.1.smithi192.stdout:5/343: creat d0/d13/d19/d64/d5f/d2c/f6f x:0 0 0 2024-04-24T22:45:30.203 INFO:tasks.workunit.client.1.smithi192.stdout:5/344: chown d0/d13/d19/d64/d5f/f27 2712 1 2024-04-24T22:45:30.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/568: write d6/d8/d1f/d20/d22/d31/f33 [5217762,96877] 0 2024-04-24T22:45:30.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/569: fsync d6/d8/f5b 0 2024-04-24T22:45:30.208 INFO:tasks.workunit.client.0.smithi063.stdout:2/626: read d0/d2/d2d/d31/d89/d1c/d3f/f42 [1267447,64859] 0 2024-04-24T22:45:30.212 INFO:tasks.workunit.client.1.smithi192.stdout:9/506: link d3/dd/de/d17/d89/ca3 d3/dd/de/cb5 0 2024-04-24T22:45:30.213 INFO:tasks.workunit.client.1.smithi192.stdout:9/507: fsync d3/dd/de/d17/f68 0 2024-04-24T22:45:30.216 INFO:tasks.workunit.client.1.smithi192.stdout:5/345: creat d0/d13/d53/f70 x:0 0 0 2024-04-24T22:45:30.217 INFO:tasks.workunit.client.0.smithi063.stdout:3/572: creat da/d18/d9e/fc4 x:0 0 0 2024-04-24T22:45:30.218 INFO:tasks.workunit.client.0.smithi063.stdout:1/631: mknod d3/da/da2/dba/d11/d5e/d6f/ccf 0 2024-04-24T22:45:30.221 INFO:tasks.workunit.client.1.smithi192.stdout:1/388: rename d9/d15/d2d/d31/d2f/c30 to d9/d15/d2d/d31/c81 0 2024-04-24T22:45:30.223 INFO:tasks.workunit.client.0.smithi063.stdout:5/533: creat d5/d11/d76/fbc x:0 0 0 2024-04-24T22:45:30.226 INFO:tasks.workunit.client.0.smithi063.stdout:8/554: rename d1/d11/d32/f65 to d1/d21/fb3 0 2024-04-24T22:45:30.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/355: mkdir d2/d9/d25/d2e/d31/d7d 0 2024-04-24T22:45:30.228 INFO:tasks.workunit.client.0.smithi063.stdout:0/484: creat d6/d64/da0/d33/d87/fa5 x:0 0 0 2024-04-24T22:45:30.229 INFO:tasks.workunit.client.1.smithi192.stdout:4/356: read d2/d9/d25/d2e/f66 [579284,96060] 0 2024-04-24T22:45:30.229 INFO:tasks.workunit.client.1.smithi192.stdout:9/508: fsync d3/dd/d62/d6d/f86 0 2024-04-24T22:45:30.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/422: dwrite db/dc/d22/f28 [0,4194304] 0 2024-04-24T22:45:30.240 INFO:tasks.workunit.client.1.smithi192.stdout:5/346: dread d0/d13/d19/d64/d5f/f33 [0,4194304] 0 2024-04-24T22:45:30.241 INFO:tasks.workunit.client.1.smithi192.stdout:5/347: stat d0/d1b/d50 0 2024-04-24T22:45:30.243 INFO:tasks.workunit.client.0.smithi063.stdout:4/505: getdents d2/d19/d82/d96 0 2024-04-24T22:45:30.243 INFO:tasks.workunit.client.1.smithi192.stdout:4/357: mkdir d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/d7e 0 2024-04-24T22:45:30.245 INFO:tasks.workunit.client.0.smithi063.stdout:2/627: creat d0/d2/d39/fb7 x:0 0 0 2024-04-24T22:45:30.249 INFO:tasks.workunit.client.1.smithi192.stdout:1/389: dwrite d9/fa [0,4194304] 0 2024-04-24T22:45:30.249 INFO:tasks.workunit.client.0.smithi063.stdout:6/567: truncate db/fb7 2474888 0 2024-04-24T22:45:30.252 INFO:tasks.workunit.client.0.smithi063.stdout:4/506: dwrite d2/d8/d36/d3a/d50/f77 [0,4194304] 0 2024-04-24T22:45:30.252 INFO:tasks.workunit.client.1.smithi192.stdout:5/348: unlink d0/d13/d19/d64/d5f/l1e 0 2024-04-24T22:45:30.253 INFO:tasks.workunit.client.0.smithi063.stdout:9/570: dwrite d6/d8/db/d6b/f8f [0,4194304] 0 2024-04-24T22:45:30.254 INFO:tasks.workunit.client.1.smithi192.stdout:5/349: fsync d0/d13/d19/f1a 0 2024-04-24T22:45:30.258 INFO:tasks.workunit.client.1.smithi192.stdout:9/509: creat d3/dd/d62/fb6 x:0 0 0 2024-04-24T22:45:30.259 INFO:tasks.workunit.client.1.smithi192.stdout:4/358: mknod d2/d9/d25/d2e/d31/d7d/c7f 0 2024-04-24T22:45:30.261 INFO:tasks.workunit.client.1.smithi192.stdout:5/350: mkdir d0/d13/d19/d71 0 2024-04-24T22:45:30.264 INFO:tasks.workunit.client.1.smithi192.stdout:8/423: rename f3 to db/d5d/f7e 0 2024-04-24T22:45:30.267 INFO:tasks.workunit.client.1.smithi192.stdout:9/510: creat d3/d5d/da1/d85/fb7 x:0 0 0 2024-04-24T22:45:30.268 INFO:tasks.workunit.client.1.smithi192.stdout:9/511: fsync d3/dd/d62/f92 0 2024-04-24T22:45:30.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/512: dread - d3/dd/de/f51 zero size 2024-04-24T22:45:30.271 INFO:tasks.workunit.client.1.smithi192.stdout:9/513: dread d3/d5d/da1/f79 [0,4194304] 0 2024-04-24T22:45:30.272 INFO:tasks.workunit.client.1.smithi192.stdout:8/424: symlink db/dc/d22/d31/d33/l7f 0 2024-04-24T22:45:30.272 INFO:tasks.workunit.client.1.smithi192.stdout:8/425: fsync db/dc/f24 0 2024-04-24T22:45:30.277 INFO:tasks.workunit.client.0.smithi063.stdout:0/485: rename d6/dd/l58 to d6/d64/d5d/la6 0 2024-04-24T22:45:30.280 INFO:tasks.workunit.client.1.smithi192.stdout:8/426: rmdir db/d43 39 2024-04-24T22:45:30.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/628: mkdir d0/d2/d6c/db8 0 2024-04-24T22:45:30.282 INFO:tasks.workunit.client.1.smithi192.stdout:5/351: creat d0/d13/d19/d64/d5f/f72 x:0 0 0 2024-04-24T22:45:30.283 INFO:tasks.workunit.client.0.smithi063.stdout:6/568: write db/d1d/d20/d84/fae [2496242,63275] 0 2024-04-24T22:45:30.287 INFO:tasks.workunit.client.0.smithi063.stdout:9/571: mkdir d6/d2f/d42/d81/dba 0 2024-04-24T22:45:30.288 INFO:tasks.workunit.client.1.smithi192.stdout:4/359: getdents d2/d9 0 2024-04-24T22:45:30.288 INFO:tasks.workunit.client.0.smithi063.stdout:3/573: truncate da/f36 717893 0 2024-04-24T22:45:30.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/390: write d9/d15/d2d/f6c [1210514,119782] 0 2024-04-24T22:45:30.290 INFO:tasks.workunit.client.1.smithi192.stdout:4/360: readlink d2/d9/d25/d2e/d31/d53/l78 0 2024-04-24T22:45:30.291 INFO:tasks.workunit.client.0.smithi063.stdout:8/555: mknod d1/d22/d33/da6/cb4 0 2024-04-24T22:45:30.292 INFO:tasks.workunit.client.1.smithi192.stdout:1/391: truncate d9/d15/d2d/d63/f79 768143 0 2024-04-24T22:45:30.293 INFO:tasks.workunit.client.1.smithi192.stdout:9/514: creat d3/dd/fb8 x:0 0 0 2024-04-24T22:45:30.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/363: sync 2024-04-24T22:45:30.293 INFO:tasks.workunit.client.1.smithi192.stdout:0/449: sync 2024-04-24T22:45:30.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/361: truncate d2/d9/d25/d2e/d31/d36/d5b/d64/f6c 101711 0 2024-04-24T22:45:30.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/362: readlink d2/db/l1b 0 2024-04-24T22:45:30.295 INFO:tasks.workunit.client.0.smithi063.stdout:0/486: creat d6/da/d2b/fa7 x:0 0 0 2024-04-24T22:45:30.296 INFO:tasks.workunit.client.0.smithi063.stdout:7/493: link d0/d4/de/d6f/d2e/l65 d0/d4/de/d6f/d5c/lac 0 2024-04-24T22:45:30.298 INFO:tasks.workunit.client.0.smithi063.stdout:6/569: mknod db/d1d/d2d/d48/cc0 0 2024-04-24T22:45:30.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/570: truncate db/d1d/d20/d3e/f87 542315 0 2024-04-24T22:45:30.301 INFO:tasks.workunit.client.1.smithi192.stdout:0/450: dread - d3/d27/d28/d2e/d4a/f6b zero size 2024-04-24T22:45:30.304 INFO:tasks.workunit.client.1.smithi192.stdout:6/364: mkdir d0/d8/d29/d4c/d4d/d83 0 2024-04-24T22:45:30.304 INFO:tasks.workunit.client.1.smithi192.stdout:6/365: chown d0/d8/d29/d4c 786 1 2024-04-24T22:45:30.305 INFO:tasks.workunit.client.1.smithi192.stdout:6/366: chown d0/d8/f76 12 1 2024-04-24T22:45:30.305 INFO:tasks.workunit.client.0.smithi063.stdout:3/574: creat da/d18/d47/d7b/fc5 x:0 0 0 2024-04-24T22:45:30.308 INFO:tasks.workunit.client.0.smithi063.stdout:5/534: write d5/f1c [161351,59106] 0 2024-04-24T22:45:30.308 INFO:tasks.workunit.client.1.smithi192.stdout:4/363: dread d2/db/f52 [0,4194304] 0 2024-04-24T22:45:30.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/352: creat d0/f73 x:0 0 0 2024-04-24T22:45:30.331 INFO:tasks.workunit.client.0.smithi063.stdout:1/632: sync 2024-04-24T22:45:30.331 INFO:tasks.workunit.client.0.smithi063.stdout:7/494: mkdir d0/d4/de/d17/d28/d31/dad 0 2024-04-24T22:45:30.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/495: write d0/d4/de/d17/d35/d6c/d7a/fa5 [721716,56832] 0 2024-04-24T22:45:30.338 INFO:tasks.workunit.client.1.smithi192.stdout:8/427: getdents db/dc/d22/d31/d33 0 2024-04-24T22:45:30.339 INFO:tasks.workunit.client.1.smithi192.stdout:8/428: dread - db/f5a zero size 2024-04-24T22:45:30.339 INFO:tasks.workunit.client.0.smithi063.stdout:8/556: symlink d1/d22/lb5 0 2024-04-24T22:45:30.340 INFO:tasks.workunit.client.0.smithi063.stdout:7/496: dwrite d0/d2f/f5a [4194304,4194304] 0 2024-04-24T22:45:30.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/364: rmdir d2/d9/d25/d2e/d31/d36/d5b/d28 39 2024-04-24T22:45:30.341 INFO:tasks.workunit.client.1.smithi192.stdout:4/365: chown d2/db/d10/c1d 81 1 2024-04-24T22:45:30.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/487: mknod d6/da/ca8 0 2024-04-24T22:45:30.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/633: rmdir d3/da/da2/dba/d11/dc6 39 2024-04-24T22:45:30.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/634: chown d3/db5/d2b 4 1 2024-04-24T22:45:30.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/451: write d3/f10 [4897211,46842] 0 2024-04-24T22:45:30.348 INFO:tasks.workunit.client.1.smithi192.stdout:6/367: write d0/f82 [3579585,47221] 0 2024-04-24T22:45:30.350 INFO:tasks.workunit.client.0.smithi063.stdout:4/507: dwrite d2/f97 [0,4194304] 0 2024-04-24T22:45:30.351 INFO:tasks.workunit.client.0.smithi063.stdout:2/629: sync 2024-04-24T22:45:30.353 INFO:tasks.workunit.client.1.smithi192.stdout:5/353: symlink d0/d13/d19/d71/l74 0 2024-04-24T22:45:30.353 INFO:tasks.workunit.client.1.smithi192.stdout:5/354: chown d0/d5b 12 1 2024-04-24T22:45:30.353 INFO:tasks.workunit.client.0.smithi063.stdout:2/630: fsync d0/d2/d27/f37 0 2024-04-24T22:45:30.354 INFO:tasks.workunit.client.1.smithi192.stdout:9/515: getdents d3/d5d/da1 0 2024-04-24T22:45:30.355 INFO:tasks.workunit.client.1.smithi192.stdout:9/516: chown d3/dd/de/d17/l72 1748514 1 2024-04-24T22:45:30.356 INFO:tasks.workunit.client.0.smithi063.stdout:8/557: write d1/d11/d3c/f55 [3515154,123197] 0 2024-04-24T22:45:30.358 INFO:tasks.workunit.client.0.smithi063.stdout:3/575: dwrite da/d40/f81 [0,4194304] 0 2024-04-24T22:45:30.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/429: creat db/dc/f80 x:0 0 0 2024-04-24T22:45:30.361 INFO:tasks.workunit.client.0.smithi063.stdout:0/488: mknod d6/dd/ca9 0 2024-04-24T22:45:30.362 INFO:tasks.workunit.client.0.smithi063.stdout:6/571: creat db/d1d/d2d/fc1 x:0 0 0 2024-04-24T22:45:30.363 INFO:tasks.workunit.client.0.smithi063.stdout:1/635: creat d3/da/da2/dba/d11/d5e/d9c/dcb/fd0 x:0 0 0 2024-04-24T22:45:30.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/572: getdents d6 0 2024-04-24T22:45:30.367 INFO:tasks.workunit.client.1.smithi192.stdout:5/355: creat d0/d5b/f75 x:0 0 0 2024-04-24T22:45:30.367 INFO:tasks.workunit.client.0.smithi063.stdout:8/558: write d1/d2/fe [1834756,51857] 0 2024-04-24T22:45:30.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/535: dwrite d5/d24/d49/d3b/f45 [0,4194304] 0 2024-04-24T22:45:30.367 INFO:tasks.workunit.client.0.smithi063.stdout:1/636: readlink d3/da/da2/dba/d39/la7 0 2024-04-24T22:45:30.369 INFO:tasks.workunit.client.1.smithi192.stdout:9/517: mkdir d3/dd/de/d17/d89/db4/db9 0 2024-04-24T22:45:30.369 INFO:tasks.workunit.client.1.smithi192.stdout:8/430: dwrite db/f5a [0,4194304] 0 2024-04-24T22:45:30.370 INFO:tasks.workunit.client.0.smithi063.stdout:9/573: fsync d6/d8/d53/f99 0 2024-04-24T22:45:30.376 INFO:tasks.workunit.client.0.smithi063.stdout:2/631: dread d0/dd/d72/d77/d87/f98 [0,4194304] 0 2024-04-24T22:45:30.377 INFO:tasks.workunit.client.0.smithi063.stdout:2/632: write d0/dd/f22 [1967752,15125] 0 2024-04-24T22:45:30.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/497: write d0/d4/de/f1a [3091101,105137] 0 2024-04-24T22:45:30.379 INFO:tasks.workunit.client.0.smithi063.stdout:6/572: write db/f2a [5867695,9091] 0 2024-04-24T22:45:30.380 INFO:tasks.workunit.client.1.smithi192.stdout:5/356: symlink d0/d13/d19/d64/l76 0 2024-04-24T22:45:30.381 INFO:tasks.workunit.client.0.smithi063.stdout:2/633: write d0/d2/d2d/d31/d89/d7/f13 [702801,95476] 0 2024-04-24T22:45:30.381 INFO:tasks.workunit.client.0.smithi063.stdout:5/536: unlink d5/d11/c60 0 2024-04-24T22:45:30.382 INFO:tasks.workunit.client.1.smithi192.stdout:0/452: getdents d3/d27/d28/d2e/d4a/d5c/d80/d83 0 2024-04-24T22:45:30.384 INFO:tasks.workunit.client.0.smithi063.stdout:0/489: dread d6/d64/d21/d25/d62/d75/f61 [0,4194304] 0 2024-04-24T22:45:30.387 INFO:tasks.workunit.client.0.smithi063.stdout:8/559: mkdir d1/d2/d3/db6 0 2024-04-24T22:45:30.388 INFO:tasks.workunit.client.1.smithi192.stdout:4/366: dwrite d2/d9/d25/f68 [0,4194304] 0 2024-04-24T22:45:30.388 INFO:tasks.workunit.client.0.smithi063.stdout:8/560: write d1/d11/f9f [12713,35215] 0 2024-04-24T22:45:30.389 INFO:tasks.workunit.client.1.smithi192.stdout:5/357: unlink d0/d13/d19/d64/d5f/f3e 0 2024-04-24T22:45:30.391 INFO:tasks.workunit.client.0.smithi063.stdout:1/637: chown d3/f23 3 1 2024-04-24T22:45:30.392 INFO:tasks.workunit.client.0.smithi063.stdout:0/490: dwrite d6/d64/da0/d33/d87/fa5 [0,4194304] 0 2024-04-24T22:45:30.394 INFO:tasks.workunit.client.0.smithi063.stdout:1/638: stat d3/da/da2/dba/d39/d5a/f8f 0 2024-04-24T22:45:30.395 INFO:tasks.workunit.client.1.smithi192.stdout:0/453: truncate d3/d27/d6a/f87 100159 0 2024-04-24T22:45:30.395 INFO:tasks.workunit.client.0.smithi063.stdout:3/576: rename da/d18/d47/c5e to da/d18/d3a/da1/cc6 0 2024-04-24T22:45:30.404 INFO:tasks.workunit.client.0.smithi063.stdout:6/573: mkdir db/d1c/dc2 0 2024-04-24T22:45:30.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/431: dwrite db/dc/d22/d31/d33/f49 [0,4194304] 0 2024-04-24T22:45:30.411 INFO:tasks.workunit.client.1.smithi192.stdout:5/358: creat d0/d13/d19/d64/d5f/d2c/d3c/f77 x:0 0 0 2024-04-24T22:45:30.412 INFO:tasks.workunit.client.1.smithi192.stdout:0/454: creat d3/d40/f9b x:0 0 0 2024-04-24T22:45:30.413 INFO:tasks.workunit.client.1.smithi192.stdout:5/359: write d0/d5b/f75 [167664,120595] 0 2024-04-24T22:45:30.414 INFO:tasks.workunit.client.1.smithi192.stdout:5/360: chown d0/d13/d19/d64 70 1 2024-04-24T22:45:30.414 INFO:tasks.workunit.client.0.smithi063.stdout:5/537: dread d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:30.415 INFO:tasks.workunit.client.1.smithi192.stdout:4/367: fdatasync d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f3c 0 2024-04-24T22:45:30.416 INFO:tasks.workunit.client.1.smithi192.stdout:4/368: stat d2/d9/d25/d2e/d31/d7d 0 2024-04-24T22:45:30.418 INFO:tasks.workunit.client.1.smithi192.stdout:9/518: dread d3/d24/f41 [0,4194304] 0 2024-04-24T22:45:30.418 INFO:tasks.workunit.client.0.smithi063.stdout:2/634: creat d0/dd/d62/d7c/fb9 x:0 0 0 2024-04-24T22:45:30.419 INFO:tasks.workunit.client.1.smithi192.stdout:9/519: fdatasync d3/f58 0 2024-04-24T22:45:30.420 INFO:tasks.workunit.client.1.smithi192.stdout:8/432: mkdir db/dc/d22/d31/d33/d81 0 2024-04-24T22:45:30.422 INFO:tasks.workunit.client.1.smithi192.stdout:8/433: write db/dc/d22/d31/d33/f49 [1751865,32095] 0 2024-04-24T22:45:30.422 INFO:tasks.workunit.client.0.smithi063.stdout:9/574: write d6/d8/db/d6b/d4f/f5a [11130,32835] 0 2024-04-24T22:45:30.423 INFO:tasks.workunit.client.1.smithi192.stdout:0/455: mknod d3/d40/c9c 0 2024-04-24T22:45:30.424 INFO:tasks.workunit.client.0.smithi063.stdout:8/561: truncate d1/f1f 3750725 0 2024-04-24T22:45:30.425 INFO:tasks.workunit.client.0.smithi063.stdout:8/562: stat d1/d11/d32/d6a/l88 0 2024-04-24T22:45:30.428 INFO:tasks.workunit.client.0.smithi063.stdout:7/498: write d0/fa2 [2594662,14002] 0 2024-04-24T22:45:30.429 INFO:tasks.workunit.client.0.smithi063.stdout:4/508: sync 2024-04-24T22:45:30.431 INFO:tasks.workunit.client.0.smithi063.stdout:9/575: dwrite d6/d8/f17 [0,4194304] 0 2024-04-24T22:45:30.431 INFO:tasks.workunit.client.1.smithi192.stdout:0/456: dwrite d3/d27/d28/d2e/d4a/d5c/f3d [0,4194304] 0 2024-04-24T22:45:30.438 INFO:tasks.workunit.client.0.smithi063.stdout:6/574: mkdir db/d1c/dc3 0 2024-04-24T22:45:30.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/575: readlink db/d1d/d2d/l92 0 2024-04-24T22:45:30.441 INFO:tasks.workunit.client.1.smithi192.stdout:4/369: mknod d2/db/d75/c80 0 2024-04-24T22:45:30.445 INFO:tasks.workunit.client.1.smithi192.stdout:0/457: mknod d3/d27/d28/d2e/d3f/c9d 0 2024-04-24T22:45:30.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/458: readlink d3/d27/l90 0 2024-04-24T22:45:30.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/635: dwrite d0/d2/d2d/d31/fab [0,4194304] 0 2024-04-24T22:45:30.450 INFO:tasks.workunit.client.1.smithi192.stdout:9/520: rename d3/db0 to d3/dd/de/dba 0 2024-04-24T22:45:30.451 INFO:tasks.workunit.client.1.smithi192.stdout:9/521: truncate d3/dd/d62/fb6 100651 0 2024-04-24T22:45:30.452 INFO:tasks.workunit.client.1.smithi192.stdout:4/370: symlink d2/d9/d25/d2e/d31/d36/d5b/d64/l81 0 2024-04-24T22:45:30.453 INFO:tasks.workunit.client.1.smithi192.stdout:9/522: truncate d3/dd/d53/f7f 1523000 0 2024-04-24T22:45:30.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/499: creat d0/d4/de/d6f/d5c/d8b/fae x:0 0 0 2024-04-24T22:45:30.454 INFO:tasks.workunit.client.1.smithi192.stdout:0/459: mknod d3/d27/d28/d2e/d4a/d5c/d80/c9e 0 2024-04-24T22:45:30.457 INFO:tasks.workunit.client.1.smithi192.stdout:0/460: truncate d3/d27/d3c/d3b/f8d 620309 0 2024-04-24T22:45:30.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/538: creat d5/d24/db6/fbd x:0 0 0 2024-04-24T22:45:30.462 INFO:tasks.workunit.client.1.smithi192.stdout:0/461: read - d3/d27/d28/d31/f3a zero size 2024-04-24T22:45:30.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/523: mknod d3/dd/d62/cbb 0 2024-04-24T22:45:30.467 INFO:tasks.workunit.client.1.smithi192.stdout:9/524: fdatasync d3/dd/d62/f92 0 2024-04-24T22:45:30.467 INFO:tasks.workunit.client.1.smithi192.stdout:5/361: truncate d0/d13/d3b/f3a 529221 0 2024-04-24T22:45:30.468 INFO:tasks.workunit.client.0.smithi063.stdout:2/636: mkdir d0/d2/d2f/dba 0 2024-04-24T22:45:30.468 INFO:tasks.workunit.client.0.smithi063.stdout:3/577: write da/f2c [452101,63141] 0 2024-04-24T22:45:30.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/578: dread da/d48/f95 [0,4194304] 0 2024-04-24T22:45:30.470 INFO:tasks.workunit.client.1.smithi192.stdout:8/434: rename db/d5d/c64 to db/dc/d22/d31/c82 0 2024-04-24T22:45:30.471 INFO:tasks.workunit.client.0.smithi063.stdout:0/491: link d6/da/lf d6/d64/d21/d25/d62/d63/d89/d95/laa 0 2024-04-24T22:45:30.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/509: mkdir d2/d8/d36/da3/da5 0 2024-04-24T22:45:30.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/576: mknod d6/d8/cbb 0 2024-04-24T22:45:30.482 INFO:tasks.workunit.client.0.smithi063.stdout:4/510: dread d2/f74 [0,4194304] 0 2024-04-24T22:45:30.485 INFO:tasks.workunit.client.0.smithi063.stdout:6/576: dwrite db/d1d/d20/d3e/f43 [0,4194304] 0 2024-04-24T22:45:30.486 INFO:tasks.workunit.client.1.smithi192.stdout:4/371: dwrite d2/db/d10/f11 [4194304,4194304] 0 2024-04-24T22:45:30.487 INFO:tasks.workunit.client.0.smithi063.stdout:1/639: getdents d3/db5 0 2024-04-24T22:45:30.488 INFO:tasks.workunit.client.1.smithi192.stdout:5/362: dread d0/d13/d19/f60 [0,4194304] 0 2024-04-24T22:45:30.489 INFO:tasks.workunit.client.0.smithi063.stdout:6/577: truncate db/d1d/d20/d29/d5d/f99 907906 0 2024-04-24T22:45:30.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/539: dread d5/d1d/faa [0,4194304] 0 2024-04-24T22:45:30.490 INFO:tasks.workunit.client.1.smithi192.stdout:5/363: write d0/d13/f43 [847746,67008] 0 2024-04-24T22:45:30.493 INFO:tasks.workunit.client.1.smithi192.stdout:0/462: dread d3/d27/d3c/d64/f84 [0,4194304] 0 2024-04-24T22:45:30.497 INFO:tasks.workunit.client.0.smithi063.stdout:7/500: dwrite d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:30.503 INFO:tasks.workunit.client.1.smithi192.stdout:9/525: dread d3/dd/f38 [0,4194304] 0 2024-04-24T22:45:30.506 INFO:tasks.workunit.client.1.smithi192.stdout:8/435: link db/dc/f44 db/d43/d71/f83 0 2024-04-24T22:45:30.506 INFO:tasks.workunit.client.1.smithi192.stdout:0/463: symlink d3/d27/d28/d2e/d3f/l9f 0 2024-04-24T22:45:30.508 INFO:tasks.workunit.client.1.smithi192.stdout:9/526: mkdir d3/dd/de/dba/dbc 0 2024-04-24T22:45:30.509 INFO:tasks.workunit.client.0.smithi063.stdout:4/511: write d2/d8/d36/d3a/f7f [1308688,65173] 0 2024-04-24T22:45:30.509 INFO:tasks.workunit.client.1.smithi192.stdout:0/464: mknod d3/d27/d6a/ca0 0 2024-04-24T22:45:30.511 INFO:tasks.workunit.client.1.smithi192.stdout:8/436: creat db/d5d/f84 x:0 0 0 2024-04-24T22:45:30.512 INFO:tasks.workunit.client.0.smithi063.stdout:1/640: rmdir d3/da/da2/dba/d11/d5e/d9c 39 2024-04-24T22:45:30.513 INFO:tasks.workunit.client.1.smithi192.stdout:7/518: sync 2024-04-24T22:45:30.514 INFO:tasks.workunit.client.0.smithi063.stdout:5/540: truncate d5/f87 999093 0 2024-04-24T22:45:30.516 INFO:tasks.workunit.client.1.smithi192.stdout:5/364: read d0/d13/f43 [353770,29423] 0 2024-04-24T22:45:30.517 INFO:tasks.workunit.client.0.smithi063.stdout:7/501: symlink d0/d4/de/d17/d35/d6c/d7a/laf 0 2024-04-24T22:45:30.518 INFO:tasks.workunit.client.1.smithi192.stdout:5/365: dread - d0/d13/d19/d64/d4f/d52/f6a zero size 2024-04-24T22:45:30.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/563: chown d1/d2/d3/f46 0 1 2024-04-24T22:45:30.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/366: write d0/d13/d53/f70 [484891,123157] 0 2024-04-24T22:45:30.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/367: stat d0/c3 0 2024-04-24T22:45:30.521 INFO:tasks.workunit.client.1.smithi192.stdout:8/437: mkdir db/d43/d7c/d85 0 2024-04-24T22:45:30.521 INFO:tasks.workunit.client.1.smithi192.stdout:8/438: stat db/dc/f1c 0 2024-04-24T22:45:30.521 INFO:tasks.workunit.client.1.smithi192.stdout:4/372: write d2/f17 [197335,31822] 0 2024-04-24T22:45:30.522 INFO:tasks.workunit.client.1.smithi192.stdout:8/439: write db/dc/d22/f23 [5042817,38028] 0 2024-04-24T22:45:30.526 INFO:tasks.workunit.client.1.smithi192.stdout:9/527: creat d3/fbd x:0 0 0 2024-04-24T22:45:30.526 INFO:tasks.workunit.client.1.smithi192.stdout:4/373: write d2/f17 [1394093,14228] 0 2024-04-24T22:45:30.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/519: rmdir d2/db/d21/d42/d5b/d63 39 2024-04-24T22:45:30.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/368: rename d0/d13/d19/c36 to d0/d1b/d50/c78 0 2024-04-24T22:45:30.535 INFO:tasks.workunit.client.0.smithi063.stdout:4/512: unlink d2/d8/d40/c6a 0 2024-04-24T22:45:30.536 INFO:tasks.workunit.client.0.smithi063.stdout:1/641: symlink d3/db5/d21/d9f/daa/db7/ld1 0 2024-04-24T22:45:30.537 INFO:tasks.workunit.client.1.smithi192.stdout:8/440: creat db/dc/d29/f86 x:0 0 0 2024-04-24T22:45:30.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/541: mkdir d5/d1d/d7d/d67/dbe 0 2024-04-24T22:45:30.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/542: dread - d5/d24/d44/fa5 zero size 2024-04-24T22:45:30.540 INFO:tasks.workunit.client.0.smithi063.stdout:5/543: write d5/d1d/f64 [5088988,53625] 0 2024-04-24T22:45:30.542 INFO:tasks.workunit.client.1.smithi192.stdout:0/465: dwrite d3/d27/d28/d2e/d4a/d5c/f51 [0,4194304] 0 2024-04-24T22:45:30.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/637: creat d0/d2/d2d/d31/d89/fbb x:0 0 0 2024-04-24T22:45:30.546 INFO:tasks.workunit.client.0.smithi063.stdout:0/492: truncate d6/d64/da0/f5f 249471 0 2024-04-24T22:45:30.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/520: chown d2/db/d21/d42/d5b/d63/l8f 6353 1 2024-04-24T22:45:30.548 INFO:tasks.workunit.client.1.smithi192.stdout:9/528: dwrite d3/f21 [0,4194304] 0 2024-04-24T22:45:30.552 INFO:tasks.workunit.client.0.smithi063.stdout:3/579: creat da/d18/d3a/fc7 x:0 0 0 2024-04-24T22:45:30.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/580: fsync da/d18/d3a/d63/f64 0 2024-04-24T22:45:30.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/578: sync 2024-04-24T22:45:30.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/577: creat d6/d8/d1f/d20/fbc x:0 0 0 2024-04-24T22:45:30.565 INFO:tasks.workunit.client.1.smithi192.stdout:4/374: mkdir d2/db/d5d/d77/d82 0 2024-04-24T22:45:30.567 INFO:tasks.workunit.client.0.smithi063.stdout:8/564: write d1/f58 [2848340,117693] 0 2024-04-24T22:45:30.569 INFO:tasks.workunit.client.1.smithi192.stdout:7/521: unlink d2/db/d2a/c7b 0 2024-04-24T22:45:30.570 INFO:tasks.workunit.client.1.smithi192.stdout:7/522: chown d2/db/d21/d42/d5b/d63/f6d 24946 1 2024-04-24T22:45:30.571 INFO:tasks.workunit.client.0.smithi063.stdout:5/544: creat d5/d24/d44/fbf x:0 0 0 2024-04-24T22:45:30.572 INFO:tasks.workunit.client.1.smithi192.stdout:5/369: mkdir d0/d13/d3b/d79 0 2024-04-24T22:45:30.574 INFO:tasks.workunit.client.0.smithi063.stdout:7/502: unlink d0/d4/de/d6f/f45 0 2024-04-24T22:45:30.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/441: mknod db/dc/d22/d31/c87 0 2024-04-24T22:45:30.576 INFO:tasks.workunit.client.0.smithi063.stdout:0/493: creat d6/d74/fab x:0 0 0 2024-04-24T22:45:30.581 INFO:tasks.workunit.client.0.smithi063.stdout:3/581: rename da/d40/f78 to da/d39/d94/db3/fc8 0 2024-04-24T22:45:30.582 INFO:tasks.workunit.client.1.smithi192.stdout:7/523: rename d2/fa to d2/db/d2a/d57/fa3 0 2024-04-24T22:45:30.586 INFO:tasks.workunit.client.0.smithi063.stdout:1/642: rmdir d3/da/da2/dba/d11/d5e/d9c 39 2024-04-24T22:45:30.587 INFO:tasks.workunit.client.1.smithi192.stdout:5/370: dread d0/d13/d19/d64/d4f/d52/f67 [0,4194304] 0 2024-04-24T22:45:30.588 INFO:tasks.workunit.client.1.smithi192.stdout:5/371: chown d0/d13/d19/f4b 579826 1 2024-04-24T22:45:30.588 INFO:tasks.workunit.client.0.smithi063.stdout:8/565: mkdir d1/d11/d32/d38/db7 0 2024-04-24T22:45:30.589 INFO:tasks.workunit.client.0.smithi063.stdout:8/566: chown d1/d2/d3/f46 3489999 1 2024-04-24T22:45:30.591 INFO:tasks.workunit.client.0.smithi063.stdout:5/545: read d5/d11/d23/f79 [583975,76339] 0 2024-04-24T22:45:30.592 INFO:tasks.workunit.client.0.smithi063.stdout:5/546: chown d5/d11/d23/d2d/c31 707766 1 2024-04-24T22:45:30.592 INFO:tasks.workunit.client.1.smithi192.stdout:0/466: link d3/d40/f9b d3/d27/d28/d2e/d4a/fa1 0 2024-04-24T22:45:30.593 INFO:tasks.workunit.client.1.smithi192.stdout:0/467: chown d3/d27/d28/c95 41561911 1 2024-04-24T22:45:30.594 INFO:tasks.workunit.client.1.smithi192.stdout:9/529: creat d3/dd/fbe x:0 0 0 2024-04-24T22:45:30.598 INFO:tasks.workunit.client.0.smithi063.stdout:0/494: symlink d6/d64/d24/lac 0 2024-04-24T22:45:30.599 INFO:tasks.workunit.client.0.smithi063.stdout:0/495: fsync d6/d64/d21/d25/d62/d63/d89/d95/da4/f8f 0 2024-04-24T22:45:30.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/432: sync 2024-04-24T22:45:30.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/433: stat d2/d4/d12/d1a 0 2024-04-24T22:45:30.600 INFO:tasks.workunit.client.0.smithi063.stdout:6/579: mkdir db/d85/db5/dc4 0 2024-04-24T22:45:30.604 INFO:tasks.workunit.client.1.smithi192.stdout:5/372: unlink d0/d13/d19/d64/d5f/l41 0 2024-04-24T22:45:30.604 INFO:tasks.workunit.client.1.smithi192.stdout:5/373: readlink d0/d1b/l26 0 2024-04-24T22:45:30.606 INFO:tasks.workunit.client.1.smithi192.stdout:4/375: rmdir d2/d9/d6b/d79 0 2024-04-24T22:45:30.606 INFO:tasks.workunit.client.0.smithi063.stdout:4/513: link d2/d8/d36/d3a/d50/f5f d2/d19/fa6 0 2024-04-24T22:45:30.607 INFO:tasks.workunit.client.0.smithi063.stdout:4/514: write d2/f97 [2321113,118623] 0 2024-04-24T22:45:30.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/567: unlink d1/c4d 0 2024-04-24T22:45:30.612 INFO:tasks.workunit.client.1.smithi192.stdout:7/524: creat d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 x:0 0 0 2024-04-24T22:45:30.613 INFO:tasks.workunit.client.0.smithi063.stdout:9/578: truncate d6/d8/db/d1c/f69 1404960 0 2024-04-24T22:45:30.616 INFO:tasks.workunit.client.0.smithi063.stdout:2/638: creat d0/fbc x:0 0 0 2024-04-24T22:45:30.619 INFO:tasks.workunit.client.1.smithi192.stdout:8/442: link f6 db/f88 0 2024-04-24T22:45:30.619 INFO:tasks.workunit.client.0.smithi063.stdout:2/639: write d0/fa6 [1143987,118643] 0 2024-04-24T22:45:30.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/568: dread d1/d22/d33/da6/d54/f71 [0,4194304] 0 2024-04-24T22:45:30.620 INFO:tasks.workunit.client.0.smithi063.stdout:2/640: chown d0/dd/f99 82743952 1 2024-04-24T22:45:30.620 INFO:tasks.workunit.client.0.smithi063.stdout:5/547: dwrite d5/d24/d44/d71/dbb/f89 [0,4194304] 0 2024-04-24T22:45:30.622 INFO:tasks.workunit.client.0.smithi063.stdout:3/582: dread da/d18/d9e/da3/fab [0,4194304] 0 2024-04-24T22:45:30.622 INFO:tasks.workunit.client.0.smithi063.stdout:0/496: mkdir d6/d64/d24/d42/dad 0 2024-04-24T22:45:30.623 INFO:tasks.workunit.client.1.smithi192.stdout:4/376: creat d2/d9/d25/d2e/d31/d36/d5b/d64/f83 x:0 0 0 2024-04-24T22:45:30.624 INFO:tasks.workunit.client.0.smithi063.stdout:6/580: rmdir db/d1d/d2d/d69/d82 39 2024-04-24T22:45:30.627 INFO:tasks.workunit.client.0.smithi063.stdout:1/643: getdents d3/db5/d21/db4 0 2024-04-24T22:45:30.628 INFO:tasks.workunit.client.0.smithi063.stdout:4/515: truncate d2/f7c 1668216 0 2024-04-24T22:45:30.629 INFO:tasks.workunit.client.0.smithi063.stdout:4/516: chown d2/d8/d3c/f75 19810 1 2024-04-24T22:45:30.631 INFO:tasks.workunit.client.0.smithi063.stdout:9/579: mkdir d6/dbd 0 2024-04-24T22:45:30.635 INFO:tasks.workunit.client.1.smithi192.stdout:5/374: link d0/d13/d19/d64/d5f/f27 d0/d5b/f7a 0 2024-04-24T22:45:30.636 INFO:tasks.workunit.client.0.smithi063.stdout:7/503: rmdir d0/d4/de/d6f/d2e/d76/d9f/da3 0 2024-04-24T22:45:30.636 INFO:tasks.workunit.client.1.smithi192.stdout:7/525: dread d2/db/d21/d42/d67/d9b/f48 [0,4194304] 0 2024-04-24T22:45:30.637 INFO:tasks.workunit.client.1.smithi192.stdout:8/443: rename db/f1d to db/dc/d29/f89 0 2024-04-24T22:45:30.638 INFO:tasks.workunit.client.0.smithi063.stdout:8/569: fdatasync d1/d11/f47 0 2024-04-24T22:45:30.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/468: link d3/d27/d28/l21 d3/d27/d3c/d64/la2 0 2024-04-24T22:45:30.641 INFO:tasks.workunit.client.0.smithi063.stdout:1/644: dread d3/db5/d2b/f43 [4194304,4194304] 0 2024-04-24T22:45:30.644 INFO:tasks.workunit.client.1.smithi192.stdout:5/375: mknod d0/d13/c7b 0 2024-04-24T22:45:30.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/530: sync 2024-04-24T22:45:30.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/368: sync 2024-04-24T22:45:30.647 INFO:tasks.workunit.client.1.smithi192.stdout:1/392: sync 2024-04-24T22:45:30.647 INFO:tasks.workunit.client.0.smithi063.stdout:3/583: read da/d18/d47/d4d/f5a [751752,24304] 0 2024-04-24T22:45:30.648 INFO:tasks.workunit.client.1.smithi192.stdout:8/444: unlink db/f3d 0 2024-04-24T22:45:30.654 INFO:tasks.workunit.client.1.smithi192.stdout:9/530: truncate d3/f71 1256233 0 2024-04-24T22:45:30.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/445: dwrite db/dc/d22/f28 [0,4194304] 0 2024-04-24T22:45:30.655 INFO:tasks.workunit.client.1.smithi192.stdout:9/531: readlink d3/dd/de/l1b 0 2024-04-24T22:45:30.657 INFO:tasks.workunit.client.1.smithi192.stdout:4/377: truncate d2/d9/d25/d2e/d31/d36/d5b/f24 1446075 0 2024-04-24T22:45:30.661 INFO:tasks.workunit.client.1.smithi192.stdout:3/434: truncate d2/d4/d7/d4f/f7a 1887 0 2024-04-24T22:45:30.664 INFO:tasks.workunit.client.1.smithi192.stdout:8/446: dwrite db/dc/f1b [8388608,4194304] 0 2024-04-24T22:45:30.670 INFO:tasks.workunit.client.1.smithi192.stdout:2/531: creat d4/d9/d10/d6c/d95/fa2 x:0 0 0 2024-04-24T22:45:30.673 INFO:tasks.workunit.client.1.smithi192.stdout:6/369: symlink d0/d8/d3d/d3e/d40/d44/l84 0 2024-04-24T22:45:30.679 INFO:tasks.workunit.client.0.smithi063.stdout:1/645: rename d3/db5/d2b/d79/fac to d3/db5/d21/d49/fd2 0 2024-04-24T22:45:30.680 INFO:tasks.workunit.client.1.smithi192.stdout:9/532: dread d3/f31 [0,4194304] 0 2024-04-24T22:45:30.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/584: creat da/d39/d94/db3/fc9 x:0 0 0 2024-04-24T22:45:30.685 INFO:tasks.workunit.client.1.smithi192.stdout:0/469: dwrite d3/d40/f50 [0,4194304] 0 2024-04-24T22:45:30.686 INFO:tasks.workunit.client.1.smithi192.stdout:0/470: dread - d3/d27/f71 zero size 2024-04-24T22:45:30.691 INFO:tasks.workunit.client.1.smithi192.stdout:4/378: unlink d2/d9/d25/d2e/d31/d53/l78 0 2024-04-24T22:45:30.694 INFO:tasks.workunit.client.0.smithi063.stdout:4/517: symlink d2/d8/d36/la7 0 2024-04-24T22:45:30.696 INFO:tasks.workunit.client.0.smithi063.stdout:4/518: write d2/d8/d36/d3a/d50/f77 [946838,109466] 0 2024-04-24T22:45:30.697 INFO:tasks.workunit.client.0.smithi063.stdout:9/580: read d6/f30 [2609752,21060] 0 2024-04-24T22:45:30.698 INFO:tasks.workunit.client.0.smithi063.stdout:7/504: fdatasync d0/d4/de/d6f/f47 0 2024-04-24T22:45:30.701 INFO:tasks.workunit.client.1.smithi192.stdout:2/532: rename d4/d9/d10/f7f to d4/d9/d37/fa3 0 2024-04-24T22:45:30.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/570: symlink d1/d22/d33/da6/dae/lb8 0 2024-04-24T22:45:30.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/533: mkdir d3/d59/dbf 0 2024-04-24T22:45:30.714 INFO:tasks.workunit.client.1.smithi192.stdout:6/370: write d0/d8/d29/d58/f5b [2555386,7812] 0 2024-04-24T22:45:30.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/371: write d0/d8/f76 [951497,13587] 0 2024-04-24T22:45:30.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/372: chown d0 205430575 1 2024-04-24T22:45:30.716 INFO:tasks.workunit.client.1.smithi192.stdout:6/373: fdatasync d0/d8/f14 0 2024-04-24T22:45:30.720 INFO:tasks.workunit.client.0.smithi063.stdout:6/581: link db/d1d/d20/d29/f2e db/d1d/d20/d3f/fc5 0 2024-04-24T22:45:30.723 INFO:tasks.workunit.client.1.smithi192.stdout:1/393: creat d9/d51/f82 x:0 0 0 2024-04-24T22:45:30.724 INFO:tasks.workunit.client.0.smithi063.stdout:4/519: truncate d2/f6 2261353 0 2024-04-24T22:45:30.724 INFO:tasks.workunit.client.1.smithi192.stdout:0/471: dread d3/d27/d3c/d5d/f86 [0,4194304] 0 2024-04-24T22:45:30.724 INFO:tasks.workunit.client.1.smithi192.stdout:1/394: read d9/d15/f1f [4194153,26551] 0 2024-04-24T22:45:30.725 INFO:tasks.workunit.client.0.smithi063.stdout:1/646: dread d3/db5/d21/f7d [0,4194304] 0 2024-04-24T22:45:30.727 INFO:tasks.workunit.client.0.smithi063.stdout:8/571: fdatasync d1/d21/f3a 0 2024-04-24T22:45:30.729 INFO:tasks.workunit.client.0.smithi063.stdout:2/641: getdents d0/d2/d2d/d31/d89/d1c 0 2024-04-24T22:45:30.730 INFO:tasks.workunit.client.0.smithi063.stdout:8/572: truncate d1/d22/d33/da6/d54/f8e 1034029 0 2024-04-24T22:45:30.730 INFO:tasks.workunit.client.0.smithi063.stdout:2/642: chown d0/l2c 521331 1 2024-04-24T22:45:30.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/376: mkdir d0/d13/d19/d64/d4f/d7c 0 2024-04-24T22:45:30.732 INFO:tasks.workunit.client.0.smithi063.stdout:5/548: getdents d5/d11/d23/d2d/d62/d81 0 2024-04-24T22:45:30.733 INFO:tasks.workunit.client.1.smithi192.stdout:7/526: getdents d2/db/d21/d42/d67/d9b/d6e/d77/d8e 0 2024-04-24T22:45:30.734 INFO:tasks.workunit.client.1.smithi192.stdout:7/527: write d2/db/d25/f69 [731794,80165] 0 2024-04-24T22:45:30.736 INFO:tasks.workunit.client.0.smithi063.stdout:3/585: mknod da/d18/d3a/cca 0 2024-04-24T22:45:30.737 INFO:tasks.workunit.client.1.smithi192.stdout:9/534: mknod d3/d5d/da1/d85/cc0 0 2024-04-24T22:45:30.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/535: dread - d3/f58 zero size 2024-04-24T22:45:30.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/497: link d6/d64/d21/d25/d31/l50 d6/da/lae 0 2024-04-24T22:45:30.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/498: chown d6/d64/d21/d80/f9e 2901 1 2024-04-24T22:45:30.740 INFO:tasks.workunit.client.0.smithi063.stdout:6/582: read - db/d1d/d20/d3e/f45 zero size 2024-04-24T22:45:30.744 INFO:tasks.workunit.client.0.smithi063.stdout:9/581: unlink d6/d8/db/d6b/c87 0 2024-04-24T22:45:30.744 INFO:tasks.workunit.client.1.smithi192.stdout:2/533: dwrite d4/d9/d37/f64 [0,4194304] 0 2024-04-24T22:45:30.752 INFO:tasks.workunit.client.1.smithi192.stdout:3/435: creat d2/d4/d12/f88 x:0 0 0 2024-04-24T22:45:30.753 INFO:tasks.workunit.client.1.smithi192.stdout:2/534: dwrite d4/d9/d37/f55 [0,4194304] 0 2024-04-24T22:45:30.755 INFO:tasks.workunit.client.1.smithi192.stdout:0/472: mknod d3/d6d/ca3 0 2024-04-24T22:45:30.756 INFO:tasks.workunit.client.0.smithi063.stdout:4/520: dwrite d2/d19/f20 [0,4194304] 0 2024-04-24T22:45:30.757 INFO:tasks.workunit.client.1.smithi192.stdout:1/395: mkdir d9/dd/d74/d83 0 2024-04-24T22:45:30.757 INFO:tasks.workunit.client.1.smithi192.stdout:1/396: chown d9/d15/d2d/d48 105096884 1 2024-04-24T22:45:30.758 INFO:tasks.workunit.client.0.smithi063.stdout:4/521: write d2/f99 [2147062,93400] 0 2024-04-24T22:45:30.758 INFO:tasks.workunit.client.0.smithi063.stdout:3/586: rmdir da/d18/d9e/da3 39 2024-04-24T22:45:30.760 INFO:tasks.workunit.client.0.smithi063.stdout:0/499: creat d6/da/d2b/d60/faf x:0 0 0 2024-04-24T22:45:30.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/582: rename d6/d8/db/d6b/d4f/dad/d36/d7d to d6/d8/d53/dbe 0 2024-04-24T22:45:30.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/374: truncate d0/d8/d29/f7c 4137798 0 2024-04-24T22:45:30.768 INFO:tasks.workunit.client.1.smithi192.stdout:6/375: truncate d0/f82 5930057 0 2024-04-24T22:45:30.768 INFO:tasks.workunit.client.1.smithi192.stdout:8/447: getdents db/d43/d71 0 2024-04-24T22:45:30.771 INFO:tasks.workunit.client.0.smithi063.stdout:5/549: dread d5/d24/d36/f5a [0,4194304] 0 2024-04-24T22:45:30.771 INFO:tasks.workunit.client.0.smithi063.stdout:7/505: sync 2024-04-24T22:45:30.782 INFO:tasks.workunit.client.1.smithi192.stdout:4/379: getdents d2/d9/d25/d2e/d31/d36/d5b/d28 0 2024-04-24T22:45:30.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/647: truncate d3/db5/d21/d9f/daa/db7/f58 5792284 0 2024-04-24T22:45:30.784 INFO:tasks.workunit.client.0.smithi063.stdout:0/500: creat d6/d3a/fb0 x:0 0 0 2024-04-24T22:45:30.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/573: rename d1/d11/d32/d38/fb0 to d1/d22/d33/da6/d54/d7b/fb9 0 2024-04-24T22:45:30.789 INFO:tasks.workunit.client.0.smithi063.stdout:0/501: dread d6/d64/d21/f86 [0,4194304] 0 2024-04-24T22:45:30.789 INFO:tasks.workunit.client.0.smithi063.stdout:8/574: chown d1/d22/d33/f57 146508 1 2024-04-24T22:45:30.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/643: link d0/d2/d2d/d31/d89/d7/d86/d8e/la7 d0/d2/d6c/db8/lbd 0 2024-04-24T22:45:30.791 INFO:tasks.workunit.client.0.smithi063.stdout:2/644: read - d0/dd/d72/d77/f78 zero size 2024-04-24T22:45:30.793 INFO:tasks.workunit.client.1.smithi192.stdout:0/473: write d3/d27/d28/d2e/d4a/d5c/f59 [258634,21714] 0 2024-04-24T22:45:30.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/522: mkdir d2/d19/d33/d9d/da8 0 2024-04-24T22:45:30.795 INFO:tasks.workunit.client.1.smithi192.stdout:5/377: truncate d0/d13/d19/d64/d4f/d52/f67 3556593 0 2024-04-24T22:45:30.795 INFO:tasks.workunit.client.0.smithi063.stdout:7/506: dread d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:30.796 INFO:tasks.workunit.client.0.smithi063.stdout:4/523: fsync d2/d80/f8f 0 2024-04-24T22:45:30.796 INFO:tasks.workunit.client.1.smithi192.stdout:9/536: write d3/d5d/fa9 [2054357,34325] 0 2024-04-24T22:45:30.797 INFO:tasks.workunit.client.0.smithi063.stdout:4/524: stat d2/d8/d40/f68 0 2024-04-24T22:45:30.797 INFO:tasks.workunit.client.0.smithi063.stdout:6/583: link db/d1d/d20/d3f/d7f/d68/f7e db/fc6 0 2024-04-24T22:45:30.797 INFO:tasks.workunit.client.1.smithi192.stdout:5/378: fsync d0/d13/d19/d64/d5f/d2c/f51 0 2024-04-24T22:45:30.799 INFO:tasks.workunit.client.1.smithi192.stdout:5/379: truncate d0/d13/d19/d64/d5f/d2c/f5c 875738 0 2024-04-24T22:45:30.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/448: getdents db/d43/d7c/d85 0 2024-04-24T22:45:30.801 INFO:tasks.workunit.client.1.smithi192.stdout:5/380: chown d0/d13/d19/d64/d5f/f20 13 1 2024-04-24T22:45:30.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/449: readlink db/dc/d29/l46 0 2024-04-24T22:45:30.804 INFO:tasks.workunit.client.1.smithi192.stdout:6/376: rename d0/d8/l13 to d0/d8/d29/d4c/l85 0 2024-04-24T22:45:30.805 INFO:tasks.workunit.client.0.smithi063.stdout:1/648: dwrite d3/f19 [0,4194304] 0 2024-04-24T22:45:30.807 INFO:tasks.workunit.client.0.smithi063.stdout:3/587: write da/f24 [1300966,77987] 0 2024-04-24T22:45:30.808 INFO:tasks.workunit.client.1.smithi192.stdout:2/535: creat d4/d9/fa4 x:0 0 0 2024-04-24T22:45:30.810 INFO:tasks.workunit.client.0.smithi063.stdout:4/525: dread d2/d8/d36/d3a/d50/f9e [0,4194304] 0 2024-04-24T22:45:30.810 INFO:tasks.workunit.client.1.smithi192.stdout:8/450: dwrite db/dc/d29/f47 [0,4194304] 0 2024-04-24T22:45:30.816 INFO:tasks.workunit.client.0.smithi063.stdout:0/502: mknod d6/d64/d21/d25/d62/cb1 0 2024-04-24T22:45:30.816 INFO:tasks.workunit.client.0.smithi063.stdout:0/503: readlink d6/da/dc/l1c 0 2024-04-24T22:45:30.819 INFO:tasks.workunit.client.0.smithi063.stdout:2/645: creat d0/d2/d2d/d31/d89/d7/d86/d8e/fbe x:0 0 0 2024-04-24T22:45:30.820 INFO:tasks.workunit.client.0.smithi063.stdout:2/646: readlink d0/l2c 0 2024-04-24T22:45:30.820 INFO:tasks.workunit.client.1.smithi192.stdout:8/451: dread db/dc/f41 [0,4194304] 0 2024-04-24T22:45:30.821 INFO:tasks.workunit.client.1.smithi192.stdout:8/452: stat db/dc/c25 0 2024-04-24T22:45:30.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/537: symlink d3/dd/d53/lc1 0 2024-04-24T22:45:30.822 INFO:tasks.workunit.client.0.smithi063.stdout:6/584: creat db/d85/fc7 x:0 0 0 2024-04-24T22:45:30.823 INFO:tasks.workunit.client.0.smithi063.stdout:3/588: dread da/d18/d47/f85 [0,4194304] 0 2024-04-24T22:45:30.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/381: rename d0/d13/d19/d64/d5f/f33 to d0/d1b/d50/f7d 0 2024-04-24T22:45:30.827 INFO:tasks.workunit.client.0.smithi063.stdout:2/647: dread d0/d2/d2d/d31/d89/d1c/f2a [0,4194304] 0 2024-04-24T22:45:30.829 INFO:tasks.workunit.client.0.smithi063.stdout:1/649: rmdir d3/db5 39 2024-04-24T22:45:30.829 INFO:tasks.workunit.client.1.smithi192.stdout:2/536: mkdir d4/d9/d10/d6c/d8c/da5 0 2024-04-24T22:45:30.831 INFO:tasks.workunit.client.0.smithi063.stdout:9/583: rename d6/l18 to d6/d8/d53/dbe/lbf 0 2024-04-24T22:45:30.832 INFO:tasks.workunit.client.1.smithi192.stdout:1/397: getdents d9 0 2024-04-24T22:45:30.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/589: dwrite da/f24 [0,4194304] 0 2024-04-24T22:45:30.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/585: dwrite db/d1d/d20/d29/d5d/f6b [0,4194304] 0 2024-04-24T22:45:30.833 INFO:tasks.workunit.client.1.smithi192.stdout:1/398: stat d9/d15/f47 0 2024-04-24T22:45:30.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/474: mknod d3/ca4 0 2024-04-24T22:45:30.837 INFO:tasks.workunit.client.1.smithi192.stdout:4/380: truncate d2/db/f52 416392 0 2024-04-24T22:45:30.837 INFO:tasks.workunit.client.1.smithi192.stdout:0/475: chown d3/d27/d3c/d3b/f8d 1047279 1 2024-04-24T22:45:30.839 INFO:tasks.workunit.client.1.smithi192.stdout:0/476: write d3/d27/d28/d2e/f4c [4330648,15547] 0 2024-04-24T22:45:30.841 INFO:tasks.workunit.client.0.smithi063.stdout:0/504: symlink d6/d64/d21/lb2 0 2024-04-24T22:45:30.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/590: dwrite da/d18/d47/d4d/fae [0,4194304] 0 2024-04-24T22:45:30.845 INFO:tasks.workunit.client.1.smithi192.stdout:5/382: rename d0/d13/d19/f38 to d0/d13/d19/d64/d4f/d52/f7e 0 2024-04-24T22:45:30.847 INFO:tasks.workunit.client.1.smithi192.stdout:6/377: getdents d0/d8/d29/d4c/d4d/d6a 0 2024-04-24T22:45:30.854 INFO:tasks.workunit.client.0.smithi063.stdout:2/648: rmdir d0/d2/d2d/d31/d89 39 2024-04-24T22:45:30.862 INFO:tasks.workunit.client.1.smithi192.stdout:9/538: mkdir d3/d59/d7d/d88/dc2 0 2024-04-24T22:45:30.864 INFO:tasks.workunit.client.0.smithi063.stdout:5/550: rename d5/d24/d44/d71 to d5/d1d/dc0 0 2024-04-24T22:45:30.871 INFO:tasks.workunit.client.1.smithi192.stdout:0/477: rmdir d3/d40 39 2024-04-24T22:45:30.873 INFO:tasks.workunit.client.0.smithi063.stdout:8/575: creat d1/d2/fba x:0 0 0 2024-04-24T22:45:30.874 INFO:tasks.workunit.client.1.smithi192.stdout:8/453: mkdir db/dc/d22/d31/d33/d81/d8a 0 2024-04-24T22:45:30.877 INFO:tasks.workunit.client.0.smithi063.stdout:0/505: creat d6/d64/d21/fb3 x:0 0 0 2024-04-24T22:45:30.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/378: readlink d0/d8/d18/d2e/l54 0 2024-04-24T22:45:30.879 INFO:tasks.workunit.client.1.smithi192.stdout:0/478: dwrite d3/d27/d28/d2e/f53 [0,4194304] 0 2024-04-24T22:45:30.881 INFO:tasks.workunit.client.1.smithi192.stdout:6/379: dread - d0/d8/d29/d4c/f6d zero size 2024-04-24T22:45:30.883 INFO:tasks.workunit.client.1.smithi192.stdout:2/537: symlink d4/d9/d10/d6c/la6 0 2024-04-24T22:45:30.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/380: write d0/f82 [5444933,51621] 0 2024-04-24T22:45:30.885 INFO:tasks.workunit.client.0.smithi063.stdout:3/591: creat da/d18/d47/d4d/d62/fcb x:0 0 0 2024-04-24T22:45:30.885 INFO:tasks.workunit.client.1.smithi192.stdout:8/454: dwrite db/dc/d29/f2b [4194304,4194304] 0 2024-04-24T22:45:30.889 INFO:tasks.workunit.client.0.smithi063.stdout:1/650: creat d3/da/da2/dba/d11/d5e/d9c/fd3 x:0 0 0 2024-04-24T22:45:30.894 INFO:tasks.workunit.client.0.smithi063.stdout:4/526: rename d2/d8/d36/d3a/f9a to d2/d19/d82/d96/fa9 0 2024-04-24T22:45:30.895 INFO:tasks.workunit.client.0.smithi063.stdout:5/551: mknod d5/d1d/d7d/cc1 0 2024-04-24T22:45:30.898 INFO:tasks.workunit.client.0.smithi063.stdout:6/586: creat db/d1c/dc3/fc8 x:0 0 0 2024-04-24T22:45:30.900 INFO:tasks.workunit.client.0.smithi063.stdout:6/587: write db/d1c/fa7 [4223705,94898] 0 2024-04-24T22:45:30.902 INFO:tasks.workunit.client.0.smithi063.stdout:4/527: dwrite d2/d8/f71 [0,4194304] 0 2024-04-24T22:45:30.902 INFO:tasks.workunit.client.0.smithi063.stdout:9/584: mkdir d6/d8/db/d6b/d4f/dad/dc0 0 2024-04-24T22:45:30.906 INFO:tasks.workunit.client.0.smithi063.stdout:0/506: rename d6/d64/d24/d42/c6d to d6/d64/d24/d42/dad/cb4 0 2024-04-24T22:45:30.906 INFO:tasks.workunit.client.0.smithi063.stdout:0/507: fdatasync d6/da/d2b/fa7 0 2024-04-24T22:45:30.908 INFO:tasks.workunit.client.0.smithi063.stdout:8/576: creat d1/d2/d3/db6/fbb x:0 0 0 2024-04-24T22:45:30.913 INFO:tasks.workunit.client.0.smithi063.stdout:0/508: dread d6/d64/d24/f6b [0,4194304] 0 2024-04-24T22:45:30.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/577: dwrite d1/d11/d3c/f55 [0,4194304] 0 2024-04-24T22:45:30.917 INFO:tasks.workunit.client.1.smithi192.stdout:0/479: fsync d3/f48 0 2024-04-24T22:45:30.918 INFO:tasks.workunit.client.0.smithi063.stdout:5/552: dread d5/d24/d49/d3b/f48 [0,4194304] 0 2024-04-24T22:45:30.918 INFO:tasks.workunit.client.0.smithi063.stdout:4/528: symlink d2/d19/d82/laa 0 2024-04-24T22:45:30.920 INFO:tasks.workunit.client.0.smithi063.stdout:5/553: chown d5/d11/d18/f7f 286037 1 2024-04-24T22:45:30.926 INFO:tasks.workunit.client.1.smithi192.stdout:9/539: mknod d3/cc3 0 2024-04-24T22:45:30.930 INFO:tasks.workunit.client.0.smithi063.stdout:0/509: mkdir d6/d64/da0/d33/db5 0 2024-04-24T22:45:30.930 INFO:tasks.workunit.client.0.smithi063.stdout:7/507: sync 2024-04-24T22:45:30.930 INFO:tasks.workunit.client.1.smithi192.stdout:8/455: dwrite db/dc/f5b [0,4194304] 0 2024-04-24T22:45:30.932 INFO:tasks.workunit.client.0.smithi063.stdout:6/588: dread db/d1d/d20/d3e/f5e [0,4194304] 0 2024-04-24T22:45:30.935 INFO:tasks.workunit.client.0.smithi063.stdout:4/529: creat d2/d19/d33/d9d/fab x:0 0 0 2024-04-24T22:45:30.939 INFO:tasks.workunit.client.1.smithi192.stdout:2/538: mkdir d4/d9/d10/da7 0 2024-04-24T22:45:30.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/530: dwrite d2/fb [0,4194304] 0 2024-04-24T22:45:30.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/554: mknod d5/d24/d36/d65/cc2 0 2024-04-24T22:45:30.943 INFO:tasks.workunit.client.1.smithi192.stdout:1/399: link d9/d51/c5c d9/c84 0 2024-04-24T22:45:30.943 INFO:tasks.workunit.client.0.smithi063.stdout:5/555: chown d5/d11/d23/d2d/l6e 22 1 2024-04-24T22:45:30.944 INFO:tasks.workunit.client.0.smithi063.stdout:5/556: chown d5/d11/d23/d2d/d62/d81/cb8 97526 1 2024-04-24T22:45:30.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/557: fsync d5/d24/f51 0 2024-04-24T22:45:30.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/510: unlink d6/d64/da0/d33/f51 0 2024-04-24T22:45:30.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/511: write d6/d64/d21/d25/d62/d63/f82 [553465,106869] 0 2024-04-24T22:45:30.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/558: dwrite d5/d1d/dc0/dbb/f89 [4194304,4194304] 0 2024-04-24T22:45:30.954 INFO:tasks.workunit.client.0.smithi063.stdout:7/508: creat d0/d4/de/d17/d35/d69/fb0 x:0 0 0 2024-04-24T22:45:30.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/539: mkdir d4/d9/d10/d6c/d6f/da8 0 2024-04-24T22:45:30.956 INFO:tasks.workunit.client.1.smithi192.stdout:2/540: fsync d4/d9/d37/f8a 0 2024-04-24T22:45:30.956 INFO:tasks.workunit.client.0.smithi063.stdout:2/649: sync 2024-04-24T22:45:30.961 INFO:tasks.workunit.client.1.smithi192.stdout:9/540: mkdir d3/dd/de/d17/d89/d8b/dc4 0 2024-04-24T22:45:30.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/480: dread d3/f14 [4194304,4194304] 0 2024-04-24T22:45:30.962 INFO:tasks.workunit.client.1.smithi192.stdout:4/381: write d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f34 [1149158,68359] 0 2024-04-24T22:45:30.964 INFO:tasks.workunit.client.1.smithi192.stdout:8/456: creat db/dc/d22/d31/d33/d81/d8a/f8b x:0 0 0 2024-04-24T22:45:30.968 INFO:tasks.workunit.client.0.smithi063.stdout:9/585: rmdir d6/d8/db/d6b/d4f/dad/d48/d94/da1/da7 0 2024-04-24T22:45:30.969 INFO:tasks.workunit.client.0.smithi063.stdout:3/592: write da/d18/f27 [685928,45952] 0 2024-04-24T22:45:30.972 INFO:tasks.workunit.client.0.smithi063.stdout:1/651: truncate d3/da/da2/dba/f8c 1593007 0 2024-04-24T22:45:30.976 INFO:tasks.workunit.client.0.smithi063.stdout:1/652: dread d3/da/d34/f93 [0,4194304] 0 2024-04-24T22:45:30.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/383: dwrite d0/d13/d19/d64/d4f/d52/f7e [0,4194304] 0 2024-04-24T22:45:30.977 INFO:tasks.workunit.client.1.smithi192.stdout:6/381: write d0/d8/d22/f27 [2849201,3181] 0 2024-04-24T22:45:30.980 INFO:tasks.workunit.client.1.smithi192.stdout:8/457: dread db/f88 [4194304,4194304] 0 2024-04-24T22:45:30.981 INFO:tasks.workunit.client.1.smithi192.stdout:8/458: write db/dc/d22/f66 [566194,25737] 0 2024-04-24T22:45:30.984 INFO:tasks.workunit.client.1.smithi192.stdout:2/541: mkdir d4/d29/da9 0 2024-04-24T22:45:30.988 INFO:tasks.workunit.client.0.smithi063.stdout:7/509: mkdir d0/d4/de/d17/d35/d69/db1 0 2024-04-24T22:45:30.989 INFO:tasks.workunit.client.1.smithi192.stdout:0/481: fdatasync d3/d40/f9b 0 2024-04-24T22:45:30.990 INFO:tasks.workunit.client.1.smithi192.stdout:0/482: fsync d3/d27/d28/d31/f52 0 2024-04-24T22:45:30.990 INFO:tasks.workunit.client.0.smithi063.stdout:2/650: mknod d0/d2/d2f/cbf 0 2024-04-24T22:45:30.991 INFO:tasks.workunit.client.0.smithi063.stdout:8/578: dwrite d1/d2/f2f [0,4194304] 0 2024-04-24T22:45:30.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/579: chown d1/d22/c6c 769046635 1 2024-04-24T22:45:30.998 INFO:tasks.workunit.client.0.smithi063.stdout:4/531: symlink d2/d8/lac 0 2024-04-24T22:45:30.999 INFO:tasks.workunit.client.1.smithi192.stdout:4/382: symlink d2/l84 0 2024-04-24T22:45:31.005 INFO:tasks.workunit.client.1.smithi192.stdout:2/542: dread d4/d9/d37/f4a [0,4194304] 0 2024-04-24T22:45:31.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/512: creat d6/d64/d21/d25/d62/d63/d7c/fb6 x:0 0 0 2024-04-24T22:45:31.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/589: dwrite db/d1d/d20/d29/f2e [0,4194304] 0 2024-04-24T22:45:31.010 INFO:tasks.workunit.client.1.smithi192.stdout:0/483: mknod d3/d27/d28/d2e/d4a/d5c/d80/d83/ca5 0 2024-04-24T22:45:31.013 INFO:tasks.workunit.client.0.smithi063.stdout:7/510: write d0/d4/de/f7c [4129225,120610] 0 2024-04-24T22:45:31.017 INFO:tasks.workunit.client.1.smithi192.stdout:7/528: sync 2024-04-24T22:45:31.017 INFO:tasks.workunit.client.1.smithi192.stdout:3/436: sync 2024-04-24T22:45:31.017 INFO:tasks.workunit.client.0.smithi063.stdout:6/590: dread db/d1d/d20/d3f/d7f/d68/d70/f67 [0,4194304] 0 2024-04-24T22:45:31.019 INFO:tasks.workunit.client.1.smithi192.stdout:5/384: dread d0/d13/d19/d64/d5f/f20 [0,4194304] 0 2024-04-24T22:45:31.019 INFO:tasks.workunit.client.1.smithi192.stdout:7/529: read - d2/db/d21/d42/f93 zero size 2024-04-24T22:45:31.020 INFO:tasks.workunit.client.0.smithi063.stdout:6/591: chown db/d1d/d20 3105 1 2024-04-24T22:45:31.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/511: dwrite d0/fa2 [0,4194304] 0 2024-04-24T22:45:31.020 INFO:tasks.workunit.client.1.smithi192.stdout:9/541: dread d3/f4 [0,4194304] 0 2024-04-24T22:45:31.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/385: stat d0/d1b/f65 0 2024-04-24T22:45:31.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/383: rmdir d2/db/d10 39 2024-04-24T22:45:31.021 INFO:tasks.workunit.client.0.smithi063.stdout:6/592: read db/d1c/f90 [1611019,103116] 0 2024-04-24T22:45:31.022 INFO:tasks.workunit.client.1.smithi192.stdout:5/386: readlink d0/d13/d19/l9 0 2024-04-24T22:45:31.022 INFO:tasks.workunit.client.0.smithi063.stdout:6/593: read db/d1d/d2d/f42 [813675,5480] 0 2024-04-24T22:45:31.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/594: write db/d1c/fa7 [746871,87073] 0 2024-04-24T22:45:31.027 INFO:tasks.workunit.client.0.smithi063.stdout:4/532: creat d2/d19/d82/d96/fad x:0 0 0 2024-04-24T22:45:31.027 INFO:tasks.workunit.client.0.smithi063.stdout:6/595: stat db/d1d/d20/d3f/d7f/d68/d70/fad 0 2024-04-24T22:45:31.028 INFO:tasks.workunit.client.1.smithi192.stdout:2/543: mknod d4/d9/d10/d6c/d95/caa 0 2024-04-24T22:45:31.029 INFO:tasks.workunit.client.0.smithi063.stdout:5/559: dwrite d5/d11/d23/f79 [0,4194304] 0 2024-04-24T22:45:31.030 INFO:tasks.workunit.client.1.smithi192.stdout:0/484: mknod d3/d40/ca6 0 2024-04-24T22:45:31.030 INFO:tasks.workunit.client.0.smithi063.stdout:5/560: chown d5/d11/d76/f7b 7540 1 2024-04-24T22:45:31.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/513: creat d6/d64/d21/d25/d62/fb7 x:0 0 0 2024-04-24T22:45:31.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/514: stat d6/d64/d21/d25/d62/d75/f81 0 2024-04-24T22:45:31.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/515: fdatasync d6/d64/f3b 0 2024-04-24T22:45:31.037 INFO:tasks.workunit.client.1.smithi192.stdout:6/382: dwrite d0/d8/d18/f3f [0,4194304] 0 2024-04-24T22:45:31.038 INFO:tasks.workunit.client.1.smithi192.stdout:6/383: chown d0/d8/d29/f3b 43392 1 2024-04-24T22:45:31.040 INFO:tasks.workunit.client.0.smithi063.stdout:2/651: write d0/d2/d2d/d31/d89/d7/f53 [753021,122337] 0 2024-04-24T22:45:31.041 INFO:tasks.workunit.client.0.smithi063.stdout:2/652: fsync d0/dd/d72/d77/d87/f9c 0 2024-04-24T22:45:31.042 INFO:tasks.workunit.client.0.smithi063.stdout:2/653: chown d0/d2/d2f/dba 36265867 1 2024-04-24T22:45:31.043 INFO:tasks.workunit.client.1.smithi192.stdout:8/459: link db/dc/d22/d31/f45 db/d43/d7c/d85/f8c 0 2024-04-24T22:45:31.046 INFO:tasks.workunit.client.1.smithi192.stdout:2/544: creat d4/d9/d10/d6c/d8c/fab x:0 0 0 2024-04-24T22:45:31.048 INFO:tasks.workunit.client.0.smithi063.stdout:9/586: creat d6/d8/db/fc1 x:0 0 0 2024-04-24T22:45:31.048 INFO:tasks.workunit.client.1.smithi192.stdout:3/437: mkdir d2/d89 0 2024-04-24T22:45:31.051 INFO:tasks.workunit.client.1.smithi192.stdout:7/530: mknod d2/db/d21/d42/d5b/ca5 0 2024-04-24T22:45:31.052 INFO:tasks.workunit.client.0.smithi063.stdout:3/593: mknod da/d18/d9e/da3/dc1/ccc 0 2024-04-24T22:45:31.055 INFO:tasks.workunit.client.0.smithi063.stdout:1/653: creat d3/db5/d21/db4/fd4 x:0 0 0 2024-04-24T22:45:31.059 INFO:tasks.workunit.client.1.smithi192.stdout:8/460: read - db/dc/d22/d31/f4c zero size 2024-04-24T22:45:31.059 INFO:tasks.workunit.client.1.smithi192.stdout:7/531: dwrite d2/db/d21/d42/d5b/d63/f6d [0,4194304] 0 2024-04-24T22:45:31.060 INFO:tasks.workunit.client.0.smithi063.stdout:0/516: creat d6/d64/da0/d33/fb8 x:0 0 0 2024-04-24T22:45:31.062 INFO:tasks.workunit.client.1.smithi192.stdout:0/485: symlink d3/d27/d28/d2e/d4a/d5c/d80/d94/la7 0 2024-04-24T22:45:31.063 INFO:tasks.workunit.client.1.smithi192.stdout:0/486: readlink d3/d27/d28/d2e/d3f/l7b 0 2024-04-24T22:45:31.064 INFO:tasks.workunit.client.1.smithi192.stdout:0/487: chown d3/d27/d28/d31/f85 107187477 1 2024-04-24T22:45:31.069 INFO:tasks.workunit.client.0.smithi063.stdout:8/580: creat d1/d2/fbc x:0 0 0 2024-04-24T22:45:31.070 INFO:tasks.workunit.client.1.smithi192.stdout:1/400: getdents d9/d15/d5e/d67 0 2024-04-24T22:45:31.073 INFO:tasks.workunit.client.0.smithi063.stdout:7/512: mknod d0/d4/de/d17/cb2 0 2024-04-24T22:45:31.074 INFO:tasks.workunit.client.1.smithi192.stdout:9/542: rmdir d3/dd/de/d17/d89/d8b/dc4 0 2024-04-24T22:45:31.075 INFO:tasks.workunit.client.1.smithi192.stdout:9/543: chown d3/dd/d53/f56 26 1 2024-04-24T22:45:31.076 INFO:tasks.workunit.client.1.smithi192.stdout:6/384: creat d0/d8/d29/d4c/d4d/d83/f86 x:0 0 0 2024-04-24T22:45:31.077 INFO:tasks.workunit.client.0.smithi063.stdout:4/533: dread d2/d19/f2b [0,4194304] 0 2024-04-24T22:45:31.079 INFO:tasks.workunit.client.1.smithi192.stdout:5/387: truncate d0/d13/d19/f60 239642 0 2024-04-24T22:45:31.080 INFO:tasks.workunit.client.1.smithi192.stdout:8/461: mknod db/d43/c8d 0 2024-04-24T22:45:31.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/534: write d2/d19/f37 [3332062,33398] 0 2024-04-24T22:45:31.083 INFO:tasks.workunit.client.0.smithi063.stdout:7/513: dwrite d0/d4/de/d17/d35/f54 [0,4194304] 0 2024-04-24T22:45:31.085 INFO:tasks.workunit.client.1.smithi192.stdout:9/544: dread d3/dd/d53/f7f [0,4194304] 0 2024-04-24T22:45:31.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/535: chown d2/d19/d33/l43 66665 1 2024-04-24T22:45:31.088 INFO:tasks.workunit.client.1.smithi192.stdout:1/401: creat d9/d15/d2d/d31/d2f/f85 x:0 0 0 2024-04-24T22:45:31.088 INFO:tasks.workunit.client.0.smithi063.stdout:9/587: dwrite d6/d2f/d42/f7b [0,4194304] 0 2024-04-24T22:45:31.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/561: dwrite d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:31.090 INFO:tasks.workunit.client.0.smithi063.stdout:7/514: chown d0/d4/de/d6f/f53 1927531 1 2024-04-24T22:45:31.091 INFO:tasks.workunit.client.0.smithi063.stdout:7/515: chown d0/d4/f11 3980 1 2024-04-24T22:45:31.094 INFO:tasks.workunit.client.1.smithi192.stdout:7/532: truncate d2/db/d25/f27 4862516 0 2024-04-24T22:45:31.097 INFO:tasks.workunit.client.0.smithi063.stdout:9/588: dread d6/d8/db/d6b/d4f/dad/d48/d94/fa6 [0,4194304] 0 2024-04-24T22:45:31.098 INFO:tasks.workunit.client.0.smithi063.stdout:6/596: creat db/d1d/fc9 x:0 0 0 2024-04-24T22:45:31.099 INFO:tasks.workunit.client.1.smithi192.stdout:6/385: dwrite d0/d8/d18/d2e/f51 [0,4194304] 0 2024-04-24T22:45:31.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/388: readlink d0/d13/d3b/l5d 0 2024-04-24T22:45:31.102 INFO:tasks.workunit.client.0.smithi063.stdout:3/594: symlink da/d39/d94/db3/dbe/lcd 0 2024-04-24T22:45:31.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/488: unlink d3/d27/d3c/d64/la2 0 2024-04-24T22:45:31.111 INFO:tasks.workunit.client.0.smithi063.stdout:2/654: truncate d0/d2/d2d/d31/fab 1896454 0 2024-04-24T22:45:31.112 INFO:tasks.workunit.client.0.smithi063.stdout:2/655: chown d0/d2/d39/f65 3 1 2024-04-24T22:45:31.113 INFO:tasks.workunit.client.0.smithi063.stdout:1/654: unlink d3/da/da2/dba/d11/d5e/d9c/dcb/lc5 0 2024-04-24T22:45:31.113 INFO:tasks.workunit.client.0.smithi063.stdout:0/517: dwrite d6/d64/d21/d25/d62/d75/f81 [0,4194304] 0 2024-04-24T22:45:31.113 INFO:tasks.workunit.client.1.smithi192.stdout:2/545: dwrite d4/d9/fb [0,4194304] 0 2024-04-24T22:45:31.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/489: dwrite d3/d27/d28/d2e/d4a/f4d [4194304,4194304] 0 2024-04-24T22:45:31.114 INFO:tasks.workunit.client.1.smithi192.stdout:9/545: creat d3/dd/d62/d6d/fc5 x:0 0 0 2024-04-24T22:45:31.116 INFO:tasks.workunit.client.1.smithi192.stdout:1/402: chown d9/d15/f1b 76008901 1 2024-04-24T22:45:31.119 INFO:tasks.workunit.client.0.smithi063.stdout:8/581: getdents d1/d11/d32/d38/db7 0 2024-04-24T22:45:31.119 INFO:tasks.workunit.client.0.smithi063.stdout:5/562: symlink d5/d24/d49/d3b/lc3 0 2024-04-24T22:45:31.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/389: mkdir d0/d13/d19/d71/d7f 0 2024-04-24T22:45:31.120 INFO:tasks.workunit.client.0.smithi063.stdout:8/582: fsync d1/d2/f1c 0 2024-04-24T22:45:31.121 INFO:tasks.workunit.client.0.smithi063.stdout:8/583: chown d1/d22/l66 1502419324 1 2024-04-24T22:45:31.134 INFO:tasks.workunit.client.1.smithi192.stdout:3/438: getdents d2/d4/d7 0 2024-04-24T22:45:31.134 INFO:tasks.workunit.client.1.smithi192.stdout:8/462: dwrite db/dc/d29/f4f [0,4194304] 0 2024-04-24T22:45:31.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/597: rmdir db/d85 39 2024-04-24T22:45:31.138 INFO:tasks.workunit.client.1.smithi192.stdout:9/546: write d3/fb [2123264,5905] 0 2024-04-24T22:45:31.138 INFO:tasks.workunit.client.1.smithi192.stdout:9/547: chown d3/d5d/da1/d85 439694 1 2024-04-24T22:45:31.139 INFO:tasks.workunit.client.0.smithi063.stdout:1/655: mkdir d3/db5/d21/d9f/dd5 0 2024-04-24T22:45:31.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/548: readlink d3/d5d/da1/d5e/d98/la0 0 2024-04-24T22:45:31.140 INFO:tasks.workunit.client.0.smithi063.stdout:5/563: symlink d5/d11/d23/d2d/lc4 0 2024-04-24T22:45:31.141 INFO:tasks.workunit.client.1.smithi192.stdout:0/490: truncate d3/d27/d28/f17 3262305 0 2024-04-24T22:45:31.141 INFO:tasks.workunit.client.0.smithi063.stdout:8/584: dwrite d1/d11/d32/fa5 [0,4194304] 0 2024-04-24T22:45:31.142 INFO:tasks.workunit.client.0.smithi063.stdout:4/536: dread d2/d8/d36/d3a/f4b [0,4194304] 0 2024-04-24T22:45:31.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/516: getdents d0/d4/de/d17/d35/d69/db1 0 2024-04-24T22:45:31.144 INFO:tasks.workunit.client.0.smithi063.stdout:4/537: write d2/ff [4985883,68872] 0 2024-04-24T22:45:31.144 INFO:tasks.workunit.client.0.smithi063.stdout:8/585: write d1/d21/f6f [1338345,116229] 0 2024-04-24T22:45:31.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/589: write d6/f7 [3236139,56069] 0 2024-04-24T22:45:31.149 INFO:tasks.workunit.client.0.smithi063.stdout:8/586: write d1/d2/d3/db6/fbb [838930,38330] 0 2024-04-24T22:45:31.150 INFO:tasks.workunit.client.0.smithi063.stdout:3/595: truncate da/f29 809239 0 2024-04-24T22:45:31.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/590: chown d6/d8/d1f/d20/d2a/d4b/l58 84461 1 2024-04-24T22:45:31.153 INFO:tasks.workunit.client.0.smithi063.stdout:6/598: rmdir db/d1d/d20/d3f/d7f/d68 39 2024-04-24T22:45:31.155 INFO:tasks.workunit.client.0.smithi063.stdout:6/599: write db/d1d/d2d/fc1 [944435,56716] 0 2024-04-24T22:45:31.160 INFO:tasks.workunit.client.1.smithi192.stdout:9/549: truncate d3/dd/f49 4090864 0 2024-04-24T22:45:31.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/403: chown d9/dd/c1e 136 1 2024-04-24T22:45:31.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/587: mkdir d1/d22/d33/dbd 0 2024-04-24T22:45:31.163 INFO:tasks.workunit.client.1.smithi192.stdout:7/533: creat d2/db/fa6 x:0 0 0 2024-04-24T22:45:31.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/390: mknod d0/d13/d19/d71/d7f/c80 0 2024-04-24T22:45:31.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/518: rename d6/d64/d21/c57 to d6/da/cb9 0 2024-04-24T22:45:31.167 INFO:tasks.workunit.client.0.smithi063.stdout:3/596: truncate da/d18/d9e/da3/fab 4644404 0 2024-04-24T22:45:31.168 INFO:tasks.workunit.client.0.smithi063.stdout:3/597: chown da/d18/d47/d4d/c9a 624011697 1 2024-04-24T22:45:31.173 INFO:tasks.workunit.client.0.smithi063.stdout:7/517: mknod d0/d4/de/d17/cb3 0 2024-04-24T22:45:31.179 INFO:tasks.workunit.client.0.smithi063.stdout:4/538: rename d2/d8/d36/c59 to d2/d19/d82/cae 0 2024-04-24T22:45:31.179 INFO:tasks.workunit.client.0.smithi063.stdout:4/539: readlink d2/d19/l60 0 2024-04-24T22:45:31.180 INFO:tasks.workunit.client.1.smithi192.stdout:2/546: dread d4/f78 [0,4194304] 0 2024-04-24T22:45:31.182 INFO:tasks.workunit.client.0.smithi063.stdout:1/656: dread d3/db5/d21/d9f/daa/db7/f73 [0,4194304] 0 2024-04-24T22:45:31.182 INFO:tasks.workunit.client.1.smithi192.stdout:7/534: rename d2/f89 to d2/db/d25/d82/fa7 0 2024-04-24T22:45:31.183 INFO:tasks.workunit.client.0.smithi063.stdout:0/519: mkdir d6/d3a/dba 0 2024-04-24T22:45:31.184 INFO:tasks.workunit.client.0.smithi063.stdout:0/520: dread - d6/da/d2b/d60/faf zero size 2024-04-24T22:45:31.184 INFO:tasks.workunit.client.1.smithi192.stdout:5/391: write d0/d13/d19/d64/d4f/d52/f61 [787751,112425] 0 2024-04-24T22:45:31.185 INFO:tasks.workunit.client.1.smithi192.stdout:6/386: link d0/d8/d29/d4c/l85 d0/d8/d29/d5a/d63/l87 0 2024-04-24T22:45:31.186 INFO:tasks.workunit.client.1.smithi192.stdout:3/439: write d2/f29 [88856,62025] 0 2024-04-24T22:45:31.190 INFO:tasks.workunit.client.1.smithi192.stdout:8/463: creat db/dc/f8e x:0 0 0 2024-04-24T22:45:31.190 INFO:tasks.workunit.client.1.smithi192.stdout:8/464: stat db/dc/d22 0 2024-04-24T22:45:31.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/518: unlink d0/d4/de/d95/fa9 0 2024-04-24T22:45:31.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/598: dread da/d18/d47/d4d/f5a [0,4194304] 0 2024-04-24T22:45:31.197 INFO:tasks.workunit.client.1.smithi192.stdout:7/535: mkdir d2/db/d21/d42/d67/da8 0 2024-04-24T22:45:31.197 INFO:tasks.workunit.client.1.smithi192.stdout:8/465: dwrite db/f5a [0,4194304] 0 2024-04-24T22:45:31.200 INFO:tasks.workunit.client.1.smithi192.stdout:6/387: fsync d0/d8/f34 0 2024-04-24T22:45:31.202 INFO:tasks.workunit.client.1.smithi192.stdout:6/388: read - d0/f7f zero size 2024-04-24T22:45:31.206 INFO:tasks.workunit.client.1.smithi192.stdout:5/392: dread d0/d13/d19/f1a [0,4194304] 0 2024-04-24T22:45:31.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/440: rename d2/d4/d12/d39/d3a to d2/d4/d4e/d8a 0 2024-04-24T22:45:31.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/393: fdatasync d0/d13/d3b/d2f/f46 0 2024-04-24T22:45:31.209 INFO:tasks.workunit.client.0.smithi063.stdout:9/591: write d6/d8/db/d6b/f46 [1009600,84023] 0 2024-04-24T22:45:31.211 INFO:tasks.workunit.client.1.smithi192.stdout:2/547: creat d4/d9/d10/d6c/d6f/da8/fac x:0 0 0 2024-04-24T22:45:31.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/519: mknod d0/d4/de/d6f/d2e/cb4 0 2024-04-24T22:45:31.213 INFO:tasks.workunit.client.1.smithi192.stdout:7/536: mkdir d2/db/d2a/d57/da9 0 2024-04-24T22:45:31.214 INFO:tasks.workunit.client.0.smithi063.stdout:8/588: write d1/d21/f61 [1953650,66227] 0 2024-04-24T22:45:31.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/466: mkdir db/dc/d22/d31/d33/d8f 0 2024-04-24T22:45:31.220 INFO:tasks.workunit.client.1.smithi192.stdout:8/467: dread db/f65 [0,4194304] 0 2024-04-24T22:45:31.228 INFO:tasks.workunit.client.1.smithi192.stdout:5/394: rename d0/d13/d19/d64/d5f/d66/f68 to d0/d13/d19/d64/d4f/d6c/f81 0 2024-04-24T22:45:31.230 INFO:tasks.workunit.client.1.smithi192.stdout:3/441: dwrite d2/f33 [0,4194304] 0 2024-04-24T22:45:31.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/537: rmdir d2/db/d21/d42/d67/d9b/d6e/d77 39 2024-04-24T22:45:31.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/442: write d2/d4/d12/f88 [884041,50231] 0 2024-04-24T22:45:31.237 INFO:tasks.workunit.client.1.smithi192.stdout:7/538: dread d2/db/d21/d42/f8c [0,4194304] 0 2024-04-24T22:45:31.239 INFO:tasks.workunit.client.1.smithi192.stdout:8/468: mknod db/dc/d22/d31/d33/d81/d8a/c90 0 2024-04-24T22:45:31.241 INFO:tasks.workunit.client.0.smithi063.stdout:3/599: symlink da/d39/d94/db3/lce 0 2024-04-24T22:45:31.242 INFO:tasks.workunit.client.1.smithi192.stdout:3/443: dread d2/d4/d12/f1c [0,4194304] 0 2024-04-24T22:45:31.242 INFO:tasks.workunit.client.1.smithi192.stdout:8/469: chown db/dc/l3a 2700 1 2024-04-24T22:45:31.242 INFO:tasks.workunit.client.1.smithi192.stdout:6/389: truncate d0/d8/d29/f30 806035 0 2024-04-24T22:45:31.242 INFO:tasks.workunit.client.0.smithi063.stdout:9/592: write d6/d8/db/d6b/d4f/d59/f76 [1032059,77942] 0 2024-04-24T22:45:31.245 INFO:tasks.workunit.client.0.smithi063.stdout:6/600: dwrite db/fc6 [0,4194304] 0 2024-04-24T22:45:31.246 INFO:tasks.workunit.client.0.smithi063.stdout:6/601: read - db/d1d/fc9 zero size 2024-04-24T22:45:31.246 INFO:tasks.workunit.client.1.smithi192.stdout:8/470: dwrite db/d43/f4e [0,4194304] 0 2024-04-24T22:45:31.248 INFO:tasks.workunit.client.1.smithi192.stdout:2/548: mknod d4/d9/cad 0 2024-04-24T22:45:31.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/540: dwrite d2/f5 [4194304,4194304] 0 2024-04-24T22:45:31.257 INFO:tasks.workunit.client.1.smithi192.stdout:5/395: mknod d0/d13/d19/d64/d4f/d6c/c82 0 2024-04-24T22:45:31.263 INFO:tasks.workunit.client.0.smithi063.stdout:2/656: sync 2024-04-24T22:45:31.263 INFO:tasks.workunit.client.0.smithi063.stdout:5/564: sync 2024-04-24T22:45:31.264 INFO:tasks.workunit.client.0.smithi063.stdout:5/565: chown d5/d1d/dc0/dbb/f55 294919 1 2024-04-24T22:45:31.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/471: dread - db/dc/d22/d31/f45 zero size 2024-04-24T22:45:31.270 INFO:tasks.workunit.client.1.smithi192.stdout:2/549: rmdir d4/d9/d10/d6c/d6f/da8 39 2024-04-24T22:45:31.270 INFO:tasks.workunit.client.1.smithi192.stdout:3/444: dwrite d2/f76 [0,4194304] 0 2024-04-24T22:45:31.272 INFO:tasks.workunit.client.1.smithi192.stdout:3/445: fdatasync d2/d4/d7/f2e 0 2024-04-24T22:45:31.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/396: mkdir d0/d13/d19/d83 0 2024-04-24T22:45:31.277 INFO:tasks.workunit.client.1.smithi192.stdout:6/390: mknod d0/d8/c88 0 2024-04-24T22:45:31.282 INFO:tasks.workunit.client.0.smithi063.stdout:5/566: dread d5/f9 [0,4194304] 0 2024-04-24T22:45:31.284 INFO:tasks.workunit.client.1.smithi192.stdout:8/472: creat db/dc/d29/f91 x:0 0 0 2024-04-24T22:45:31.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/473: write db/dc/f1c [3142524,104571] 0 2024-04-24T22:45:31.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/567: dread d5/d11/d18/f19 [0,4194304] 0 2024-04-24T22:45:31.293 INFO:tasks.workunit.client.1.smithi192.stdout:8/474: dwrite db/dc/f24 [0,4194304] 0 2024-04-24T22:45:31.296 INFO:tasks.workunit.client.0.smithi063.stdout:3/600: rmdir da/d18 39 2024-04-24T22:45:31.298 INFO:tasks.workunit.client.1.smithi192.stdout:6/391: rename d0/d8/d18/d2e/l37 to d0/d8/d29/d58/l89 0 2024-04-24T22:45:31.303 INFO:tasks.workunit.client.1.smithi192.stdout:2/550: truncate d4/d9/d10/d6c/d8c/f93 2856224 0 2024-04-24T22:45:31.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/593: rmdir d6/d8/db/d93/da0 39 2024-04-24T22:45:31.306 INFO:tasks.workunit.client.0.smithi063.stdout:0/521: sync 2024-04-24T22:45:31.306 INFO:tasks.workunit.client.0.smithi063.stdout:0/522: stat d6/dd/c1b 0 2024-04-24T22:45:31.306 INFO:tasks.workunit.client.1.smithi192.stdout:6/392: dwrite d0/d8/d29/d5a/f60 [4194304,4194304] 0 2024-04-24T22:45:31.311 INFO:tasks.workunit.client.1.smithi192.stdout:7/539: write d2/db/d36/f64 [723696,67275] 0 2024-04-24T22:45:31.315 INFO:tasks.workunit.client.0.smithi063.stdout:1/657: getdents d3 0 2024-04-24T22:45:31.317 INFO:tasks.workunit.client.1.smithi192.stdout:6/393: mknod d0/d8/d29/d58/c8a 0 2024-04-24T22:45:31.319 INFO:tasks.workunit.client.1.smithi192.stdout:4/384: sync 2024-04-24T22:45:31.322 INFO:tasks.workunit.client.1.smithi192.stdout:7/540: dread d2/db/d21/d42/f8c [0,4194304] 0 2024-04-24T22:45:31.322 INFO:tasks.workunit.client.0.smithi063.stdout:5/568: mkdir d5/d24/d49/d3b/dc5 0 2024-04-24T22:45:31.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/541: write f0 [4334677,65077] 0 2024-04-24T22:45:31.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/551: unlink d4/d29/d60/c61 0 2024-04-24T22:45:31.324 INFO:tasks.workunit.client.1.smithi192.stdout:7/542: stat d2/db/d21/d42 0 2024-04-24T22:45:31.325 INFO:tasks.workunit.client.0.smithi063.stdout:8/589: write d1/d22/d33/da6/d54/f71 [1676670,23141] 0 2024-04-24T22:45:31.326 INFO:tasks.workunit.client.0.smithi063.stdout:3/601: chown da/d18/d47/d7b/c9f 3026 1 2024-04-24T22:45:31.327 INFO:tasks.workunit.client.0.smithi063.stdout:5/569: dread d5/d1d/dc0/dbb/f89 [0,4194304] 0 2024-04-24T22:45:31.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/543: truncate d2/f91 198934 0 2024-04-24T22:45:31.328 INFO:tasks.workunit.client.1.smithi192.stdout:4/385: dread d2/d9/d25/d2e/d31/d36/d5b/f6e [0,4194304] 0 2024-04-24T22:45:31.329 INFO:tasks.workunit.client.1.smithi192.stdout:3/446: dread d2/d4/d7/f9 [0,4194304] 0 2024-04-24T22:45:31.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/520: truncate d0/d4/de/f12 1505891 0 2024-04-24T22:45:31.331 INFO:tasks.workunit.client.0.smithi063.stdout:6/602: mknod db/cca 0 2024-04-24T22:45:31.335 INFO:tasks.workunit.client.0.smithi063.stdout:9/594: rename d6/d8/db/d6b/d4f/dad/d48/d94/fa6 to d6/d2f/d42/d81/fc2 0 2024-04-24T22:45:31.336 INFO:tasks.workunit.client.0.smithi063.stdout:9/595: readlink d6/d8/db/d6b/l5f 0 2024-04-24T22:45:31.340 INFO:tasks.workunit.client.0.smithi063.stdout:2/657: link d0/dd/f99 d0/d2/d2d/d31/d89/d7/d86/da0/d94/fc0 0 2024-04-24T22:45:31.341 INFO:tasks.workunit.client.1.smithi192.stdout:8/475: dwrite db/d5d/f7e [0,4194304] 0 2024-04-24T22:45:31.342 INFO:tasks.workunit.client.0.smithi063.stdout:1/658: dread d3/db5/d21/f69 [0,4194304] 0 2024-04-24T22:45:31.342 INFO:tasks.workunit.client.1.smithi192.stdout:8/476: read - db/d5d/f84 zero size 2024-04-24T22:45:31.344 INFO:tasks.workunit.client.0.smithi063.stdout:1/659: write d3/da/da2/dba/d39/f66 [1569565,128919] 0 2024-04-24T22:45:31.349 INFO:tasks.workunit.client.0.smithi063.stdout:2/658: dwrite d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:31.351 INFO:tasks.workunit.client.1.smithi192.stdout:3/447: symlink d2/d27/d28/d38/l8b 0 2024-04-24T22:45:31.351 INFO:tasks.workunit.client.1.smithi192.stdout:6/394: dwrite d0/d8/d29/f3b [0,4194304] 0 2024-04-24T22:45:31.355 INFO:tasks.workunit.client.0.smithi063.stdout:5/570: rmdir d5/d1d/d7d 39 2024-04-24T22:45:31.355 INFO:tasks.workunit.client.0.smithi063.stdout:9/596: dread d6/d8/db/d1c/f57 [0,4194304] 0 2024-04-24T22:45:31.356 INFO:tasks.workunit.client.0.smithi063.stdout:5/571: readlink d5/d24/d49/d3b/lc3 0 2024-04-24T22:45:31.360 INFO:tasks.workunit.client.0.smithi063.stdout:0/523: mknod d6/d64/cbb 0 2024-04-24T22:45:31.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/552: rename d4/d9/c74 to d4/d9/d9f/cae 0 2024-04-24T22:45:31.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/477: fdatasync f6 0 2024-04-24T22:45:31.368 INFO:tasks.workunit.client.1.smithi192.stdout:4/386: creat d2/d9/d25/d2e/d31/d36/d70/f85 x:0 0 0 2024-04-24T22:45:31.369 INFO:tasks.workunit.client.0.smithi063.stdout:4/541: creat d2/d8/d36/faf x:0 0 0 2024-04-24T22:45:31.369 INFO:tasks.workunit.client.1.smithi192.stdout:4/387: chown d2/d9/d25/d2e/d31/d36/d5b/d28/f2b 924 1 2024-04-24T22:45:31.370 INFO:tasks.workunit.client.1.smithi192.stdout:6/395: truncate d0/f19 1432996 0 2024-04-24T22:45:31.371 INFO:tasks.workunit.client.1.smithi192.stdout:4/388: write d2/db/f3d [1898363,101117] 0 2024-04-24T22:45:31.372 INFO:tasks.workunit.client.1.smithi192.stdout:6/396: write d0/d8/d18/f3f [1106052,20341] 0 2024-04-24T22:45:31.372 INFO:tasks.workunit.client.0.smithi063.stdout:1/660: symlink d3/db5/d2b/d79/d96/ld6 0 2024-04-24T22:45:31.372 INFO:tasks.workunit.client.0.smithi063.stdout:8/590: creat d1/d22/d33/dbd/fbe x:0 0 0 2024-04-24T22:45:31.374 INFO:tasks.workunit.client.0.smithi063.stdout:8/591: dread d1/d11/f9f [0,4194304] 0 2024-04-24T22:45:31.388 INFO:tasks.workunit.client.1.smithi192.stdout:6/397: chown d0/d8/d29/d5a/d63 983446 1 2024-04-24T22:45:31.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/544: rename d2/db/f50 to d2/db/d21/d42/d67/d9b/d6e/da2/faa 0 2024-04-24T22:45:31.389 INFO:tasks.workunit.client.0.smithi063.stdout:3/602: mknod da/d39/d94/dbc/ccf 0 2024-04-24T22:45:31.389 INFO:tasks.workunit.client.1.smithi192.stdout:7/545: chown d2/db/fa6 3897732 1 2024-04-24T22:45:31.390 INFO:tasks.workunit.client.1.smithi192.stdout:2/553: write d4/d9/d10/f58 [1685749,71967] 0 2024-04-24T22:45:31.392 INFO:tasks.workunit.client.0.smithi063.stdout:9/597: creat d6/d8/db/d1c/d78/d9a/daa/fc3 x:0 0 0 2024-04-24T22:45:31.393 INFO:tasks.workunit.client.0.smithi063.stdout:9/598: chown d6/d8/db/fc1 61069 1 2024-04-24T22:45:31.396 INFO:tasks.workunit.client.1.smithi192.stdout:6/398: dwrite d0/d8/d22/f27 [4194304,4194304] 0 2024-04-24T22:45:31.397 INFO:tasks.workunit.client.1.smithi192.stdout:6/399: chown d0/d8/f14 2988776 1 2024-04-24T22:45:31.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/448: dread d2/d4/d12/d1a/d1d/d55/f24 [0,4194304] 0 2024-04-24T22:45:31.399 INFO:tasks.workunit.client.0.smithi063.stdout:9/599: dwrite d6/d8/d1f/d20/d2a/d4b/d5e/d8e/f90 [0,4194304] 0 2024-04-24T22:45:31.400 INFO:tasks.workunit.client.1.smithi192.stdout:3/449: truncate d2/d4/d12/d1a/d1d/d55/d5f/f80 580108 0 2024-04-24T22:45:31.401 INFO:tasks.workunit.client.0.smithi063.stdout:0/524: read d6/da/f14 [3217124,19809] 0 2024-04-24T22:45:31.402 INFO:tasks.workunit.client.1.smithi192.stdout:4/389: creat d2/d5c/f86 x:0 0 0 2024-04-24T22:45:31.403 INFO:tasks.workunit.client.0.smithi063.stdout:7/521: mkdir d0/db5 0 2024-04-24T22:45:31.406 INFO:tasks.workunit.client.0.smithi063.stdout:0/525: write d6/d64/d21/fb3 [762996,25769] 0 2024-04-24T22:45:31.410 INFO:tasks.workunit.client.0.smithi063.stdout:1/661: rename d3/db5/l4e to d3/ld7 0 2024-04-24T22:45:31.412 INFO:tasks.workunit.client.0.smithi063.stdout:9/600: dwrite d6/d8/db/d6b/d4f/d59/f76 [0,4194304] 0 2024-04-24T22:45:31.414 INFO:tasks.workunit.client.1.smithi192.stdout:8/478: mkdir db/d43/d76/d92 0 2024-04-24T22:45:31.416 INFO:tasks.workunit.client.0.smithi063.stdout:2/659: dread d0/d2/f47 [0,4194304] 0 2024-04-24T22:45:31.416 INFO:tasks.workunit.client.0.smithi063.stdout:8/592: mknod d1/d22/d33/cbf 0 2024-04-24T22:45:31.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/522: dwrite d0/fa [0,4194304] 0 2024-04-24T22:45:31.421 INFO:tasks.workunit.client.0.smithi063.stdout:5/572: write d5/d1d/dc0/dbb/f50 [2068797,100444] 0 2024-04-24T22:45:31.422 INFO:tasks.workunit.client.1.smithi192.stdout:6/400: read d0/d8/fe [489333,62895] 0 2024-04-24T22:45:31.427 INFO:tasks.workunit.client.1.smithi192.stdout:7/546: creat d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fab x:0 0 0 2024-04-24T22:45:31.428 INFO:tasks.workunit.client.0.smithi063.stdout:7/523: dwrite d0/fa2 [0,4194304] 0 2024-04-24T22:45:31.428 INFO:tasks.workunit.client.1.smithi192.stdout:7/547: stat d2/db/d21/d42/d67/d9b/f99 0 2024-04-24T22:45:31.429 INFO:tasks.workunit.client.1.smithi192.stdout:8/479: mknod db/d5d/d74/c93 0 2024-04-24T22:45:31.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/524: chown d0/d4/de/d6f/d2e 868282 1 2024-04-24T22:45:31.430 INFO:tasks.workunit.client.1.smithi192.stdout:8/480: dread - db/dc/f6e zero size 2024-04-24T22:45:31.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/542: write d2/f35 [508492,34365] 0 2024-04-24T22:45:31.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/554: symlink d4/d9/d10/da7/laf 0 2024-04-24T22:45:31.436 INFO:tasks.workunit.client.1.smithi192.stdout:3/450: truncate d2/d4/d7/f2e 4098265 0 2024-04-24T22:45:31.440 INFO:tasks.workunit.client.0.smithi063.stdout:0/526: dread d6/d64/da0/f5f [0,4194304] 0 2024-04-24T22:45:31.441 INFO:tasks.workunit.client.0.smithi063.stdout:0/527: stat d6/d64/d21/d25/f7b 0 2024-04-24T22:45:31.443 INFO:tasks.workunit.client.1.smithi192.stdout:4/390: dread d2/d9/f3f [0,4194304] 0 2024-04-24T22:45:31.445 INFO:tasks.workunit.client.0.smithi063.stdout:9/601: rename d6/d8/db/d6b/d4f/d59/l7c to d6/d8/d8a/lc4 0 2024-04-24T22:45:31.445 INFO:tasks.workunit.client.1.smithi192.stdout:3/451: dwrite d2/d62/f7d [0,4194304] 0 2024-04-24T22:45:31.446 INFO:tasks.workunit.client.1.smithi192.stdout:8/481: creat db/d43/d76/f94 x:0 0 0 2024-04-24T22:45:31.447 INFO:tasks.workunit.client.1.smithi192.stdout:8/482: write db/dc/d29/f4f [3559890,72666] 0 2024-04-24T22:45:31.450 INFO:tasks.workunit.client.1.smithi192.stdout:8/483: read db/dc/f18 [2637800,81048] 0 2024-04-24T22:45:31.450 INFO:tasks.workunit.client.0.smithi063.stdout:9/602: dwrite d6/d8/db/d6b/f46 [0,4194304] 0 2024-04-24T22:45:31.450 INFO:tasks.workunit.client.0.smithi063.stdout:3/603: mknod da/d18/d9e/cd0 0 2024-04-24T22:45:31.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/573: readlink d5/la 0 2024-04-24T22:45:31.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/525: mkdir d0/d4/de/d6f/d2e/d76/db6 0 2024-04-24T22:45:31.455 INFO:tasks.workunit.client.0.smithi063.stdout:4/543: creat d2/d80/fb0 x:0 0 0 2024-04-24T22:45:31.457 INFO:tasks.workunit.client.1.smithi192.stdout:7/548: unlink d2/db/d2a/c90 0 2024-04-24T22:45:31.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/549: stat d2/db/d21/d42/f93 0 2024-04-24T22:45:31.461 INFO:tasks.workunit.client.0.smithi063.stdout:8/593: mkdir d1/dc0 0 2024-04-24T22:45:31.463 INFO:tasks.workunit.client.1.smithi192.stdout:3/452: creat d2/d4/d7/d4f/f8c x:0 0 0 2024-04-24T22:45:31.464 INFO:tasks.workunit.client.0.smithi063.stdout:1/662: dwrite d3/db5/d21/db4/fd4 [0,4194304] 0 2024-04-24T22:45:31.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/663: chown d3 97760 1 2024-04-24T22:45:31.470 INFO:tasks.workunit.client.0.smithi063.stdout:2/660: link d0/dd/d72/d77/d87/f9c d0/d2/d2d/d31/d89/d7/d86/da0/d94/fc1 0 2024-04-24T22:45:31.472 INFO:tasks.workunit.client.1.smithi192.stdout:8/484: mknod db/dc/d29/c95 0 2024-04-24T22:45:31.473 INFO:tasks.workunit.client.0.smithi063.stdout:7/526: dread d0/d4/de/f25 [0,4194304] 0 2024-04-24T22:45:31.475 INFO:tasks.workunit.client.1.smithi192.stdout:6/401: rmdir d0/d8/d29/d4c/d7b 0 2024-04-24T22:45:31.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/402: chown d0/d8/f14 170996 1 2024-04-24T22:45:31.477 INFO:tasks.workunit.client.1.smithi192.stdout:6/403: chown d0/l11 12 1 2024-04-24T22:45:31.480 INFO:tasks.workunit.client.1.smithi192.stdout:6/404: read d0/d8/d3d/f55 [97951,66498] 0 2024-04-24T22:45:31.482 INFO:tasks.workunit.client.1.smithi192.stdout:3/453: creat d2/d4/d7/d4b/f8d x:0 0 0 2024-04-24T22:45:31.485 INFO:tasks.workunit.client.0.smithi063.stdout:4/544: symlink d2/d80/d81/d90/lb1 0 2024-04-24T22:45:31.488 INFO:tasks.workunit.client.1.smithi192.stdout:2/555: link d4/d9/d10/d6c/d8c/f8f d4/d9/d10/d6c/d6f/da8/fb0 0 2024-04-24T22:45:31.490 INFO:tasks.workunit.client.1.smithi192.stdout:8/485: mkdir db/d5d/d74/d96 0 2024-04-24T22:45:31.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/545: dwrite d2/fb [8388608,4194304] 0 2024-04-24T22:45:31.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/528: unlink d6/da/lf 0 2024-04-24T22:45:31.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/664: symlink d3/db5/d21/d9f/daa/ld8 0 2024-04-24T22:45:31.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/603: symlink d6/d8/d1f/d20/d22/lc5 0 2024-04-24T22:45:31.501 INFO:tasks.workunit.client.0.smithi063.stdout:2/661: mknod d0/d2/d2d/d31/d89/d7/d86/d8e/cc2 0 2024-04-24T22:45:31.504 INFO:tasks.workunit.client.1.smithi192.stdout:6/405: mknod d0/d8/d29/d5a/c8b 0 2024-04-24T22:45:31.509 INFO:tasks.workunit.client.1.smithi192.stdout:3/454: symlink d2/d4/d12/d1a/d1d/d71/l8e 0 2024-04-24T22:45:31.511 INFO:tasks.workunit.client.1.smithi192.stdout:4/391: truncate d2/db/d10/f4f 1266856 0 2024-04-24T22:45:31.514 INFO:tasks.workunit.client.0.smithi063.stdout:3/604: truncate da/d48/f5c 7313115 0 2024-04-24T22:45:31.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/603: link db/cc db/d1d/d2d/d69/d82/d96/ccb 0 2024-04-24T22:45:31.516 INFO:tasks.workunit.client.0.smithi063.stdout:6/604: read db/f1a [3758240,47547] 0 2024-04-24T22:45:31.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/605: stat db/d1d/d20/d84/lb2 0 2024-04-24T22:45:31.519 INFO:tasks.workunit.client.0.smithi063.stdout:5/574: creat d5/d11/d23/d2d/d62/d81/da4/da6/fc6 x:0 0 0 2024-04-24T22:45:31.520 INFO:tasks.workunit.client.0.smithi063.stdout:3/605: dwrite da/d18/f27 [0,4194304] 0 2024-04-24T22:45:31.521 INFO:tasks.workunit.client.1.smithi192.stdout:6/406: fsync d0/d8/f6f 0 2024-04-24T22:45:31.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/594: mkdir d1/d11/d32/d38/db7/dc1 0 2024-04-24T22:45:31.523 INFO:tasks.workunit.client.0.smithi063.stdout:3/606: fdatasync da/d18/d47/d4d/fae 0 2024-04-24T22:45:31.527 INFO:tasks.workunit.client.0.smithi063.stdout:6/606: dwrite db/d1d/d20/fa1 [0,4194304] 0 2024-04-24T22:45:31.529 INFO:tasks.workunit.client.0.smithi063.stdout:3/607: write da/d18/d47/d7b/fc5 [1011624,53690] 0 2024-04-24T22:45:31.529 INFO:tasks.workunit.client.1.smithi192.stdout:4/392: unlink d2/db/l1b 0 2024-04-24T22:45:31.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/529: truncate d6/dd/f12 487799 0 2024-04-24T22:45:31.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/393: chown d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/c42 976504 1 2024-04-24T22:45:31.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/575: dwrite d5/d24/f3d [0,4194304] 0 2024-04-24T22:45:31.534 INFO:tasks.workunit.client.1.smithi192.stdout:8/486: mkdir db/d5d/d74/d96/d97 0 2024-04-24T22:45:31.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/595: dwrite d1/d22/d33/da6/fa2 [0,4194304] 0 2024-04-24T22:45:31.540 INFO:tasks.workunit.client.1.smithi192.stdout:3/455: write d2/d4/d7/d4f/f5a [1420840,71311] 0 2024-04-24T22:45:31.541 INFO:tasks.workunit.client.0.smithi063.stdout:8/596: dread d1/d11/d8b/f8d [0,4194304] 0 2024-04-24T22:45:31.543 INFO:tasks.workunit.client.0.smithi063.stdout:8/597: write d1/d11/d32/fb2 [294796,77747] 0 2024-04-24T22:45:31.545 INFO:tasks.workunit.client.1.smithi192.stdout:6/407: mknod d0/d8/d18/d2e/c8c 0 2024-04-24T22:45:31.546 INFO:tasks.workunit.client.1.smithi192.stdout:8/487: dwrite db/dc/f24 [4194304,4194304] 0 2024-04-24T22:45:31.553 INFO:tasks.workunit.client.1.smithi192.stdout:2/556: rmdir d4/d29/da9 0 2024-04-24T22:45:31.554 INFO:tasks.workunit.client.1.smithi192.stdout:4/394: dread - d2/d9/d25/d2e/d31/f4a zero size 2024-04-24T22:45:31.558 INFO:tasks.workunit.client.0.smithi063.stdout:4/546: rename d2/d80/l93 to d2/d8/d36/d3a/lb2 0 2024-04-24T22:45:31.560 INFO:tasks.workunit.client.0.smithi063.stdout:6/607: write f3 [1858747,113471] 0 2024-04-24T22:45:31.561 INFO:tasks.workunit.client.1.smithi192.stdout:3/456: creat d2/d4/d4e/f8f x:0 0 0 2024-04-24T22:45:31.564 INFO:tasks.workunit.client.0.smithi063.stdout:1/665: readlink d3/da/l4b 0 2024-04-24T22:45:31.566 INFO:tasks.workunit.client.0.smithi063.stdout:3/608: symlink da/d18/d47/d4d/ld1 0 2024-04-24T22:45:31.567 INFO:tasks.workunit.client.1.smithi192.stdout:6/408: rmdir d0/d8/d18 39 2024-04-24T22:45:31.568 INFO:tasks.workunit.client.0.smithi063.stdout:6/608: read db/d1c/f89 [1797960,126103] 0 2024-04-24T22:45:31.568 INFO:tasks.workunit.client.1.smithi192.stdout:8/488: creat db/d75/f98 x:0 0 0 2024-04-24T22:45:31.570 INFO:tasks.workunit.client.1.smithi192.stdout:4/395: mkdir d2/d9/d25/d2e/d31/d36/d70/d87 0 2024-04-24T22:45:31.571 INFO:tasks.workunit.client.1.smithi192.stdout:4/396: readlink d2/d9/l2a 0 2024-04-24T22:45:31.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/530: creat d6/d64/d24/fbc x:0 0 0 2024-04-24T22:45:31.572 INFO:tasks.workunit.client.1.smithi192.stdout:3/457: mknod d2/d4/d7/c90 0 2024-04-24T22:45:31.574 INFO:tasks.workunit.client.0.smithi063.stdout:3/609: dwrite da/d18/d3a/fb7 [0,4194304] 0 2024-04-24T22:45:31.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/576: dread - d5/d1d/d7d/d67/f78 zero size 2024-04-24T22:45:31.574 INFO:tasks.workunit.client.1.smithi192.stdout:3/458: fdatasync d2/d4/d7/d4f/f5a 0 2024-04-24T22:45:31.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/489: creat db/d43/d71/f99 x:0 0 0 2024-04-24T22:45:31.576 INFO:tasks.workunit.client.0.smithi063.stdout:3/610: write da/d18/d9e/fc4 [187103,117320] 0 2024-04-24T22:45:31.578 INFO:tasks.workunit.client.0.smithi063.stdout:8/598: rmdir d1/d22/d33/da6/d54 39 2024-04-24T22:45:31.578 INFO:tasks.workunit.client.0.smithi063.stdout:7/527: write d0/d4/f19 [2763737,81667] 0 2024-04-24T22:45:31.584 INFO:tasks.workunit.client.1.smithi192.stdout:6/409: creat d0/d8/d18/d2e/f8d x:0 0 0 2024-04-24T22:45:31.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/410: fsync d0/d8/f17 0 2024-04-24T22:45:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:7/528: dwrite d0/d4/de/d6f/d5c/d8b/fae [0,4194304] 0 2024-04-24T22:45:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:2/662: rename d0/dd/d72/d77/d87/fa3 to d0/d2/d2d/d31/d89/d7/d86/da0/d94/fc3 0 2024-04-24T22:45:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:1/666: symlink d3/da/d34/d68/ld9 0 2024-04-24T22:45:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:7/529: dwrite d0/d4/de/d17/d28/f64 [0,4194304] 0 2024-04-24T22:45:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:9/604: rmdir d6/d2f/d42/d81/dba 0 2024-04-24T22:45:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:4/547: dwrite d2/d80/f8a [0,4194304] 0 2024-04-24T22:45:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:6/609: creat db/d1d/d20/d29/fcc x:0 0 0 2024-04-24T22:45:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:6/610: fdatasync db/d1c/fa7 0 2024-04-24T22:45:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:6/611: dread - db/d1d/d20/d3e/f45 zero size 2024-04-24T22:45:31.605 INFO:tasks.workunit.client.0.smithi063.stdout:6/612: write db/d1d/d20/fa1 [3454914,72257] 0 2024-04-24T22:45:31.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/490: mkdir db/dc/d9a 0 2024-04-24T22:45:31.608 INFO:tasks.workunit.client.1.smithi192.stdout:2/557: dwrite d4/f13 [0,4194304] 0 2024-04-24T22:45:31.609 INFO:tasks.workunit.client.1.smithi192.stdout:4/397: rmdir d2/d9/d25/d2e/d31/d36/d70/d87 0 2024-04-24T22:45:31.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/558: write d4/d9/d37/f8a [1565840,54734] 0 2024-04-24T22:45:31.611 INFO:tasks.workunit.client.0.smithi063.stdout:3/611: creat da/d39/d94/db3/fd2 x:0 0 0 2024-04-24T22:45:31.612 INFO:tasks.workunit.client.1.smithi192.stdout:2/559: stat d4/d9/d10/d6c/d6f/da8/fac 0 2024-04-24T22:45:31.614 INFO:tasks.workunit.client.1.smithi192.stdout:6/411: creat d0/d8/d3d/d3e/d7e/f8e x:0 0 0 2024-04-24T22:45:31.614 INFO:tasks.workunit.client.1.smithi192.stdout:3/459: write d2/d4/f69 [715272,68434] 0 2024-04-24T22:45:31.615 INFO:tasks.workunit.client.1.smithi192.stdout:3/460: stat d2/d4/d7/d4f/f5a 0 2024-04-24T22:45:31.615 INFO:tasks.workunit.client.1.smithi192.stdout:6/412: write d0/d8/d29/d5a/d63/f81 [1382307,13357] 0 2024-04-24T22:45:31.616 INFO:tasks.workunit.client.1.smithi192.stdout:8/491: unlink db/d5d/l6b 0 2024-04-24T22:45:31.617 INFO:tasks.workunit.client.1.smithi192.stdout:6/413: stat d0/d8/d3d 0 2024-04-24T22:45:31.620 INFO:tasks.workunit.client.1.smithi192.stdout:2/560: mknod d4/d9/d10/d6c/d95/cb1 0 2024-04-24T22:45:31.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/461: fsync d2/d4/d7/d4b/f50 0 2024-04-24T22:45:31.626 INFO:tasks.workunit.client.1.smithi192.stdout:8/492: unlink db/dc/d29/f91 0 2024-04-24T22:45:31.627 INFO:tasks.workunit.client.1.smithi192.stdout:8/493: fsync db/dc/d22/f5e 0 2024-04-24T22:45:31.627 INFO:tasks.workunit.client.0.smithi063.stdout:9/605: dread d6/d8/d1f/d20/d22/d31/f33 [0,4194304] 0 2024-04-24T22:45:31.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/398: mknod d2/d9/d25/d2e/d31/c88 0 2024-04-24T22:45:31.635 INFO:tasks.workunit.client.0.smithi063.stdout:3/612: dread da/d18/f61 [0,4194304] 0 2024-04-24T22:45:31.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/530: rename d0/d4/de/d6f/d2e/c5f to d0/d4/de/d17/d35/d69/da0/cb7 0 2024-04-24T22:45:31.641 INFO:tasks.workunit.client.1.smithi192.stdout:0/491: sync 2024-04-24T22:45:31.642 INFO:tasks.workunit.client.1.smithi192.stdout:8/494: rename db/d43/d7c/d85/f8c to db/d43/d76/d92/f9b 0 2024-04-24T22:45:31.644 INFO:tasks.workunit.client.1.smithi192.stdout:8/495: write db/dc/d22/d31/d33/f6d [483667,85078] 0 2024-04-24T22:45:31.644 INFO:tasks.workunit.client.0.smithi063.stdout:0/531: creat d6/d64/d21/d9c/fbd x:0 0 0 2024-04-24T22:45:31.645 INFO:tasks.workunit.client.1.smithi192.stdout:8/496: fdatasync db/d43/d71/f99 0 2024-04-24T22:45:31.646 INFO:tasks.workunit.client.1.smithi192.stdout:4/399: mknod d2/d9/c89 0 2024-04-24T22:45:31.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/414: write d0/d8/f33 [1740159,40047] 0 2024-04-24T22:45:31.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/667: write d3/da/da2/dba/d11/d5e/d6f/faf [1044830,58825] 0 2024-04-24T22:45:31.650 INFO:tasks.workunit.client.0.smithi063.stdout:5/577: mkdir d5/d1d/dc7 0 2024-04-24T22:45:31.653 INFO:tasks.workunit.client.1.smithi192.stdout:2/561: dwrite d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:31.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/599: symlink d1/d22/d33/da6/d54/d79/lc2 0 2024-04-24T22:45:31.655 INFO:tasks.workunit.client.0.smithi063.stdout:8/600: write d1/d11/f93 [378784,5687] 0 2024-04-24T22:45:31.656 INFO:tasks.workunit.client.0.smithi063.stdout:5/578: dwrite f0 [0,4194304] 0 2024-04-24T22:45:31.657 INFO:tasks.workunit.client.0.smithi063.stdout:5/579: readlink d5/d11/l94 0 2024-04-24T22:45:31.658 INFO:tasks.workunit.client.0.smithi063.stdout:5/580: stat d5/d11/d76/d92 0 2024-04-24T22:45:31.662 INFO:tasks.workunit.client.0.smithi063.stdout:9/606: dread d6/d8/d1f/d20/d22/f47 [0,4194304] 0 2024-04-24T22:45:31.666 INFO:tasks.workunit.client.0.smithi063.stdout:7/531: creat d0/d4/de/d17/d28/fb8 x:0 0 0 2024-04-24T22:45:31.666 INFO:tasks.workunit.client.0.smithi063.stdout:0/532: creat d6/d64/d24/d42/fbe x:0 0 0 2024-04-24T22:45:31.667 INFO:tasks.workunit.client.0.smithi063.stdout:1/668: creat d3/da/da2/dba/d39/fda x:0 0 0 2024-04-24T22:45:31.668 INFO:tasks.workunit.client.0.smithi063.stdout:8/601: rmdir d1/d22/d33/da6/d54/d7b 39 2024-04-24T22:45:31.669 INFO:tasks.workunit.client.0.smithi063.stdout:0/533: chown d6/d64/d21/d25/d62/d75/f81 100391172 1 2024-04-24T22:45:31.669 INFO:tasks.workunit.client.0.smithi063.stdout:2/663: link d0/d2/d39/f65 d0/d2/d27/fc4 0 2024-04-24T22:45:31.671 INFO:tasks.workunit.client.0.smithi063.stdout:9/607: dread d6/d8/db/d6b/d4f/d59/f76 [0,4194304] 0 2024-04-24T22:45:31.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/613: symlink db/d85/db5/dc4/lcd 0 2024-04-24T22:45:31.675 INFO:tasks.workunit.client.0.smithi063.stdout:1/669: rmdir d3/da/d34/d68 39 2024-04-24T22:45:31.676 INFO:tasks.workunit.client.0.smithi063.stdout:5/581: dread d5/d24/d49/d3b/f48 [0,4194304] 0 2024-04-24T22:45:31.677 INFO:tasks.workunit.client.0.smithi063.stdout:8/602: unlink d1/d21/faa 0 2024-04-24T22:45:31.677 INFO:tasks.workunit.client.0.smithi063.stdout:0/534: dwrite d6/d74/f7e [0,4194304] 0 2024-04-24T22:45:31.678 INFO:tasks.workunit.client.0.smithi063.stdout:5/582: write d5/d24/d44/fbf [1003971,6219] 0 2024-04-24T22:45:31.679 INFO:tasks.workunit.client.0.smithi063.stdout:0/535: stat d6/d64/f3b 0 2024-04-24T22:45:31.680 INFO:tasks.workunit.client.0.smithi063.stdout:8/603: truncate d1/d11/d3c/f9e 367515 0 2024-04-24T22:45:31.681 INFO:tasks.workunit.client.0.smithi063.stdout:8/604: write d1/d11/d3c/f9e [1276351,71125] 0 2024-04-24T22:45:31.682 INFO:tasks.workunit.client.0.smithi063.stdout:9/608: stat d6/d2f/l73 0 2024-04-24T22:45:31.684 INFO:tasks.workunit.client.0.smithi063.stdout:1/670: mknod d3/da/da2/dba/d11/d5e/d9c/cdb 0 2024-04-24T22:45:31.685 INFO:tasks.workunit.client.0.smithi063.stdout:0/536: fsync d6/d64/d21/d25/d62/d75/f61 0 2024-04-24T22:45:31.685 INFO:tasks.workunit.client.0.smithi063.stdout:2/664: read d0/d2/d39/f65 [1907849,73905] 0 2024-04-24T22:45:31.689 INFO:tasks.workunit.client.1.smithi192.stdout:8/497: read db/dc/f18 [2124902,28476] 0 2024-04-24T22:45:31.690 INFO:tasks.workunit.client.0.smithi063.stdout:8/605: creat d1/d21/fc3 x:0 0 0 2024-04-24T22:45:31.690 INFO:tasks.workunit.client.1.smithi192.stdout:0/492: mkdir d3/d27/d28/d2e/d4a/da8 0 2024-04-24T22:45:31.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/415: readlink d0/l1a 0 2024-04-24T22:45:31.692 INFO:tasks.workunit.client.1.smithi192.stdout:9/550: sync 2024-04-24T22:45:31.692 INFO:tasks.workunit.client.1.smithi192.stdout:0/493: dread d3/d27/d28/d2e/d4a/d5c/f9a [0,4194304] 0 2024-04-24T22:45:31.693 INFO:tasks.workunit.client.1.smithi192.stdout:9/551: write d3/d5d/da1/f70 [574604,50405] 0 2024-04-24T22:45:31.695 INFO:tasks.workunit.client.1.smithi192.stdout:2/562: symlink d4/d9/d10/d6c/d8c/da5/lb2 0 2024-04-24T22:45:31.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/552: read d3/d24/f4e [1236087,62227] 0 2024-04-24T22:45:31.696 INFO:tasks.workunit.client.0.smithi063.stdout:4/548: dwrite d2/f29 [0,4194304] 0 2024-04-24T22:45:31.697 INFO:tasks.workunit.client.1.smithi192.stdout:9/553: write d3/dd/fbe [617906,70314] 0 2024-04-24T22:45:31.698 INFO:tasks.workunit.client.1.smithi192.stdout:8/498: mknod db/dc/d29/c9c 0 2024-04-24T22:45:31.699 INFO:tasks.workunit.client.1.smithi192.stdout:8/499: dread - db/dc/d22/d31/f4c zero size 2024-04-24T22:45:31.702 INFO:tasks.workunit.client.0.smithi063.stdout:3/613: write da/f28 [5198465,76800] 0 2024-04-24T22:45:31.706 INFO:tasks.workunit.client.1.smithi192.stdout:8/500: dwrite db/d5d/f7e [0,4194304] 0 2024-04-24T22:45:31.709 INFO:tasks.workunit.client.1.smithi192.stdout:0/494: creat d3/d27/d3c/d3b/fa9 x:0 0 0 2024-04-24T22:45:31.711 INFO:tasks.workunit.client.0.smithi063.stdout:0/537: mknod d6/d64/d21/d9c/cbf 0 2024-04-24T22:45:31.714 INFO:tasks.workunit.client.0.smithi063.stdout:7/532: dwrite d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:31.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/400: rmdir d2/d9/d25/d2e/d31/d36/d70 39 2024-04-24T22:45:31.718 INFO:tasks.workunit.client.0.smithi063.stdout:8/606: creat d1/d22/d33/fc4 x:0 0 0 2024-04-24T22:45:31.719 INFO:tasks.workunit.client.0.smithi063.stdout:9/609: write d6/d8/db/d6b/f5c [293227,20497] 0 2024-04-24T22:45:31.720 INFO:tasks.workunit.client.0.smithi063.stdout:6/614: mkdir db/d1d/d8d/dce 0 2024-04-24T22:45:31.722 INFO:tasks.workunit.client.1.smithi192.stdout:8/501: read db/dc/f1b [7477936,116406] 0 2024-04-24T22:45:31.722 INFO:tasks.workunit.client.1.smithi192.stdout:9/554: mkdir d3/d59/d7d/dc6 0 2024-04-24T22:45:31.723 INFO:tasks.workunit.client.0.smithi063.stdout:5/583: truncate f0 4343060 0 2024-04-24T22:45:31.725 INFO:tasks.workunit.client.0.smithi063.stdout:4/549: symlink d2/d19/d82/d96/lb3 0 2024-04-24T22:45:31.728 INFO:tasks.workunit.client.0.smithi063.stdout:2/665: dwrite d0/d2/d2d/d31/d89/d48/f5d [0,4194304] 0 2024-04-24T22:45:31.729 INFO:tasks.workunit.client.0.smithi063.stdout:4/550: fsync d2/d19/d82/d96/fad 0 2024-04-24T22:45:31.730 INFO:tasks.workunit.client.0.smithi063.stdout:3/614: rename da/d18/d47/d4d/fae to da/d39/d94/dbc/fd3 0 2024-04-24T22:45:31.731 INFO:tasks.workunit.client.0.smithi063.stdout:6/615: read db/d1d/d20/d29/d5d/f6b [1988404,57156] 0 2024-04-24T22:45:31.733 INFO:tasks.workunit.client.1.smithi192.stdout:0/495: creat d3/d27/d3c/d64/faa x:0 0 0 2024-04-24T22:45:31.733 INFO:tasks.workunit.client.0.smithi063.stdout:3/615: write da/d39/d94/faf [153464,90085] 0 2024-04-24T22:45:31.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/616: fdatasync da/d39/f41 0 2024-04-24T22:45:31.737 INFO:tasks.workunit.client.1.smithi192.stdout:6/416: dread d0/d8/f2b [0,4194304] 0 2024-04-24T22:45:31.737 INFO:tasks.workunit.client.1.smithi192.stdout:2/563: unlink d4/d9/l2e 0 2024-04-24T22:45:31.737 INFO:tasks.workunit.client.1.smithi192.stdout:2/564: chown d4/d9/d37/f3e 26678785 1 2024-04-24T22:45:31.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/551: dwrite d2/d8/d36/d3a/f9b [0,4194304] 0 2024-04-24T22:45:31.738 INFO:tasks.workunit.client.0.smithi063.stdout:3/617: write da/d18/d3a/fc7 [275178,8408] 0 2024-04-24T22:45:31.739 INFO:tasks.workunit.client.1.smithi192.stdout:8/502: creat db/d43/d7c/f9d x:0 0 0 2024-04-24T22:45:31.739 INFO:tasks.workunit.client.0.smithi063.stdout:8/607: mknod d1/d11/d3c/cc5 0 2024-04-24T22:45:31.740 INFO:tasks.workunit.client.1.smithi192.stdout:8/503: write db/dc/f1c [2974237,23724] 0 2024-04-24T22:45:31.744 INFO:tasks.workunit.client.0.smithi063.stdout:9/610: unlink d6/d8/d1f/d20/d22/lc5 0 2024-04-24T22:45:31.746 INFO:tasks.workunit.client.1.smithi192.stdout:6/417: dread d0/d8/f76 [0,4194304] 0 2024-04-24T22:45:31.755 INFO:tasks.workunit.client.1.smithi192.stdout:2/565: mkdir d4/d29/d60/d9a/db3 0 2024-04-24T22:45:31.759 INFO:tasks.workunit.client.0.smithi063.stdout:1/671: dwrite d3/da/da2/dba/d39/f65 [0,4194304] 0 2024-04-24T22:45:31.761 INFO:tasks.workunit.client.1.smithi192.stdout:8/504: creat db/d43/f9e x:0 0 0 2024-04-24T22:45:31.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/418: write d0/d8/d3d/f55 [3574476,7674] 0 2024-04-24T22:45:31.763 INFO:tasks.workunit.client.1.smithi192.stdout:6/419: dread - d0/d8/d29/d5a/f6b zero size 2024-04-24T22:45:31.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/401: creat d2/d9/f8a x:0 0 0 2024-04-24T22:45:31.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/538: dread d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:31.767 INFO:tasks.workunit.client.0.smithi063.stdout:8/608: mknod d1/d11/d32/d38/db7/cc6 0 2024-04-24T22:45:31.768 INFO:tasks.workunit.client.1.smithi192.stdout:2/566: stat d4/d9/c77 0 2024-04-24T22:45:31.769 INFO:tasks.workunit.client.0.smithi063.stdout:9/611: creat d6/d2f/d42/d81/fc6 x:0 0 0 2024-04-24T22:45:31.770 INFO:tasks.workunit.client.1.smithi192.stdout:9/555: dread d3/fb [0,4194304] 0 2024-04-24T22:45:31.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/505: fsync f8 0 2024-04-24T22:45:31.772 INFO:tasks.workunit.client.1.smithi192.stdout:6/420: mknod d0/d8/d3d/d3e/d40/c8f 0 2024-04-24T22:45:31.773 INFO:tasks.workunit.client.1.smithi192.stdout:8/506: write db/dc/d22/f23 [1713593,37787] 0 2024-04-24T22:45:31.776 INFO:tasks.workunit.client.0.smithi063.stdout:2/666: mknod d0/d2/cc5 0 2024-04-24T22:45:31.780 INFO:tasks.workunit.client.1.smithi192.stdout:8/507: dread db/d43/d71/f83 [0,4194304] 0 2024-04-24T22:45:31.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/508: fsync db/dc/d29/f2b 0 2024-04-24T22:45:31.782 INFO:tasks.workunit.client.0.smithi063.stdout:7/533: dwrite d0/f73 [0,4194304] 0 2024-04-24T22:45:31.783 INFO:tasks.workunit.client.1.smithi192.stdout:0/496: dread d3/d27/d28/f34 [0,4194304] 0 2024-04-24T22:45:31.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/509: chown db/dc/l3a 8163319 1 2024-04-24T22:45:31.783 INFO:tasks.workunit.client.0.smithi063.stdout:3/618: dread da/d18/d9e/da3/fab [0,4194304] 0 2024-04-24T22:45:31.784 INFO:tasks.workunit.client.1.smithi192.stdout:8/510: readlink db/dc/d29/l36 0 2024-04-24T22:45:31.784 INFO:tasks.workunit.client.0.smithi063.stdout:3/619: chown da/c11 1998 1 2024-04-24T22:45:31.785 INFO:tasks.workunit.client.0.smithi063.stdout:1/672: creat d3/da/da2/dba/d39/dbf/fdc x:0 0 0 2024-04-24T22:45:31.788 INFO:tasks.workunit.client.0.smithi063.stdout:5/584: write d5/d24/f86 [3144117,33934] 0 2024-04-24T22:45:31.789 INFO:tasks.workunit.client.1.smithi192.stdout:4/402: mknod d2/db/d5d/d77/d82/c8b 0 2024-04-24T22:45:31.794 INFO:tasks.workunit.client.0.smithi063.stdout:2/667: mkdir d0/d2/d35/dc6 0 2024-04-24T22:45:31.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/668: read - d0/d2/d63/fb0 zero size 2024-04-24T22:45:31.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/552: write d2/d19/f25 [4870796,84439] 0 2024-04-24T22:45:31.796 INFO:tasks.workunit.client.0.smithi063.stdout:2/669: chown d0/d2/d27/d3a/c3d 788 1 2024-04-24T22:45:31.797 INFO:tasks.workunit.client.0.smithi063.stdout:3/620: mknod da/d39/d94/db3/dbe/cd4 0 2024-04-24T22:45:31.802 INFO:tasks.workunit.client.1.smithi192.stdout:8/511: mkdir db/dc/d29/d9f 0 2024-04-24T22:45:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: pgmap v13: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 193 MiB/s wr, 320 op/s 2024-04-24T22:45:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi063 to 1557M 2024-04-24T22:45:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:31 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.805 INFO:tasks.workunit.client.1.smithi192.stdout:4/403: readlink d2/d9/d25/d2e/d31/d36/l65 0 2024-04-24T22:45:31.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/612: fsync d6/d8/db/d1c/d78/d9a/daa/fab 0 2024-04-24T22:45:31.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/556: dwrite d3/dd/f3a [0,4194304] 0 2024-04-24T22:45:31.814 INFO:tasks.workunit.client.1.smithi192.stdout:2/567: getdents d4/d9/d10/da7 0 2024-04-24T22:45:31.820 INFO:tasks.workunit.client.1.smithi192.stdout:4/404: creat d2/db/d75/f8c x:0 0 0 2024-04-24T22:45:31.821 INFO:tasks.workunit.client.1.smithi192.stdout:6/421: dwrite d0/f19 [0,4194304] 0 2024-04-24T22:45:31.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/422: chown d0/d8/l57 6 1 2024-04-24T22:45:31.825 INFO:tasks.workunit.client.0.smithi063.stdout:4/553: unlink d2/d19/d33/c3b 0 2024-04-24T22:45:31.825 INFO:tasks.workunit.client.0.smithi063.stdout:6/616: getdents db/d1d/d2d/d69 0 2024-04-24T22:45:31.827 INFO:tasks.workunit.client.1.smithi192.stdout:0/497: creat d3/d27/d3c/fab x:0 0 0 2024-04-24T22:45:31.827 INFO:tasks.workunit.client.1.smithi192.stdout:0/498: dread - d3/d27/d28/d2e/d4a/f6b zero size 2024-04-24T22:45:31.828 INFO:tasks.workunit.client.0.smithi063.stdout:2/670: creat d0/dd/fc7 x:0 0 0 2024-04-24T22:45:31.828 INFO:tasks.workunit.client.0.smithi063.stdout:3/621: truncate da/d18/d47/d4d/d62/f80 4939801 0 2024-04-24T22:45:31.828 INFO:tasks.workunit.client.0.smithi063.stdout:1/673: mknod d3/da/da2/dba/d11/dc6/cdd 0 2024-04-24T22:45:31.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/554: chown d2/l42 491883 1 2024-04-24T22:45:31.829 INFO:tasks.workunit.client.0.smithi063.stdout:2/671: dread - d0/fbc zero size 2024-04-24T22:45:31.829 INFO:tasks.workunit.client.0.smithi063.stdout:0/539: rename d6/dd/l1f to d6/d64/lc0 0 2024-04-24T22:45:31.830 INFO:tasks.workunit.client.1.smithi192.stdout:8/512: getdents db/d5d/d74/d96/d97 0 2024-04-24T22:45:31.830 INFO:tasks.workunit.client.1.smithi192.stdout:0/499: chown d3/d27/d28/d2e/d4a/d5c/f46 1485398162 1 2024-04-24T22:45:31.830 INFO:tasks.workunit.client.1.smithi192.stdout:2/568: creat d4/d9/d10/d6c/d6f/da8/fb4 x:0 0 0 2024-04-24T22:45:31.833 INFO:tasks.workunit.client.1.smithi192.stdout:6/423: mknod d0/d8/d29/d4c/d4d/c90 0 2024-04-24T22:45:31.836 INFO:tasks.workunit.client.1.smithi192.stdout:2/569: dwrite d4/d9/f8b [0,4194304] 0 2024-04-24T22:45:31.837 INFO:tasks.workunit.client.1.smithi192.stdout:2/570: write d4/f13 [692650,123584] 0 2024-04-24T22:45:31.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/571: write d4/d9/d37/f40 [4752620,130706] 0 2024-04-24T22:45:31.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/513: mkdir db/d5d/d74/da0 0 2024-04-24T22:45:31.849 INFO:tasks.workunit.client.1.smithi192.stdout:6/424: chown d0/d8/d29/d5a/d63/l87 192305304 1 2024-04-24T22:45:31.849 INFO:tasks.workunit.client.1.smithi192.stdout:8/514: dwrite db/d5d/f84 [0,4194304] 0 2024-04-24T22:45:31.850 INFO:tasks.workunit.client.1.smithi192.stdout:6/425: stat d0/d8/d3d/d3e/d40 0 2024-04-24T22:45:31.851 INFO:tasks.workunit.client.1.smithi192.stdout:9/557: write d3/f45 [630703,19822] 0 2024-04-24T22:45:31.854 INFO:tasks.workunit.client.1.smithi192.stdout:9/558: chown d3/dd/d62/c78 10 1 2024-04-24T22:45:31.854 INFO:tasks.workunit.client.1.smithi192.stdout:2/572: unlink d4/d9/l9d 0 2024-04-24T22:45:31.855 INFO:tasks.workunit.client.1.smithi192.stdout:0/500: unlink d3/d27/d28/d2e/c76 0 2024-04-24T22:45:31.857 INFO:tasks.workunit.client.1.smithi192.stdout:8/515: dread db/dc/d22/d31/f4d [0,4194304] 0 2024-04-24T22:45:31.861 INFO:tasks.workunit.client.1.smithi192.stdout:8/516: dread db/dc/d29/f4f [0,4194304] 0 2024-04-24T22:45:31.862 INFO:tasks.workunit.client.1.smithi192.stdout:9/559: dread - d3/dd/de/d17/f4f zero size 2024-04-24T22:45:31.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/405: getdents d2/d9/d25/d2e/d31/d53 0 2024-04-24T22:45:31.864 INFO:tasks.workunit.client.1.smithi192.stdout:0/501: mkdir d3/d27/d28/d2e/d4a/d5c/d80/dac 0 2024-04-24T22:45:31.864 INFO:tasks.workunit.client.1.smithi192.stdout:4/406: fsync d2/d9/d25/d2e/d31/d36/d5b/f47 0 2024-04-24T22:45:31.867 INFO:tasks.workunit.client.1.smithi192.stdout:7/550: sync 2024-04-24T22:45:31.867 INFO:tasks.workunit.client.1.smithi192.stdout:5/397: sync 2024-04-24T22:45:31.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/404: sync 2024-04-24T22:45:31.868 INFO:tasks.workunit.client.0.smithi063.stdout:5/585: link d5/d24/d49/d3b/cae d5/d24/d36/cc8 0 2024-04-24T22:45:31.869 INFO:tasks.workunit.client.0.smithi063.stdout:6/617: creat db/d1d/d20/d3f/d5f/d93/fcf x:0 0 0 2024-04-24T22:45:31.869 INFO:tasks.workunit.client.1.smithi192.stdout:8/517: rmdir db/d75 39 2024-04-24T22:45:31.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/609: rename d1/d2/f1c to d1/d11/d32/fc7 0 2024-04-24T22:45:31.869 INFO:tasks.workunit.client.0.smithi063.stdout:4/555: read d2/d8/d40/d51/f64 [563317,50356] 0 2024-04-24T22:45:31.871 INFO:tasks.workunit.client.0.smithi063.stdout:4/556: chown d2/d8/d36/f7d 13 1 2024-04-24T22:45:31.872 INFO:tasks.workunit.client.0.smithi063.stdout:4/557: write d2/d80/d81/d90/f92 [1393474,74729] 0 2024-04-24T22:45:31.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/560: fdatasync d3/dd/d53/d60/faa 0 2024-04-24T22:45:31.875 INFO:tasks.workunit.client.1.smithi192.stdout:9/561: truncate d3/d24/f7e 117360 0 2024-04-24T22:45:31.875 INFO:tasks.workunit.client.1.smithi192.stdout:0/502: creat d3/d27/d28/d2e/d4a/d5c/d8a/fad x:0 0 0 2024-04-24T22:45:31.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/503: write d3/d27/d3c/fab [248816,114929] 0 2024-04-24T22:45:31.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/426: creat d0/d8/f91 x:0 0 0 2024-04-24T22:45:31.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/405: read d9/d15/f1f [1413397,34726] 0 2024-04-24T22:45:31.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/406: stat d9/d15/c5d 0 2024-04-24T22:45:31.885 INFO:tasks.workunit.client.0.smithi063.stdout:3/622: rename da/d18/d47/d4d/c8a to da/d39/d94/dbc/cd5 0 2024-04-24T22:45:31.886 INFO:tasks.workunit.client.0.smithi063.stdout:3/623: stat da/d18/d47/l51 0 2024-04-24T22:45:31.886 INFO:tasks.workunit.client.1.smithi192.stdout:7/551: symlink d2/db/d2a/lac 0 2024-04-24T22:45:31.887 INFO:tasks.workunit.client.0.smithi063.stdout:9/613: dread d6/d8/db/d6b/d4f/f5a [0,4194304] 0 2024-04-24T22:45:31.888 INFO:tasks.workunit.client.0.smithi063.stdout:3/624: chown da/d18/d47/d4d/d62/l76 14412926 1 2024-04-24T22:45:31.889 INFO:tasks.workunit.client.0.smithi063.stdout:6/618: write db/d1d/d20/d3f/d7f/d68/d70/fad [527907,121153] 0 2024-04-24T22:45:31.889 INFO:tasks.workunit.client.1.smithi192.stdout:5/398: creat d0/d13/d3b/d2f/f84 x:0 0 0 2024-04-24T22:45:31.891 INFO:tasks.workunit.client.0.smithi063.stdout:0/540: write d6/d64/d21/d25/d62/d75/f61 [1203393,39069] 0 2024-04-24T22:45:31.891 INFO:tasks.workunit.client.1.smithi192.stdout:2/573: creat d4/d9/d10/d6c/fb5 x:0 0 0 2024-04-24T22:45:31.892 INFO:tasks.workunit.client.0.smithi063.stdout:8/610: dread d1/d2/d17/f4b [0,4194304] 0 2024-04-24T22:45:31.893 INFO:tasks.workunit.client.1.smithi192.stdout:2/574: dread - d4/d9/d37/f3e zero size 2024-04-24T22:45:31.894 INFO:tasks.workunit.client.0.smithi063.stdout:1/674: dwrite d3/f3c [0,4194304] 0 2024-04-24T22:45:31.896 INFO:tasks.workunit.client.0.smithi063.stdout:4/558: rename d2/d8/d36/l58 to d2/d19/d33/d9d/lb4 0 2024-04-24T22:45:31.899 INFO:tasks.workunit.client.0.smithi063.stdout:9/614: dwrite d6/d8/d1f/f52 [0,4194304] 0 2024-04-24T22:45:31.900 INFO:tasks.workunit.client.0.smithi063.stdout:4/559: truncate d2/f5 8949103 0 2024-04-24T22:45:31.902 INFO:tasks.workunit.client.0.smithi063.stdout:0/541: dread d6/dd/f54 [0,4194304] 0 2024-04-24T22:45:31.903 INFO:tasks.workunit.client.0.smithi063.stdout:8/611: creat d1/d22/d33/da6/dae/fc8 x:0 0 0 2024-04-24T22:45:31.914 INFO:tasks.workunit.client.0.smithi063.stdout:2/672: getdents d0/d2/d2f 0 2024-04-24T22:45:31.919 INFO:tasks.workunit.client.0.smithi063.stdout:3/625: write da/f3d [3669393,24606] 0 2024-04-24T22:45:31.920 INFO:tasks.workunit.client.0.smithi063.stdout:8/612: symlink d1/d22/d33/da6/dae/lc9 0 2024-04-24T22:45:31.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/613: dread d1/d11/f93 [0,4194304] 0 2024-04-24T22:45:31.924 INFO:tasks.workunit.client.0.smithi063.stdout:6/619: dread db/d1d/d2d/d48/f63 [0,4194304] 0 2024-04-24T22:45:31.924 INFO:tasks.workunit.client.0.smithi063.stdout:2/673: creat d0/dd/d72/d77/d87/fc8 x:0 0 0 2024-04-24T22:45:31.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/614: write d1/f62 [5186397,120822] 0 2024-04-24T22:45:31.926 INFO:tasks.workunit.client.0.smithi063.stdout:2/674: chown d0/d2/d2d/d31/d89/d7/f13 961 1 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: pgmap v13: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 193 MiB/s wr, 320 op/s 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi063 to 1557M 2024-04-24T22:45:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:31 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:31.928 INFO:tasks.workunit.client.0.smithi063.stdout:1/675: link d3/da/da2/dba/d39/f66 d3/da/da2/dba/d11/d5e/d9c/dcb/fde 0 2024-04-24T22:45:31.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/676: write d3/db5/d21/d9f/daa/db7/fce [584833,90698] 0 2024-04-24T22:45:32.010 INFO:tasks.workunit.client.0.smithi063.stdout:7/534: sync 2024-04-24T22:45:32.015 INFO:tasks.workunit.client.0.smithi063.stdout:7/535: dread d0/d4/de/d17/f42 [0,4194304] 0 2024-04-24T22:45:32.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/536: dwrite d0/d4/de/f10 [4194304,4194304] 0 2024-04-24T22:45:32.026 INFO:tasks.workunit.client.0.smithi063.stdout:7/537: dwrite d0/d4/de/d17/d35/d69/f7b [0,4194304] 0 2024-04-24T22:45:32.040 INFO:tasks.workunit.client.1.smithi192.stdout:9/562: unlink d3/dd/c15 0 2024-04-24T22:45:32.040 INFO:tasks.workunit.client.1.smithi192.stdout:9/563: chown d3/d24/f65 82192 1 2024-04-24T22:45:32.041 INFO:tasks.workunit.client.1.smithi192.stdout:9/564: chown d3/dd/de/d17/l22 79210 1 2024-04-24T22:45:32.042 INFO:tasks.workunit.client.0.smithi063.stdout:3/626: mknod da/d39/d94/dbc/cd6 0 2024-04-24T22:45:32.047 INFO:tasks.workunit.client.1.smithi192.stdout:4/407: truncate d2/d9/d25/d2e/d31/d36/d5b/f24 2061971 0 2024-04-24T22:45:32.050 INFO:tasks.workunit.client.1.smithi192.stdout:9/565: dwrite d3/d5d/f7c [4194304,4194304] 0 2024-04-24T22:45:32.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/462: sync 2024-04-24T22:45:32.051 INFO:tasks.workunit.client.1.smithi192.stdout:1/407: write d9/dd/f19 [1251291,130408] 0 2024-04-24T22:45:32.052 INFO:tasks.workunit.client.1.smithi192.stdout:3/463: dread - d2/d4/d12/d1a/f81 zero size 2024-04-24T22:45:32.054 INFO:tasks.workunit.client.1.smithi192.stdout:7/552: rmdir d2/db/d21/d42/d67/d9b 39 2024-04-24T22:45:32.057 INFO:tasks.workunit.client.1.smithi192.stdout:7/553: chown d2/db/d49/l65 331 1 2024-04-24T22:45:32.057 INFO:tasks.workunit.client.0.smithi063.stdout:4/560: dwrite d2/d8/d36/d3a/d50/f9e [0,4194304] 0 2024-04-24T22:45:32.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/677: rmdir d3/da/da2/dba/d11 39 2024-04-24T22:45:32.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/542: dwrite d6/dd/f12 [0,4194304] 0 2024-04-24T22:45:32.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/543: truncate d6/dd/f4d 4425277 0 2024-04-24T22:45:32.068 INFO:tasks.workunit.client.0.smithi063.stdout:0/544: write d6/d3a/f7a [4476546,71231] 0 2024-04-24T22:45:32.069 INFO:tasks.workunit.client.1.smithi192.stdout:0/504: dwrite d3/d27/d28/d2e/d4a/fa1 [0,4194304] 0 2024-04-24T22:45:32.070 INFO:tasks.workunit.client.0.smithi063.stdout:0/545: write d6/dd/f4d [190718,22611] 0 2024-04-24T22:45:32.072 INFO:tasks.workunit.client.1.smithi192.stdout:4/408: mkdir d2/db/d5d/d77/d82/d8d 0 2024-04-24T22:45:32.073 INFO:tasks.workunit.client.1.smithi192.stdout:0/505: chown d3/d27/d28/c95 59 1 2024-04-24T22:45:32.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/427: write d0/d8/d29/d4c/d4d/f56 [64130,119308] 0 2024-04-24T22:45:32.078 INFO:tasks.workunit.client.1.smithi192.stdout:1/408: truncate d9/f6a 840717 0 2024-04-24T22:45:32.080 INFO:tasks.workunit.client.0.smithi063.stdout:3/627: creat da/d48/fd7 x:0 0 0 2024-04-24T22:45:32.082 INFO:tasks.workunit.client.0.smithi063.stdout:6/620: dread db/d1d/d20/d29/d5d/f6b [0,4194304] 0 2024-04-24T22:45:32.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/561: rename d2/d8/d36/la7 to d2/d19/d33/d9d/da8/lb5 0 2024-04-24T22:45:32.087 INFO:tasks.workunit.client.1.smithi192.stdout:5/399: symlink d0/d13/d53/l85 0 2024-04-24T22:45:32.088 INFO:tasks.workunit.client.1.smithi192.stdout:5/400: stat d0/d13/d19/d64/d4f/d6c 0 2024-04-24T22:45:32.088 INFO:tasks.workunit.client.1.smithi192.stdout:5/401: read - d0/d13/d19/f4b zero size 2024-04-24T22:45:32.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/575: truncate d4/d9/d10/d6c/f85 79999 0 2024-04-24T22:45:32.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/586: sync 2024-04-24T22:45:32.090 INFO:tasks.workunit.client.0.smithi063.stdout:5/587: chown d5/d1d/fba 259690 1 2024-04-24T22:45:32.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/675: dread d0/d2/d2d/d31/d89/d1c/f2a [0,4194304] 0 2024-04-24T22:45:32.097 INFO:tasks.workunit.client.0.smithi063.stdout:2/676: dread - d0/d2/d39/fb7 zero size 2024-04-24T22:45:32.098 INFO:tasks.workunit.client.0.smithi063.stdout:2/677: readlink d0/d2/d2d/d31/d89/d7/d86/l8d 0 2024-04-24T22:45:32.100 INFO:tasks.workunit.client.1.smithi192.stdout:9/566: symlink d3/dd/lc7 0 2024-04-24T22:45:32.101 INFO:tasks.workunit.client.1.smithi192.stdout:1/409: chown d9/f2c 4587519 1 2024-04-24T22:45:32.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/621: mkdir db/d1d/d20/d29/dab/dd0 0 2024-04-24T22:45:32.105 INFO:tasks.workunit.client.1.smithi192.stdout:3/464: mknod d2/d4/d12/d39/c91 0 2024-04-24T22:45:32.110 INFO:tasks.workunit.client.0.smithi063.stdout:1/678: mkdir d3/da/da2/dba/d11/d5e/d9c/ddf 0 2024-04-24T22:45:32.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/518: sync 2024-04-24T22:45:32.113 INFO:tasks.workunit.client.0.smithi063.stdout:5/588: mkdir d5/d24/d44/dc9 0 2024-04-24T22:45:32.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/538: truncate d0/d4/f3a 2030862 0 2024-04-24T22:45:32.117 INFO:tasks.workunit.client.0.smithi063.stdout:0/546: unlink d6/d64/da0/f83 0 2024-04-24T22:45:32.118 INFO:tasks.workunit.client.1.smithi192.stdout:1/410: dread d9/dd/f41 [0,4194304] 0 2024-04-24T22:45:32.118 INFO:tasks.workunit.client.1.smithi192.stdout:2/576: dwrite d4/f69 [0,4194304] 0 2024-04-24T22:45:32.119 INFO:tasks.workunit.client.1.smithi192.stdout:0/506: creat d3/d27/d3c/d6c/fae x:0 0 0 2024-04-24T22:45:32.121 INFO:tasks.workunit.client.0.smithi063.stdout:2/678: rename d0/d2/d2d/d31/d89/d7/dac to d0/dd/d72/dc9 0 2024-04-24T22:45:32.122 INFO:tasks.workunit.client.0.smithi063.stdout:9/615: sync 2024-04-24T22:45:32.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/554: truncate d2/db/d49/f70 3162943 0 2024-04-24T22:45:32.127 INFO:tasks.workunit.client.0.smithi063.stdout:8/615: link d1/d21/c35 d1/d11/cca 0 2024-04-24T22:45:32.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/577: dwrite d4/d9/d10/d6c/d8c/fab [0,4194304] 0 2024-04-24T22:45:32.131 INFO:tasks.workunit.client.1.smithi192.stdout:3/465: truncate d2/d27/d28/f30 3045651 0 2024-04-24T22:45:32.132 INFO:tasks.workunit.client.1.smithi192.stdout:7/555: dread d2/db/d21/d42/d5b/d63/f6d [0,4194304] 0 2024-04-24T22:45:32.133 INFO:tasks.workunit.client.0.smithi063.stdout:5/589: mknod d5/d11/d23/d2d/d62/cca 0 2024-04-24T22:45:32.138 INFO:tasks.workunit.client.0.smithi063.stdout:2/679: mkdir d0/d2/d2d/d31/d89/d1c/d3f/dca 0 2024-04-24T22:45:32.151 INFO:tasks.workunit.client.0.smithi063.stdout:4/562: creat d2/d8/d36/fb6 x:0 0 0 2024-04-24T22:45:32.154 INFO:tasks.workunit.client.0.smithi063.stdout:5/590: rmdir d5/d24/d36/d65 39 2024-04-24T22:45:32.157 INFO:tasks.workunit.client.1.smithi192.stdout:9/567: dwrite d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:32.158 INFO:tasks.workunit.client.0.smithi063.stdout:0/547: write d6/d64/d21/f36 [2379466,129680] 0 2024-04-24T22:45:32.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/568: read - d3/d59/f5a zero size 2024-04-24T22:45:32.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/578: symlink d4/d9/d10/da7/lb6 0 2024-04-24T22:45:32.161 INFO:tasks.workunit.client.0.smithi063.stdout:7/539: dwrite d0/d4/de/d6f/f47 [0,4194304] 0 2024-04-24T22:45:32.162 INFO:tasks.workunit.client.0.smithi063.stdout:5/591: dwrite d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:32.163 INFO:tasks.workunit.client.1.smithi192.stdout:1/411: dread d9/d15/d2d/d31/d2f/f3f [0,4194304] 0 2024-04-24T22:45:32.163 INFO:tasks.workunit.client.0.smithi063.stdout:7/540: fsync d0/d4/de/d17/f42 0 2024-04-24T22:45:32.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/519: symlink db/d75/la1 0 2024-04-24T22:45:32.167 INFO:tasks.workunit.client.1.smithi192.stdout:5/402: creat d0/d13/f86 x:0 0 0 2024-04-24T22:45:32.169 INFO:tasks.workunit.client.0.smithi063.stdout:3/628: getdents da/d39 0 2024-04-24T22:45:32.170 INFO:tasks.workunit.client.1.smithi192.stdout:0/507: unlink d3/l6 0 2024-04-24T22:45:32.173 INFO:tasks.workunit.client.1.smithi192.stdout:4/409: getdents d2 0 2024-04-24T22:45:32.173 INFO:tasks.workunit.client.0.smithi063.stdout:6/622: link db/d1d/d2d/d48/cc0 db/d1d/d2d/d48/cd1 0 2024-04-24T22:45:32.175 INFO:tasks.workunit.client.0.smithi063.stdout:4/563: dread - d2/d8/d40/d51/f6c zero size 2024-04-24T22:45:32.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/629: dwrite da/d39/d94/faf [0,4194304] 0 2024-04-24T22:45:32.177 INFO:tasks.workunit.client.1.smithi192.stdout:6/428: link d0/d8/d29/d58/c8a d0/c92 0 2024-04-24T22:45:32.177 INFO:tasks.workunit.client.0.smithi063.stdout:6/623: write db/d1d/d20/d3f/d7f/f97 [195023,98121] 0 2024-04-24T22:45:32.180 INFO:tasks.workunit.client.1.smithi192.stdout:3/466: creat d2/d89/f92 x:0 0 0 2024-04-24T22:45:32.182 INFO:tasks.workunit.client.1.smithi192.stdout:2/579: mkdir d4/d9/d10/d6c/d95/db7 0 2024-04-24T22:45:32.184 INFO:tasks.workunit.client.0.smithi063.stdout:1/679: rename d3/db5/d2b to d3/de0 0 2024-04-24T22:45:32.184 INFO:tasks.workunit.client.0.smithi063.stdout:4/564: dwrite d2/d19/d82/f84 [0,4194304] 0 2024-04-24T22:45:32.186 INFO:tasks.workunit.client.0.smithi063.stdout:4/565: fsync d2/d19/fa2 0 2024-04-24T22:45:32.188 INFO:tasks.workunit.client.0.smithi063.stdout:5/592: creat d5/d11/d23/d2d/d62/d73/d95/fcb x:0 0 0 2024-04-24T22:45:32.189 INFO:tasks.workunit.client.0.smithi063.stdout:7/541: mknod d0/d4/de/d17/d28/d31/cb9 0 2024-04-24T22:45:32.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/542: stat d0/d4/de/d17/d28/d31/c62 0 2024-04-24T22:45:32.192 INFO:tasks.workunit.client.0.smithi063.stdout:9/616: link d6/d2f/f35 d6/d8/d1f/d20/d2a/d4b/d5e/fc7 0 2024-04-24T22:45:32.194 INFO:tasks.workunit.client.0.smithi063.stdout:5/593: dwrite d5/d24/db6/fbd [0,4194304] 0 2024-04-24T22:45:32.199 INFO:tasks.workunit.client.0.smithi063.stdout:6/624: mkdir db/d85/dd2 0 2024-04-24T22:45:32.203 INFO:tasks.workunit.client.0.smithi063.stdout:5/594: dwrite d5/d24/d36/f63 [0,4194304] 0 2024-04-24T22:45:32.204 INFO:tasks.workunit.client.1.smithi192.stdout:6/429: rename d0/d8/d3d/d3e/d40/f68 to d0/d8/d3d/d3e/d40/d44/f93 0 2024-04-24T22:45:32.207 INFO:tasks.workunit.client.0.smithi063.stdout:0/548: rename d6/d64/d21/d25/c4c to d6/d64/d21/d25/d62/d63/d89/cc1 0 2024-04-24T22:45:32.208 INFO:tasks.workunit.client.1.smithi192.stdout:3/467: mkdir d2/d4/d12/d65/d93 0 2024-04-24T22:45:32.208 INFO:tasks.workunit.client.0.smithi063.stdout:0/549: write d6/d74/f97 [404346,124229] 0 2024-04-24T22:45:32.216 INFO:tasks.workunit.client.1.smithi192.stdout:9/569: dwrite d3/d24/f4e [0,4194304] 0 2024-04-24T22:45:32.218 INFO:tasks.workunit.client.1.smithi192.stdout:9/570: fdatasync d3/d5d/da1/f70 0 2024-04-24T22:45:32.220 INFO:tasks.workunit.client.1.smithi192.stdout:1/412: creat d9/d6f/f86 x:0 0 0 2024-04-24T22:45:32.221 INFO:tasks.workunit.client.1.smithi192.stdout:5/403: dread d0/d5b/f7a [0,4194304] 0 2024-04-24T22:45:32.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/556: truncate d2/db/f20 3587264 0 2024-04-24T22:45:32.225 INFO:tasks.workunit.client.1.smithi192.stdout:8/520: creat db/dc/d22/d31/d33/d8f/fa2 x:0 0 0 2024-04-24T22:45:32.226 INFO:tasks.workunit.client.1.smithi192.stdout:8/521: write db/d5d/f84 [849885,50774] 0 2024-04-24T22:45:32.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/522: readlink db/l19 0 2024-04-24T22:45:32.228 INFO:tasks.workunit.client.1.smithi192.stdout:0/508: mknod d3/d27/d3c/caf 0 2024-04-24T22:45:32.234 INFO:tasks.workunit.client.1.smithi192.stdout:2/580: rename d4/d29/d60/d9a/l7c to d4/d29/d60/lb8 0 2024-04-24T22:45:32.235 INFO:tasks.workunit.client.0.smithi063.stdout:8/616: getdents d1/d22 0 2024-04-24T22:45:32.236 INFO:tasks.workunit.client.1.smithi192.stdout:2/581: read d4/d9/f5b [3025431,122887] 0 2024-04-24T22:45:32.236 INFO:tasks.workunit.client.0.smithi063.stdout:9/617: fdatasync d6/d8/db/d6b/d4f/d59/f76 0 2024-04-24T22:45:32.237 INFO:tasks.workunit.client.1.smithi192.stdout:2/582: dread - d4/d9/fa4 zero size 2024-04-24T22:45:32.242 INFO:tasks.workunit.client.1.smithi192.stdout:3/468: truncate d2/d27/f4a 626429 0 2024-04-24T22:45:32.244 INFO:tasks.workunit.client.0.smithi063.stdout:1/680: write d3/db5/f7f [2060495,55053] 0 2024-04-24T22:45:32.244 INFO:tasks.workunit.client.0.smithi063.stdout:3/630: mknod da/d18/d47/d4d/dc2/cd8 0 2024-04-24T22:45:32.246 INFO:tasks.workunit.client.0.smithi063.stdout:0/550: dread d6/d64/f3b [0,4194304] 0 2024-04-24T22:45:32.249 INFO:tasks.workunit.client.0.smithi063.stdout:5/595: creat d5/d24/db6/fcc x:0 0 0 2024-04-24T22:45:32.250 INFO:tasks.workunit.client.1.smithi192.stdout:4/410: write d2/db/f4e [3343771,50777] 0 2024-04-24T22:45:32.252 INFO:tasks.workunit.client.0.smithi063.stdout:2/680: getdents d0/d2/d39 0 2024-04-24T22:45:32.255 INFO:tasks.workunit.client.1.smithi192.stdout:7/557: chown d2/db/d25/l31 694 1 2024-04-24T22:45:32.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/596: dread d5/d24/d36/f5d [0,4194304] 0 2024-04-24T22:45:32.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/543: symlink d0/d4/de/d6f/lba 0 2024-04-24T22:45:32.265 INFO:tasks.workunit.client.0.smithi063.stdout:9/618: mknod d6/d8/d53/cc8 0 2024-04-24T22:45:32.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/523: rename db/d5d/d74/d96/d97 to db/dc/d22/da3 0 2024-04-24T22:45:32.271 INFO:tasks.workunit.client.0.smithi063.stdout:8/617: write d1/f1f [2374135,27338] 0 2024-04-24T22:45:32.274 INFO:tasks.workunit.client.1.smithi192.stdout:2/583: symlink d4/d29/d60/lb9 0 2024-04-24T22:45:32.275 INFO:tasks.workunit.client.1.smithi192.stdout:2/584: fdatasync d4/d9/d10/f58 0 2024-04-24T22:45:32.276 INFO:tasks.workunit.client.1.smithi192.stdout:2/585: truncate d4/d9/fb 4358077 0 2024-04-24T22:45:32.277 INFO:tasks.workunit.client.1.smithi192.stdout:2/586: write d4/d9/d37/f3e [445607,87970] 0 2024-04-24T22:45:32.277 INFO:tasks.workunit.client.1.smithi192.stdout:3/469: truncate d2/d4/d7/d4b/f50 856968 0 2024-04-24T22:45:32.278 INFO:tasks.workunit.client.1.smithi192.stdout:6/430: truncate d0/d8/f2a 2084911 0 2024-04-24T22:45:32.282 INFO:tasks.workunit.client.1.smithi192.stdout:7/558: symlink d2/db/d49/lad 0 2024-04-24T22:45:32.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/587: mkdir d4/d9/d9f/dba 0 2024-04-24T22:45:32.291 INFO:tasks.workunit.client.1.smithi192.stdout:9/571: symlink d3/dd/de/dba/dbc/lc8 0 2024-04-24T22:45:32.293 INFO:tasks.workunit.client.1.smithi192.stdout:4/411: truncate d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f34 602469 0 2024-04-24T22:45:32.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/412: fdatasync d2/db/d75/f8c 0 2024-04-24T22:45:32.294 INFO:tasks.workunit.client.1.smithi192.stdout:7/559: dread - d2/db/d21/d42/d5b/f75 zero size 2024-04-24T22:45:32.299 INFO:tasks.workunit.client.1.smithi192.stdout:8/524: truncate db/dc/d22/d31/f45 64166 0 2024-04-24T22:45:32.301 INFO:tasks.workunit.client.1.smithi192.stdout:2/588: read d4/d9/d37/f62 [2108,8892] 0 2024-04-24T22:45:32.303 INFO:tasks.workunit.client.1.smithi192.stdout:4/413: dwrite d2/d9/d25/d2e/d31/d36/d5b/d64/f6c [0,4194304] 0 2024-04-24T22:45:32.303 INFO:tasks.workunit.client.1.smithi192.stdout:4/414: read - d2/db/f7a zero size 2024-04-24T22:45:32.304 INFO:tasks.workunit.client.1.smithi192.stdout:2/589: write d4/d9/f4c [2245542,82389] 0 2024-04-24T22:45:32.304 INFO:tasks.workunit.client.1.smithi192.stdout:3/470: mknod d2/d4/d12/d65/d93/c94 0 2024-04-24T22:45:32.307 INFO:tasks.workunit.client.1.smithi192.stdout:9/572: symlink d3/dd/d62/d6d/lc9 0 2024-04-24T22:45:32.312 INFO:tasks.workunit.client.1.smithi192.stdout:9/573: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:32.315 INFO:tasks.workunit.client.0.smithi063.stdout:0/551: write d6/d64/d21/f86 [2270106,80512] 0 2024-04-24T22:45:32.317 INFO:tasks.workunit.client.0.smithi063.stdout:4/566: link d2/d19/d82/laa d2/d8/d36/da3/lb7 0 2024-04-24T22:45:32.320 INFO:tasks.workunit.client.0.smithi063.stdout:5/597: readlink d5/d1d/dc0/dbb/l34 0 2024-04-24T22:45:32.321 INFO:tasks.workunit.client.1.smithi192.stdout:3/471: read d2/f13 [1158774,10392] 0 2024-04-24T22:45:32.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/472: readlink d2/l4c 0 2024-04-24T22:45:32.322 INFO:tasks.workunit.client.0.smithi063.stdout:5/598: chown d5/d1d/dc0/dbb/f89 343 1 2024-04-24T22:45:32.323 INFO:tasks.workunit.client.0.smithi063.stdout:0/552: dwrite d6/d64/d21/d9c/fbd [0,4194304] 0 2024-04-24T22:45:32.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/590: creat d4/d9/d10/da7/fbb x:0 0 0 2024-04-24T22:45:32.328 INFO:tasks.workunit.client.0.smithi063.stdout:4/567: dwrite d2/f35 [0,4194304] 0 2024-04-24T22:45:32.328 INFO:tasks.workunit.client.0.smithi063.stdout:9/619: dread - d6/d8/db/d93/da0/fae zero size 2024-04-24T22:45:32.329 INFO:tasks.workunit.client.1.smithi192.stdout:7/560: dread d2/db/d49/f70 [0,4194304] 0 2024-04-24T22:45:32.329 INFO:tasks.workunit.client.1.smithi192.stdout:0/509: write d3/d27/d28/d2e/d4a/f4b [4467426,95539] 0 2024-04-24T22:45:32.329 INFO:tasks.workunit.client.1.smithi192.stdout:8/525: creat db/dc/d29/d9f/fa4 x:0 0 0 2024-04-24T22:45:32.330 INFO:tasks.workunit.client.1.smithi192.stdout:8/526: chown db/l32 45776 1 2024-04-24T22:45:32.331 INFO:tasks.workunit.client.0.smithi063.stdout:1/681: symlink d3/le1 0 2024-04-24T22:45:32.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/527: write db/dc/f17 [2333236,118008] 0 2024-04-24T22:45:32.335 INFO:tasks.workunit.client.0.smithi063.stdout:3/631: mknod da/d18/d3a/cd9 0 2024-04-24T22:45:32.338 INFO:tasks.workunit.client.0.smithi063.stdout:2/681: symlink d0/d2/d2d/d31/d89/lcb 0 2024-04-24T22:45:32.338 INFO:tasks.workunit.client.0.smithi063.stdout:2/682: readlink d0/l2c 0 2024-04-24T22:45:32.339 INFO:tasks.workunit.client.1.smithi192.stdout:3/473: write d2/d4/d12/d1a/d1d/d55/f24 [1141271,10168] 0 2024-04-24T22:45:32.340 INFO:tasks.workunit.client.0.smithi063.stdout:3/632: read da/f3d [3317408,25719] 0 2024-04-24T22:45:32.340 INFO:tasks.workunit.client.0.smithi063.stdout:6/625: truncate db/f2a 4352027 0 2024-04-24T22:45:32.344 INFO:tasks.workunit.client.0.smithi063.stdout:7/544: mkdir d0/db5/dbb 0 2024-04-24T22:45:32.344 INFO:tasks.workunit.client.0.smithi063.stdout:2/683: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:32.345 INFO:tasks.workunit.client.0.smithi063.stdout:7/545: chown d0/d4 206 1 2024-04-24T22:45:32.346 INFO:tasks.workunit.client.1.smithi192.stdout:0/510: mknod d3/d27/d28/d2e/d4a/d5c/d80/cb0 0 2024-04-24T22:45:32.346 INFO:tasks.workunit.client.1.smithi192.stdout:2/591: dwrite d4/d9/f5b [0,4194304] 0 2024-04-24T22:45:32.346 INFO:tasks.workunit.client.0.smithi063.stdout:0/553: dread d6/d64/d24/f72 [0,4194304] 0 2024-04-24T22:45:32.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/599: mkdir d5/d11/d23/d2d/d62/d73/dcd 0 2024-04-24T22:45:32.352 INFO:tasks.workunit.client.1.smithi192.stdout:4/415: rename d2/d9/d25/d2e/d31/d36/d5b/f6a to d2/d9/d25/f8e 0 2024-04-24T22:45:32.355 INFO:tasks.workunit.client.0.smithi063.stdout:4/568: dread d2/d8/d40/f68 [0,4194304] 0 2024-04-24T22:45:32.356 INFO:tasks.workunit.client.1.smithi192.stdout:0/511: dwrite d3/d27/d3c/d64/faa [0,4194304] 0 2024-04-24T22:45:32.357 INFO:tasks.workunit.client.1.smithi192.stdout:7/561: mkdir d2/db/d21/d42/d67/dae 0 2024-04-24T22:45:32.359 INFO:tasks.workunit.client.1.smithi192.stdout:7/562: chown d2/db/d25/d82/c95 24313 1 2024-04-24T22:45:32.360 INFO:tasks.workunit.client.0.smithi063.stdout:4/569: dread d2/d8/f3e [0,4194304] 0 2024-04-24T22:45:32.361 INFO:tasks.workunit.client.0.smithi063.stdout:4/570: fdatasync d2/d8/d40/d51/f66 0 2024-04-24T22:45:32.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/592: creat d4/d9/d10/da7/fbc x:0 0 0 2024-04-24T22:45:32.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/574: rename d3/dd/de/d17/d89/d8b/da8 to d3/dd/de/dba/dbc/dca 0 2024-04-24T22:45:32.370 INFO:tasks.workunit.client.1.smithi192.stdout:4/416: symlink d2/d9/d25/d2e/l8f 0 2024-04-24T22:45:32.381 INFO:tasks.workunit.client.1.smithi192.stdout:3/474: rename d2/d89 to d2/d4/d12/d1a/d1d/d55/d95 0 2024-04-24T22:45:32.382 INFO:tasks.workunit.client.1.smithi192.stdout:3/475: chown d2/d4/d7/d4b/c59 4 1 2024-04-24T22:45:32.385 INFO:tasks.workunit.client.0.smithi063.stdout:1/682: link d3/da/da2/dba/fc4 d3/db5/d21/d49/fe2 0 2024-04-24T22:45:32.385 INFO:tasks.workunit.client.0.smithi063.stdout:6/626: dread db/d1d/d20/f71 [0,4194304] 0 2024-04-24T22:45:32.385 INFO:tasks.workunit.client.0.smithi063.stdout:3/633: dread da/d18/d47/f6c [0,4194304] 0 2024-04-24T22:45:32.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/563: symlink d2/db/d21/d42/d5b/d63/d80/laf 0 2024-04-24T22:45:32.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/684: getdents d0/d2/d2d/d31/d89/d1c/d3f/dca 0 2024-04-24T22:45:32.389 INFO:tasks.workunit.client.0.smithi063.stdout:5/600: dwrite f0 [0,4194304] 0 2024-04-24T22:45:32.391 INFO:tasks.workunit.client.0.smithi063.stdout:5/601: chown d5/d24/d44/fa5 18360 1 2024-04-24T22:45:32.391 INFO:tasks.workunit.client.0.smithi063.stdout:9/620: dread d6/d8/d1f/d20/d22/f3b [0,4194304] 0 2024-04-24T22:45:32.394 INFO:tasks.workunit.client.0.smithi063.stdout:9/621: write d6/d8/d1f/d20/f21 [501831,101550] 0 2024-04-24T22:45:32.394 INFO:tasks.workunit.client.1.smithi192.stdout:4/417: dread d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:32.395 INFO:tasks.workunit.client.0.smithi063.stdout:8/618: getdents d1/d22/d33/da6/d91 0 2024-04-24T22:45:32.396 INFO:tasks.workunit.client.1.smithi192.stdout:4/418: stat d2/d9/d25/d2e/d31/c41 0 2024-04-24T22:45:32.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/476: mknod d2/d4/d12/d1a/d1d/d71/c96 0 2024-04-24T22:45:32.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/683: symlink d3/de0/d79/le3 0 2024-04-24T22:45:32.401 INFO:tasks.workunit.client.0.smithi063.stdout:9/622: dwrite d6/d2f/d42/d81/f95 [0,4194304] 0 2024-04-24T22:45:32.403 INFO:tasks.workunit.client.1.smithi192.stdout:2/593: creat d4/fbd x:0 0 0 2024-04-24T22:45:32.405 INFO:tasks.workunit.client.1.smithi192.stdout:0/512: link d3/d27/d28/c95 d3/d27/d28/d2e/d4a/d5c/d80/dac/cb1 0 2024-04-24T22:45:32.406 INFO:tasks.workunit.client.0.smithi063.stdout:3/634: creat da/d18/d3a/d63/fda x:0 0 0 2024-04-24T22:45:32.410 INFO:tasks.workunit.client.1.smithi192.stdout:4/419: mkdir d2/d9/d25/d90 0 2024-04-24T22:45:32.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/575: getdents d3/d5d/da1/d85 0 2024-04-24T22:45:32.414 INFO:tasks.workunit.client.0.smithi063.stdout:3/635: dwrite da/d18/d3a/d63/fda [0,4194304] 0 2024-04-24T22:45:32.414 INFO:tasks.workunit.client.0.smithi063.stdout:5/602: dread d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:32.415 INFO:tasks.workunit.client.1.smithi192.stdout:2/594: rename d4/d9/d37/f44 to d4/d29/fbe 0 2024-04-24T22:45:32.417 INFO:tasks.workunit.client.1.smithi192.stdout:0/513: symlink d3/d27/d28/d2e/d3f/lb2 0 2024-04-24T22:45:32.419 INFO:tasks.workunit.client.1.smithi192.stdout:2/595: read d4/d9/d37/f64 [2556349,26063] 0 2024-04-24T22:45:32.421 INFO:tasks.workunit.client.1.smithi192.stdout:4/420: stat d2/d9/l4b 0 2024-04-24T22:45:32.421 INFO:tasks.workunit.client.1.smithi192.stdout:2/596: write d4/d9/f52 [1687774,92350] 0 2024-04-24T22:45:32.424 INFO:tasks.workunit.client.1.smithi192.stdout:7/564: dread d2/f1a [0,4194304] 0 2024-04-24T22:45:32.427 INFO:tasks.workunit.client.1.smithi192.stdout:3/477: dread d2/d4/d7/d4f/f52 [0,4194304] 0 2024-04-24T22:45:32.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/619: dread d1/d11/f63 [0,4194304] 0 2024-04-24T22:45:32.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/571: truncate d2/d8/f1a 1964802 0 2024-04-24T22:45:32.431 INFO:tasks.workunit.client.0.smithi063.stdout:1/684: mknod d3/db5/d21/d9f/ce4 0 2024-04-24T22:45:32.434 INFO:tasks.workunit.client.1.smithi192.stdout:0/514: rename d3/d27/d3c/d5d to d3/d27/d3c/d6c/db3 0 2024-04-24T22:45:32.436 INFO:tasks.workunit.client.0.smithi063.stdout:4/572: dwrite d2/d19/f20 [0,4194304] 0 2024-04-24T22:45:32.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/623: unlink d6/d8/db/d6b/f1e 0 2024-04-24T22:45:32.437 INFO:tasks.workunit.client.0.smithi063.stdout:4/573: chown d2/d19/f37 0 1 2024-04-24T22:45:32.437 INFO:tasks.workunit.client.0.smithi063.stdout:4/574: chown d2/d19/d82 72 1 2024-04-24T22:45:32.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/627: creat db/d1c/dc2/fd3 x:0 0 0 2024-04-24T22:45:32.440 INFO:tasks.workunit.client.0.smithi063.stdout:6/628: stat db/d1d/d2d/d48 0 2024-04-24T22:45:32.441 INFO:tasks.workunit.client.0.smithi063.stdout:7/546: sync 2024-04-24T22:45:32.442 INFO:tasks.workunit.client.0.smithi063.stdout:2/685: dwrite d0/d2/d63/f71 [0,4194304] 0 2024-04-24T22:45:32.443 INFO:tasks.workunit.client.0.smithi063.stdout:7/547: read d0/d4/de/d17/d35/d69/f7b [1945690,41958] 0 2024-04-24T22:45:32.444 INFO:tasks.workunit.client.0.smithi063.stdout:2/686: write d0/d2/d2d/d31/d89/d7/f53 [80639,87035] 0 2024-04-24T22:45:32.452 INFO:tasks.workunit.client.1.smithi192.stdout:9/576: dwrite d3/dd/de/dba/f81 [0,4194304] 0 2024-04-24T22:45:32.457 INFO:tasks.workunit.client.0.smithi063.stdout:3/636: rename da/d39/d94/db3/fc9 to da/d18/d3a/da1/fdb 0 2024-04-24T22:45:32.461 INFO:tasks.workunit.client.1.smithi192.stdout:2/597: mkdir d4/d9/d10/d6c/d95/db7/dbf 0 2024-04-24T22:45:32.465 INFO:tasks.workunit.client.1.smithi192.stdout:7/565: dwrite d2/db/d21/f22 [4194304,4194304] 0 2024-04-24T22:45:32.471 INFO:tasks.workunit.client.1.smithi192.stdout:3/478: dwrite d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:32.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/603: dwrite d5/d1d/dc0/dbb/f55 [4194304,4194304] 0 2024-04-24T22:45:32.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/421: rename d2/d9/d25/f5a to d2/d9/d25/d2e/d31/d36/d5b/d64/f91 0 2024-04-24T22:45:32.479 INFO:tasks.workunit.client.1.smithi192.stdout:0/515: fsync d3/d27/d28/f17 0 2024-04-24T22:45:32.479 INFO:tasks.workunit.client.1.smithi192.stdout:1/413: sync 2024-04-24T22:45:32.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/624: dwrite d6/f30 [0,4194304] 0 2024-04-24T22:45:32.481 INFO:tasks.workunit.client.0.smithi063.stdout:6/629: mkdir db/d1d/d2d/d69/dd4 0 2024-04-24T22:45:32.485 INFO:tasks.workunit.client.1.smithi192.stdout:2/598: mkdir d4/d9/d10/d6c/d95/dc0 0 2024-04-24T22:45:32.485 INFO:tasks.workunit.client.1.smithi192.stdout:2/599: dread - d4/d9/fa4 zero size 2024-04-24T22:45:32.487 INFO:tasks.workunit.client.1.smithi192.stdout:2/600: read - d4/d9/d37/f91 zero size 2024-04-24T22:45:32.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/630: dwrite db/d1d/d2d/fc1 [0,4194304] 0 2024-04-24T22:45:32.488 INFO:tasks.workunit.client.1.smithi192.stdout:9/577: creat d3/dd/de/da7/fcb x:0 0 0 2024-04-24T22:45:32.490 INFO:tasks.workunit.client.0.smithi063.stdout:2/687: creat d0/dd/fcc x:0 0 0 2024-04-24T22:45:32.491 INFO:tasks.workunit.client.0.smithi063.stdout:2/688: chown d0/dd/d62/d7c/f61 16 1 2024-04-24T22:45:32.492 INFO:tasks.workunit.client.1.smithi192.stdout:4/422: truncate d2/d9/d25/d2e/d31/f4a 605641 0 2024-04-24T22:45:32.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/554: creat d6/d64/d21/d25/d62/d63/d89/d95/da4/fc2 x:0 0 0 2024-04-24T22:45:32.495 INFO:tasks.workunit.client.1.smithi192.stdout:0/516: unlink d3/d27/d3c/d64/f8f 0 2024-04-24T22:45:32.497 INFO:tasks.workunit.client.0.smithi063.stdout:8/620: mkdir d1/d11/dcb 0 2024-04-24T22:45:32.499 INFO:tasks.workunit.client.0.smithi063.stdout:1/685: creat d3/da/d34/d99/fe5 x:0 0 0 2024-04-24T22:45:32.503 INFO:tasks.workunit.client.1.smithi192.stdout:9/578: creat d3/d5d/fcc x:0 0 0 2024-04-24T22:45:32.504 INFO:tasks.workunit.client.0.smithi063.stdout:5/604: mknod d5/d24/d44/cce 0 2024-04-24T22:45:32.504 INFO:tasks.workunit.client.0.smithi063.stdout:5/605: stat d5/d11/d23/f69 0 2024-04-24T22:45:32.506 INFO:tasks.workunit.client.1.smithi192.stdout:0/517: dwrite d3/d6d/f91 [0,4194304] 0 2024-04-24T22:45:32.506 INFO:tasks.workunit.client.1.smithi192.stdout:1/414: dwrite d9/dd/f29 [4194304,4194304] 0 2024-04-24T22:45:32.508 INFO:tasks.workunit.client.1.smithi192.stdout:0/518: readlink d3/d27/d28/d2e/d3f/lb2 0 2024-04-24T22:45:32.511 INFO:tasks.workunit.client.0.smithi063.stdout:2/689: rename d0/d2/d6c/db8 to d0/dd/dcd 0 2024-04-24T22:45:32.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/631: dread db/d1d/d2d/f42 [0,4194304] 0 2024-04-24T22:45:32.512 INFO:tasks.workunit.client.0.smithi063.stdout:6/632: read - db/d1c/dc2/fd3 zero size 2024-04-24T22:45:32.513 INFO:tasks.workunit.client.1.smithi192.stdout:3/479: rename d2/d4/d4e/d8a to d2/d4/d12/d39/d97 0 2024-04-24T22:45:32.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/621: chown d1/d22/d33/da6/d54/d7b/la4 136 1 2024-04-24T22:45:32.520 INFO:tasks.workunit.client.0.smithi063.stdout:1/686: mkdir d3/da/da2/de6 0 2024-04-24T22:45:32.523 INFO:tasks.workunit.client.0.smithi063.stdout:4/575: sync 2024-04-24T22:45:32.523 INFO:tasks.workunit.client.1.smithi192.stdout:7/566: getdents d2/db/d2a 0 2024-04-24T22:45:32.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/687: dread d3/db5/d21/d9f/daa/db7/fce [0,4194304] 0 2024-04-24T22:45:32.528 INFO:tasks.workunit.client.1.smithi192.stdout:3/480: rmdir d2/d4/d7 39 2024-04-24T22:45:32.529 INFO:tasks.workunit.client.1.smithi192.stdout:2/601: write d4/d9/d37/f64 [3171881,24072] 0 2024-04-24T22:45:32.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/423: rmdir d2/d9/d25/d90 0 2024-04-24T22:45:32.533 INFO:tasks.workunit.client.1.smithi192.stdout:9/579: mknod d3/d59/d7d/dc6/ccd 0 2024-04-24T22:45:32.534 INFO:tasks.workunit.client.0.smithi063.stdout:2/690: fdatasync d0/dd/d62/d7c/f61 0 2024-04-24T22:45:32.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/567: write d2/f17 [878839,52150] 0 2024-04-24T22:45:32.538 INFO:tasks.workunit.client.0.smithi063.stdout:8/622: rename d1/c3e to d1/d11/d32/d38/ccc 0 2024-04-24T22:45:32.542 INFO:tasks.workunit.client.0.smithi063.stdout:6/633: dread db/f1a [4194304,4194304] 0 2024-04-24T22:45:32.543 INFO:tasks.workunit.client.0.smithi063.stdout:3/637: truncate da/d18/f27 1043000 0 2024-04-24T22:45:32.544 INFO:tasks.workunit.client.1.smithi192.stdout:3/481: read - d2/d4/d7/d4b/f8d zero size 2024-04-24T22:45:32.544 INFO:tasks.workunit.client.1.smithi192.stdout:3/482: readlink d2/d27/d28/d38/l8b 0 2024-04-24T22:45:32.544 INFO:tasks.workunit.client.0.smithi063.stdout:8/623: dwrite d1/d22/d33/da6/d54/f71 [4194304,4194304] 0 2024-04-24T22:45:32.545 INFO:tasks.workunit.client.0.smithi063.stdout:8/624: fdatasync d1/d2/d3/f82 0 2024-04-24T22:45:32.546 INFO:tasks.workunit.client.1.smithi192.stdout:2/602: read d4/d9/d10/f5d [254548,113080] 0 2024-04-24T22:45:32.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/576: creat d2/d19/d82/fb8 x:0 0 0 2024-04-24T22:45:32.548 INFO:tasks.workunit.client.1.smithi192.stdout:2/603: write d4/d9/fb [1533566,63387] 0 2024-04-24T22:45:32.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/688: rmdir d3/da/da2/dba 39 2024-04-24T22:45:32.556 INFO:tasks.workunit.client.0.smithi063.stdout:7/548: sync 2024-04-24T22:45:32.557 INFO:tasks.workunit.client.1.smithi192.stdout:2/604: dwrite d4/d9/d37/f9c [0,4194304] 0 2024-04-24T22:45:32.559 INFO:tasks.workunit.client.1.smithi192.stdout:9/580: dread d3/dd/d53/d60/faa [0,4194304] 0 2024-04-24T22:45:32.561 INFO:tasks.workunit.client.1.smithi192.stdout:4/424: dread d2/d9/d25/d2e/d31/d36/d5b/f47 [0,4194304] 0 2024-04-24T22:45:32.564 INFO:tasks.workunit.client.0.smithi063.stdout:5/606: truncate d5/d24/d36/f63 163802 0 2024-04-24T22:45:32.566 INFO:tasks.workunit.client.0.smithi063.stdout:2/691: unlink d0/dd/d62/f6e 0 2024-04-24T22:45:32.567 INFO:tasks.workunit.client.0.smithi063.stdout:1/689: dread d3/db5/f71 [0,4194304] 0 2024-04-24T22:45:32.567 INFO:tasks.workunit.client.0.smithi063.stdout:2/692: write d0/d2/fa1 [152111,114885] 0 2024-04-24T22:45:32.569 INFO:tasks.workunit.client.0.smithi063.stdout:6/634: creat db/da0/fd5 x:0 0 0 2024-04-24T22:45:32.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/635: fsync db/d1d/d20/d29/d5d/f99 0 2024-04-24T22:45:32.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/415: rename d9/d15/d2d/d31/c22 to d9/c87 0 2024-04-24T22:45:32.572 INFO:tasks.workunit.client.0.smithi063.stdout:9/625: getdents d6 0 2024-04-24T22:45:32.574 INFO:tasks.workunit.client.0.smithi063.stdout:3/638: mknod da/d18/d47/d4d/d62/cdc 0 2024-04-24T22:45:32.576 INFO:tasks.workunit.client.1.smithi192.stdout:3/483: mknod d2/d4/d12/d39/d86/c98 0 2024-04-24T22:45:32.578 INFO:tasks.workunit.client.0.smithi063.stdout:8/625: creat d1/d22/d33/da6/d54/d7b/fcd x:0 0 0 2024-04-24T22:45:32.582 INFO:tasks.workunit.client.0.smithi063.stdout:7/549: rename d0/f73 to d0/d4/de/d17/d35/d69/fbc 0 2024-04-24T22:45:32.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/484: dwrite d2/d4/d12/d65/f66 [0,4194304] 0 2024-04-24T22:45:32.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/485: fsync d2/d4/d7/f87 0 2024-04-24T22:45:32.584 INFO:tasks.workunit.client.0.smithi063.stdout:0/555: getdents d6/d64/d21/d9c 0 2024-04-24T22:45:32.585 INFO:tasks.workunit.client.1.smithi192.stdout:2/605: write d4/d29/f87 [251676,50380] 0 2024-04-24T22:45:32.586 INFO:tasks.workunit.client.1.smithi192.stdout:3/486: chown d2/d4/d7/fd 1030 1 2024-04-24T22:45:32.589 INFO:tasks.workunit.client.1.smithi192.stdout:0/519: rename d3/l92 to d3/d27/d28/d2e/d4a/d5c/d80/dac/lb4 0 2024-04-24T22:45:32.591 INFO:tasks.workunit.client.1.smithi192.stdout:9/581: mkdir d3/dd/dce 0 2024-04-24T22:45:32.593 INFO:tasks.workunit.client.0.smithi063.stdout:1/690: write d3/da/da2/dba/d11/f46 [4948198,23954] 0 2024-04-24T22:45:32.597 INFO:tasks.workunit.client.0.smithi063.stdout:3/639: symlink da/d39/d94/dbc/ldd 0 2024-04-24T22:45:32.598 INFO:tasks.workunit.client.0.smithi063.stdout:8/626: creat d1/d11/d32/d38/db7/fce x:0 0 0 2024-04-24T22:45:32.599 INFO:tasks.workunit.client.1.smithi192.stdout:1/416: creat d9/dd/d74/f88 x:0 0 0 2024-04-24T22:45:32.600 INFO:tasks.workunit.client.1.smithi192.stdout:3/487: dread d2/f1e [0,4194304] 0 2024-04-24T22:45:32.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/627: dread - d1/d11/d3c/faf zero size 2024-04-24T22:45:32.600 INFO:tasks.workunit.client.0.smithi063.stdout:1/691: dwrite d3/da/d34/d6d/f92 [0,4194304] 0 2024-04-24T22:45:32.602 INFO:tasks.workunit.client.0.smithi063.stdout:8/628: chown d1/l53 133 1 2024-04-24T22:45:32.606 INFO:tasks.workunit.client.1.smithi192.stdout:9/582: mkdir d3/dd/d53/dab/dcf 0 2024-04-24T22:45:32.606 INFO:tasks.workunit.client.1.smithi192.stdout:9/583: stat d3/f8d 0 2024-04-24T22:45:32.608 INFO:tasks.workunit.client.0.smithi063.stdout:7/550: mkdir d0/d4/de/d6f/d2e/d76/d9f/dbd 0 2024-04-24T22:45:32.612 INFO:tasks.workunit.client.0.smithi063.stdout:0/556: creat d6/d64/da0/d33/fc3 x:0 0 0 2024-04-24T22:45:32.614 INFO:tasks.workunit.client.1.smithi192.stdout:7/568: link d2/db/d21/d42/d67/d9b/d6e/f9e d2/db/d21/d42/fb0 0 2024-04-24T22:45:32.614 INFO:tasks.workunit.client.0.smithi063.stdout:6/636: write db/ff [4973250,124415] 0 2024-04-24T22:45:32.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/637: chown db/d1d/d2d/d48/f6e 4100090 1 2024-04-24T22:45:32.619 INFO:tasks.workunit.client.1.smithi192.stdout:5/404: sync 2024-04-24T22:45:32.620 INFO:tasks.workunit.client.1.smithi192.stdout:3/488: symlink d2/d4/d12/d1a/l99 0 2024-04-24T22:45:32.621 INFO:tasks.workunit.client.1.smithi192.stdout:2/606: dwrite d4/d9/d10/d6c/f8e [0,4194304] 0 2024-04-24T22:45:32.621 INFO:tasks.workunit.client.1.smithi192.stdout:5/405: read d0/f31 [438025,22234] 0 2024-04-24T22:45:32.622 INFO:tasks.workunit.client.0.smithi063.stdout:3/640: chown da/c2b 302 1 2024-04-24T22:45:32.623 INFO:tasks.workunit.client.0.smithi063.stdout:4/577: dwrite d2/f6 [0,4194304] 0 2024-04-24T22:45:32.624 INFO:tasks.workunit.client.0.smithi063.stdout:4/578: write d2/d8/f5b [930463,13084] 0 2024-04-24T22:45:32.624 INFO:tasks.workunit.client.1.smithi192.stdout:4/425: getdents d2/d9/d25/d2e/d31 0 2024-04-24T22:45:32.626 INFO:tasks.workunit.client.1.smithi192.stdout:7/569: creat d2/db/d25/d82/fb1 x:0 0 0 2024-04-24T22:45:32.627 INFO:tasks.workunit.client.1.smithi192.stdout:7/570: chown d2/db/c81 26033438 1 2024-04-24T22:45:32.628 INFO:tasks.workunit.client.0.smithi063.stdout:7/551: truncate d0/d4/de/d6f/d5c/d63/f84 2622857 0 2024-04-24T22:45:32.629 INFO:tasks.workunit.client.1.smithi192.stdout:1/417: mknod d9/d15/d2d/d63/d68/c89 0 2024-04-24T22:45:32.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/607: link d5/d24/d36/l98 d5/d1d/d7d/lcf 0 2024-04-24T22:45:32.638 INFO:tasks.workunit.client.1.smithi192.stdout:3/489: mkdir d2/d4/d12/d1a/d9a 0 2024-04-24T22:45:32.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/584: write d3/dd/f38 [3819715,82823] 0 2024-04-24T22:45:32.642 INFO:tasks.workunit.client.1.smithi192.stdout:9/585: write d3/f45 [598672,103124] 0 2024-04-24T22:45:32.643 INFO:tasks.workunit.client.1.smithi192.stdout:2/607: creat d4/d9/d10/d6c/d6f/da8/fc1 x:0 0 0 2024-04-24T22:45:32.645 INFO:tasks.workunit.client.0.smithi063.stdout:9/626: rename d6/d8/d53/dbe/la3 to d6/d8/d1f/d20/d2a/d4b/d5e/lc9 0 2024-04-24T22:45:32.645 INFO:tasks.workunit.client.0.smithi063.stdout:7/552: symlink d0/d4/de/d6f/d5c/lbe 0 2024-04-24T22:45:32.646 INFO:tasks.workunit.client.1.smithi192.stdout:4/426: creat d2/db/d5d/d77/f92 x:0 0 0 2024-04-24T22:45:32.649 INFO:tasks.workunit.client.0.smithi063.stdout:9/627: dread d6/d8/d1f/f52 [0,4194304] 0 2024-04-24T22:45:32.650 INFO:tasks.workunit.client.1.smithi192.stdout:7/571: rename d2/db/d25/d82 to d2/db/d21/d42/d67/d9b/db2 0 2024-04-24T22:45:32.654 INFO:tasks.workunit.client.0.smithi063.stdout:6/638: truncate db/f2a 1129523 0 2024-04-24T22:45:32.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/579: write d2/f74 [3078613,119820] 0 2024-04-24T22:45:32.654 INFO:tasks.workunit.client.1.smithi192.stdout:0/520: link d3/d27/d28/d31/f3a d3/d27/d28/d31/fb5 0 2024-04-24T22:45:32.656 INFO:tasks.workunit.client.0.smithi063.stdout:3/641: truncate da/d18/d3a/f3f 1086724 0 2024-04-24T22:45:32.657 INFO:tasks.workunit.client.1.smithi192.stdout:3/490: mkdir d2/d4/d12/d1a/d1d/d55/d5f/d9b 0 2024-04-24T22:45:32.661 INFO:tasks.workunit.client.0.smithi063.stdout:1/692: rename d3/da/l4b to d3/db5/d21/d9f/le7 0 2024-04-24T22:45:32.664 INFO:tasks.workunit.client.0.smithi063.stdout:0/557: creat d6/d64/d21/d25/d31/da3/fc4 x:0 0 0 2024-04-24T22:45:32.667 INFO:tasks.workunit.client.0.smithi063.stdout:5/608: creat d5/d24/d49/d3b/dc5/fd0 x:0 0 0 2024-04-24T22:45:32.668 INFO:tasks.workunit.client.0.smithi063.stdout:6/639: rmdir db 39 2024-04-24T22:45:32.671 INFO:tasks.workunit.client.1.smithi192.stdout:3/491: truncate d2/d4/d12/d1a/d1d/f3f 2689136 0 2024-04-24T22:45:32.672 INFO:tasks.workunit.client.0.smithi063.stdout:4/580: creat d2/d80/d81/d90/fb9 x:0 0 0 2024-04-24T22:45:32.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/406: creat d0/d13/d19/d83/f87 x:0 0 0 2024-04-24T22:45:32.678 INFO:tasks.workunit.client.0.smithi063.stdout:2/693: sync 2024-04-24T22:45:32.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/586: mknod d3/dd/de/d17/d89/db4/db9/cd0 0 2024-04-24T22:45:32.680 INFO:tasks.workunit.client.1.smithi192.stdout:2/608: dread d4/f5e [0,4194304] 0 2024-04-24T22:45:32.685 INFO:tasks.workunit.client.1.smithi192.stdout:4/427: dwrite d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f34 [0,4194304] 0 2024-04-24T22:45:32.687 INFO:tasks.workunit.client.1.smithi192.stdout:0/521: mknod d3/d27/d28/d2e/d4a/cb6 0 2024-04-24T22:45:32.690 INFO:tasks.workunit.client.0.smithi063.stdout:0/558: creat d6/d3a/fc5 x:0 0 0 2024-04-24T22:45:32.691 INFO:tasks.workunit.client.1.smithi192.stdout:3/492: mknod d2/d27/d28/d38/c9c 0 2024-04-24T22:45:32.692 INFO:tasks.workunit.client.1.smithi192.stdout:5/407: fdatasync d0/d13/d19/d64/d5f/f20 0 2024-04-24T22:45:32.693 INFO:tasks.workunit.client.0.smithi063.stdout:9/628: mkdir d6/d8/db/dca 0 2024-04-24T22:45:32.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/587: mkdir d3/d59/d7d/d9b/dd1 0 2024-04-24T22:45:32.696 INFO:tasks.workunit.client.1.smithi192.stdout:9/588: write d3/dd/fbe [1592725,66869] 0 2024-04-24T22:45:32.700 INFO:tasks.workunit.client.0.smithi063.stdout:8/629: sync 2024-04-24T22:45:32.700 INFO:tasks.workunit.client.1.smithi192.stdout:1/418: link d9/dd/l5a d9/d15/d5e/l8a 0 2024-04-24T22:45:32.701 INFO:tasks.workunit.client.1.smithi192.stdout:9/589: dread d3/d5d/f7c [4194304,4194304] 0 2024-04-24T22:45:32.701 INFO:tasks.workunit.client.0.smithi063.stdout:7/553: truncate d0/d4/de/f7c 2654857 0 2024-04-24T22:45:32.705 INFO:tasks.workunit.client.0.smithi063.stdout:2/694: creat d0/dd/d72/d77/fce x:0 0 0 2024-04-24T22:45:32.707 INFO:tasks.workunit.client.0.smithi063.stdout:7/554: dwrite d0/d4/f11 [0,4194304] 0 2024-04-24T22:45:32.707 INFO:tasks.workunit.client.1.smithi192.stdout:0/522: unlink d3/l4 0 2024-04-24T22:45:32.716 INFO:tasks.workunit.client.1.smithi192.stdout:5/408: mkdir d0/d13/d19/d71/d88 0 2024-04-24T22:45:32.717 INFO:tasks.workunit.client.0.smithi063.stdout:4/581: dwrite d2/d8/d36/f6b [0,4194304] 0 2024-04-24T22:45:32.719 INFO:tasks.workunit.client.0.smithi063.stdout:5/609: link f0 d5/d11/d23/d2d/fd1 0 2024-04-24T22:45:32.720 INFO:tasks.workunit.client.0.smithi063.stdout:1/693: write d3/db5/d21/d9f/daa/db7/f58 [5729337,52474] 0 2024-04-24T22:45:32.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/629: rename d6/d8/d1f/d20/d22/d31 to d6/d8/d1f/d20/d2a/dcb 0 2024-04-24T22:45:32.724 INFO:tasks.workunit.client.0.smithi063.stdout:4/582: truncate d2/d8/d36/f44 4838782 0 2024-04-24T22:45:32.725 INFO:tasks.workunit.client.0.smithi063.stdout:3/642: dwrite da/d18/f27 [0,4194304] 0 2024-04-24T22:45:32.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/640: fdatasync db/f1a 0 2024-04-24T22:45:32.727 INFO:tasks.workunit.client.0.smithi063.stdout:1/694: stat d3/db5/d21/d9f/daa/db7/c7e 0 2024-04-24T22:45:32.728 INFO:tasks.workunit.client.0.smithi063.stdout:1/695: chown d3/da/d34 3110 1 2024-04-24T22:45:32.730 INFO:tasks.workunit.client.0.smithi063.stdout:8/630: unlink d1/d11/c76 0 2024-04-24T22:45:32.732 INFO:tasks.workunit.client.0.smithi063.stdout:7/555: dread d0/d4/de/d17/d28/f64 [0,4194304] 0 2024-04-24T22:45:32.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/428: mknod d2/d9/d25/c93 0 2024-04-24T22:45:32.734 INFO:tasks.workunit.client.1.smithi192.stdout:3/493: dwrite d2/d4/d7/fd [0,4194304] 0 2024-04-24T22:45:32.735 INFO:tasks.workunit.client.0.smithi063.stdout:4/583: dwrite d2/d8/d36/d3a/f9b [0,4194304] 0 2024-04-24T22:45:32.735 INFO:tasks.workunit.client.0.smithi063.stdout:4/584: chown d2/d19/c61 621 1 2024-04-24T22:45:32.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/429: chown d2/d9/d25/f32 408221395 1 2024-04-24T22:45:32.738 INFO:tasks.workunit.client.1.smithi192.stdout:0/523: creat d3/d27/d28/d2e/d4a/d5c/d80/fb7 x:0 0 0 2024-04-24T22:45:32.745 INFO:tasks.workunit.client.1.smithi192.stdout:1/419: write d9/d15/f2e [761484,18823] 0 2024-04-24T22:45:32.752 INFO:tasks.workunit.client.1.smithi192.stdout:9/590: unlink d3/dd/de/f2f 0 2024-04-24T22:45:32.753 INFO:tasks.workunit.client.1.smithi192.stdout:9/591: chown d3/dd/de/dba/f66 6515500 1 2024-04-24T22:45:32.754 INFO:tasks.workunit.client.1.smithi192.stdout:3/494: creat d2/d27/f9d x:0 0 0 2024-04-24T22:45:32.756 INFO:tasks.workunit.client.1.smithi192.stdout:4/430: symlink d2/db/d10/l94 0 2024-04-24T22:45:32.760 INFO:tasks.workunit.client.1.smithi192.stdout:1/420: creat d9/d15/d2d/d4a/f8b x:0 0 0 2024-04-24T22:45:32.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/696: unlink d3/da/da2/dba/d11/d5e/d6f/faf 0 2024-04-24T22:45:32.762 INFO:tasks.workunit.client.1.smithi192.stdout:2/609: getdents d4 0 2024-04-24T22:45:32.764 INFO:tasks.workunit.client.1.smithi192.stdout:2/610: chown d4/d29/d60 525875 1 2024-04-24T22:45:32.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/431: dwrite d2/db/f7a [0,4194304] 0 2024-04-24T22:45:32.767 INFO:tasks.workunit.client.0.smithi063.stdout:2/695: symlink d0/d2/lcf 0 2024-04-24T22:45:32.767 INFO:tasks.workunit.client.0.smithi063.stdout:2/696: readlink d0/d2/d35/l41 0 2024-04-24T22:45:32.770 INFO:tasks.workunit.client.1.smithi192.stdout:5/409: creat d0/d1b/f89 x:0 0 0 2024-04-24T22:45:32.772 INFO:tasks.workunit.client.0.smithi063.stdout:2/697: dwrite d0/d2/d27/f85 [0,4194304] 0 2024-04-24T22:45:32.773 INFO:tasks.workunit.client.1.smithi192.stdout:9/592: dwrite d3/d5d/f61 [4194304,4194304] 0 2024-04-24T22:45:32.773 INFO:tasks.workunit.client.0.smithi063.stdout:2/698: readlink d0/d2/lcf 0 2024-04-24T22:45:32.779 INFO:tasks.workunit.client.1.smithi192.stdout:4/432: dwrite d2/db/f7a [0,4194304] 0 2024-04-24T22:45:32.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/559: truncate d6/da/dc/f1e 3126603 0 2024-04-24T22:45:32.791 INFO:tasks.workunit.client.0.smithi063.stdout:9/630: write d6/d8/d1f/d20/d22/f47 [783868,129744] 0 2024-04-24T22:45:32.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/631: dread - d6/d8/d53/f88 zero size 2024-04-24T22:45:32.795 INFO:tasks.workunit.client.0.smithi063.stdout:6/641: write db/d1d/d20/f40 [1213697,19171] 0 2024-04-24T22:45:32.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/593: readlink d3/dd/de/l1f 0 2024-04-24T22:45:32.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/611: dwrite d4/f5e [0,4194304] 0 2024-04-24T22:45:32.797 INFO:tasks.workunit.client.1.smithi192.stdout:1/421: rmdir d9/d15/d2d 39 2024-04-24T22:45:32.798 INFO:tasks.workunit.client.1.smithi192.stdout:6/431: sync 2024-04-24T22:45:32.798 INFO:tasks.workunit.client.1.smithi192.stdout:8/528: sync 2024-04-24T22:45:32.799 INFO:tasks.workunit.client.0.smithi063.stdout:9/632: dwrite d6/d8/d1f/d20/fbc [0,4194304] 0 2024-04-24T22:45:32.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/529: readlink db/l16 0 2024-04-24T22:45:32.801 INFO:tasks.workunit.client.1.smithi192.stdout:6/432: chown d0/d8/d18/d2e/f51 150489 1 2024-04-24T22:45:32.802 INFO:tasks.workunit.client.1.smithi192.stdout:6/433: chown d0/d8/f31 182931 1 2024-04-24T22:45:32.803 INFO:tasks.workunit.client.1.smithi192.stdout:4/433: mknod d2/d9/d25/d2e/d31/c95 0 2024-04-24T22:45:32.807 INFO:tasks.workunit.client.1.smithi192.stdout:3/495: rmdir d2/d4/d12/d1a/d9a 0 2024-04-24T22:45:32.807 INFO:tasks.workunit.client.1.smithi192.stdout:1/422: read f3 [29125,97492] 0 2024-04-24T22:45:32.808 INFO:tasks.workunit.client.0.smithi063.stdout:5/610: truncate d5/d24/d44/fbf 906342 0 2024-04-24T22:45:32.810 INFO:tasks.workunit.client.1.smithi192.stdout:8/530: dwrite db/dc/d22/f23 [0,4194304] 0 2024-04-24T22:45:32.811 INFO:tasks.workunit.client.1.smithi192.stdout:8/531: chown f9 41 1 2024-04-24T22:45:32.813 INFO:tasks.workunit.client.1.smithi192.stdout:0/524: getdents d3/d6d 0 2024-04-24T22:45:32.816 INFO:tasks.workunit.client.0.smithi063.stdout:2/699: unlink d0/d2/d63/fb0 0 2024-04-24T22:45:32.816 INFO:tasks.workunit.client.1.smithi192.stdout:0/525: truncate d3/d27/d28/d2e/d4a/d5c/d80/fb7 716031 0 2024-04-24T22:45:32.817 INFO:tasks.workunit.client.1.smithi192.stdout:9/594: mknod d3/dd/d53/cd2 0 2024-04-24T22:45:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:2/700: write d0/dd/f22 [3228492,56470] 0 2024-04-24T22:45:32.820 INFO:tasks.workunit.client.0.smithi063.stdout:6/642: rmdir db/d1c 39 2024-04-24T22:45:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:8/631: creat d1/d2/fcf x:0 0 0 2024-04-24T22:45:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:9/633: truncate d6/d8/d1f/d20/d22/f3b 3830425 0 2024-04-24T22:45:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/634: chown d6/d8/db/d6b/d4f 3604 1 2024-04-24T22:45:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:9/635: chown d6/d8/d1f/f52 0 1 2024-04-24T22:45:32.829 INFO:tasks.workunit.client.1.smithi192.stdout:3/496: creat d2/d4/d12/d39/d97/f9e x:0 0 0 2024-04-24T22:45:32.830 INFO:tasks.workunit.client.1.smithi192.stdout:3/497: fdatasync d2/d4/d12/d1a/d1d/d71/f85 0 2024-04-24T22:45:32.831 INFO:tasks.workunit.client.1.smithi192.stdout:1/423: write d9/d15/d2d/f6c [509019,83185] 0 2024-04-24T22:45:32.832 INFO:tasks.workunit.client.0.smithi063.stdout:4/585: rename d2/d8/f1d to d2/d8/d36/fba 0 2024-04-24T22:45:32.833 INFO:tasks.workunit.client.1.smithi192.stdout:8/532: rename db/d43/l55 to db/d43/d71/la5 0 2024-04-24T22:45:32.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/612: link d4/d9/d37/f8d d4/d9/d9f/dba/fc2 0 2024-04-24T22:45:32.840 INFO:tasks.workunit.client.1.smithi192.stdout:6/434: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:32.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/643: getdents da/d39/d94/dbc 0 2024-04-24T22:45:32.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/644: chown da/d39/f7a 1322555 1 2024-04-24T22:45:32.845 INFO:tasks.workunit.client.0.smithi063.stdout:2/701: rmdir d0/d2/d27/d3a 39 2024-04-24T22:45:32.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/643: read db/d1c/f89 [423619,17199] 0 2024-04-24T22:45:32.849 INFO:tasks.workunit.client.0.smithi063.stdout:0/560: sync 2024-04-24T22:45:32.853 INFO:tasks.workunit.client.0.smithi063.stdout:1/697: link d3/da/da2/dba/d39/d5a/lad d3/da/d34/d99/le8 0 2024-04-24T22:45:32.854 INFO:tasks.workunit.client.0.smithi063.stdout:1/698: write d3/da/da2/dba/d39/fda [540067,77293] 0 2024-04-24T22:45:32.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/595: rename d3/dd/d53/d60/faa to d3/dd/de/da7/fd3 0 2024-04-24T22:45:32.855 INFO:tasks.workunit.client.0.smithi063.stdout:7/556: getdents d0/d4/de/d6f/d5c/d63 0 2024-04-24T22:45:32.857 INFO:tasks.workunit.client.1.smithi192.stdout:8/533: mkdir db/dc/d22/d31/d33/d8f/da6 0 2024-04-24T22:45:32.858 INFO:tasks.workunit.client.1.smithi192.stdout:8/534: write f9 [3430013,124846] 0 2024-04-24T22:45:32.859 INFO:tasks.workunit.client.0.smithi063.stdout:4/586: symlink d2/d9f/lbb 0 2024-04-24T22:45:32.860 INFO:tasks.workunit.client.1.smithi192.stdout:8/535: truncate db/d43/d76/f94 510104 0 2024-04-24T22:45:32.861 INFO:tasks.workunit.client.1.smithi192.stdout:0/526: mknod d3/d27/d28/d2e/cb8 0 2024-04-24T22:45:32.862 INFO:tasks.workunit.client.0.smithi063.stdout:6/644: dread db/d1d/d2d/f36 [0,4194304] 0 2024-04-24T22:45:32.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/645: dread da/f36 [0,4194304] 0 2024-04-24T22:45:32.868 INFO:tasks.workunit.client.1.smithi192.stdout:6/435: rmdir d0/d8/d29/d4c 39 2024-04-24T22:45:32.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/557: dread d0/d4/de/f25 [0,4194304] 0 2024-04-24T22:45:32.870 INFO:tasks.workunit.client.0.smithi063.stdout:2/702: creat d0/dd/d62/d7c/fd0 x:0 0 0 2024-04-24T22:45:32.870 INFO:tasks.workunit.client.1.smithi192.stdout:6/436: read d0/f82 [5792070,107667] 0 2024-04-24T22:45:32.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/410: link d0/d13/d19/d64/d5f/d2c/f44 d0/d13/d19/d64/d5f/f8a 0 2024-04-24T22:45:32.872 INFO:tasks.workunit.client.0.smithi063.stdout:8/632: mknod d1/d11/d32/d38/db7/dc1/cd0 0 2024-04-24T22:45:32.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/561: symlink d6/d64/d21/d25/d31/da3/lc6 0 2024-04-24T22:45:32.877 INFO:tasks.workunit.client.0.smithi063.stdout:9/636: write d6/d8/d1f/f52 [5240746,73800] 0 2024-04-24T22:45:32.878 INFO:tasks.workunit.client.1.smithi192.stdout:1/424: mknod d9/d15/d2d/d78/c8c 0 2024-04-24T22:45:32.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/637: read - d6/d8/db/fc1 zero size 2024-04-24T22:45:32.879 INFO:tasks.workunit.client.1.smithi192.stdout:7/572: sync 2024-04-24T22:45:32.880 INFO:tasks.workunit.client.0.smithi063.stdout:1/699: rmdir d3/de0 39 2024-04-24T22:45:32.880 INFO:tasks.workunit.client.1.smithi192.stdout:7/573: readlink d2/db/d21/d42/l7f 0 2024-04-24T22:45:32.881 INFO:tasks.workunit.client.1.smithi192.stdout:7/574: truncate d2/f91 695714 0 2024-04-24T22:45:32.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/587: symlink d2/d8/d36/da3/lbc 0 2024-04-24T22:45:32.884 INFO:tasks.workunit.client.0.smithi063.stdout:4/588: readlink d2/d9f/lbb 0 2024-04-24T22:45:32.886 INFO:tasks.workunit.client.0.smithi063.stdout:5/611: creat d5/d24/d36/fd2 x:0 0 0 2024-04-24T22:45:32.891 INFO:tasks.workunit.client.0.smithi063.stdout:7/558: rename d0/d4/de/l6d to d0/d4/de/d17/d35/d6c/d7a/lbf 0 2024-04-24T22:45:32.892 INFO:tasks.workunit.client.1.smithi192.stdout:1/425: dread d9/dd/f2b [0,4194304] 0 2024-04-24T22:45:32.892 INFO:tasks.workunit.client.1.smithi192.stdout:8/536: write db/d43/f69 [1852322,44099] 0 2024-04-24T22:45:32.893 INFO:tasks.workunit.client.0.smithi063.stdout:2/703: mknod d0/d2/d2d/d31/d89/d1c/d3f/cd1 0 2024-04-24T22:45:32.894 INFO:tasks.workunit.client.1.smithi192.stdout:8/537: chown db/dc/d22/d31/d33/d77 4 1 2024-04-24T22:45:32.894 INFO:tasks.workunit.client.0.smithi063.stdout:2/704: stat d0/d2/d2d/fb6 0 2024-04-24T22:45:32.896 INFO:tasks.workunit.client.1.smithi192.stdout:5/411: fsync d0/d13/d19/d64/d5f/d2c/d3c/f5a 0 2024-04-24T22:45:32.897 INFO:tasks.workunit.client.1.smithi192.stdout:3/498: creat d2/f9f x:0 0 0 2024-04-24T22:45:32.898 INFO:tasks.workunit.client.0.smithi063.stdout:8/633: mkdir d1/d2/d17/dd1 0 2024-04-24T22:45:32.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/527: dwrite d3/d27/d28/d2e/d4a/f6b [0,4194304] 0 2024-04-24T22:45:32.903 INFO:tasks.workunit.client.0.smithi063.stdout:9/638: symlink d6/d8/d1f/d20/d2a/dcb/lcc 0 2024-04-24T22:45:32.904 INFO:tasks.workunit.client.1.smithi192.stdout:5/412: dwrite d0/d13/f86 [0,4194304] 0 2024-04-24T22:45:32.904 INFO:tasks.workunit.client.1.smithi192.stdout:0/528: chown d3/d27/d28/d2e/d4a/d5c/d80/fb7 29609272 1 2024-04-24T22:45:32.908 INFO:tasks.workunit.client.0.smithi063.stdout:1/700: symlink d3/db5/d21/d9f/daa/db7/le9 0 2024-04-24T22:45:32.909 INFO:tasks.workunit.client.0.smithi063.stdout:4/589: creat d2/d9f/fbd x:0 0 0 2024-04-24T22:45:32.913 INFO:tasks.workunit.client.0.smithi063.stdout:6/645: truncate db/d1d/d20/d3e/f5e 2285661 0 2024-04-24T22:45:32.913 INFO:tasks.workunit.client.0.smithi063.stdout:3/646: mkdir da/d18/d47/d4d/d9b/dde 0 2024-04-24T22:45:32.914 INFO:tasks.workunit.client.0.smithi063.stdout:3/647: fdatasync da/d40/f9c 0 2024-04-24T22:45:32.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/559: mkdir d0/d4/de/d6f/d5c/d8b/dc0 0 2024-04-24T22:45:32.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/705: creat d0/dd/d72/d77/d87/fd2 x:0 0 0 2024-04-24T22:45:32.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/560: readlink d0/d4/de/d6f/l7d 0 2024-04-24T22:45:32.921 INFO:tasks.workunit.client.0.smithi063.stdout:5/612: creat d5/d1d/dc7/fd3 x:0 0 0 2024-04-24T22:45:32.923 INFO:tasks.workunit.client.0.smithi063.stdout:4/590: mknod d2/d80/d8d/cbe 0 2024-04-24T22:45:32.927 INFO:tasks.workunit.client.0.smithi063.stdout:3/648: write da/d18/d47/f6a [82800,53560] 0 2024-04-24T22:45:32.928 INFO:tasks.workunit.client.1.smithi192.stdout:9/596: getdents d3/d59/d7d/d88/dc2 0 2024-04-24T22:45:32.929 INFO:tasks.workunit.client.1.smithi192.stdout:9/597: chown d3/dd/d62/cbb 390 1 2024-04-24T22:45:32.934 INFO:tasks.workunit.client.1.smithi192.stdout:7/575: creat d2/db/d2a/d57/fb3 x:0 0 0 2024-04-24T22:45:32.935 INFO:tasks.workunit.client.1.smithi192.stdout:7/576: chown d2/db/d21/l9d 3130319 1 2024-04-24T22:45:32.935 INFO:tasks.workunit.client.0.smithi063.stdout:5/613: creat d5/d11/d23/d2d/d62/d81/da4/fd4 x:0 0 0 2024-04-24T22:45:32.938 INFO:tasks.workunit.client.0.smithi063.stdout:4/591: unlink d2/d8/l6e 0 2024-04-24T22:45:32.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/562: write d6/da/f3d [3070235,30706] 0 2024-04-24T22:45:32.941 INFO:tasks.workunit.client.1.smithi192.stdout:7/577: dwrite d2/db/d2a/f61 [0,4194304] 0 2024-04-24T22:45:32.941 INFO:tasks.workunit.client.0.smithi063.stdout:3/649: creat da/d39/d94/db3/fdf x:0 0 0 2024-04-24T22:45:32.943 INFO:tasks.workunit.client.0.smithi063.stdout:3/650: chown da/d18/d9e/da3/dc1 0 1 2024-04-24T22:45:32.944 INFO:tasks.workunit.client.0.smithi063.stdout:4/592: dwrite d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:32.945 INFO:tasks.workunit.client.0.smithi063.stdout:2/706: unlink d0/d2/d2d/l4e 0 2024-04-24T22:45:32.947 INFO:tasks.workunit.client.0.smithi063.stdout:9/639: dwrite d6/d8/d1f/d20/d2a/d4b/d5e/fc7 [0,4194304] 0 2024-04-24T22:45:32.949 INFO:tasks.workunit.client.1.smithi192.stdout:1/426: creat d9/d15/d2d/d48/f8d x:0 0 0 2024-04-24T22:45:32.956 INFO:tasks.workunit.client.0.smithi063.stdout:6/646: creat db/d1d/d20/fd6 x:0 0 0 2024-04-24T22:45:32.957 INFO:tasks.workunit.client.0.smithi063.stdout:0/563: mkdir d6/d64/da0/d98/dc7 0 2024-04-24T22:45:32.959 INFO:tasks.workunit.client.1.smithi192.stdout:3/499: mkdir d2/d4/d7/da0 0 2024-04-24T22:45:32.961 INFO:tasks.workunit.client.0.smithi063.stdout:3/651: rename da/d18/d9e/da3/fab to da/d40/fe0 0 2024-04-24T22:45:32.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/593: creat d2/d19/d33/d9d/fbf x:0 0 0 2024-04-24T22:45:32.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/594: truncate d2/d19/d33/d9d/fbf 19421 0 2024-04-24T22:45:32.965 INFO:tasks.workunit.client.0.smithi063.stdout:2/707: chown d0/d2/d2d/d31/f5b 8062309 1 2024-04-24T22:45:32.965 INFO:tasks.workunit.client.0.smithi063.stdout:8/634: sync 2024-04-24T22:45:32.966 INFO:tasks.workunit.client.0.smithi063.stdout:2/708: fdatasync d0/d2/d6c/faf 0 2024-04-24T22:45:32.966 INFO:tasks.workunit.client.0.smithi063.stdout:4/595: write d2/d8/f71 [4126319,29261] 0 2024-04-24T22:45:32.966 INFO:tasks.workunit.client.1.smithi192.stdout:0/529: unlink d3/d27/d28/d2e/d4a/d5c/d8a/c93 0 2024-04-24T22:45:32.967 INFO:tasks.workunit.client.1.smithi192.stdout:9/598: creat d3/d5d/fd4 x:0 0 0 2024-04-24T22:45:32.968 INFO:tasks.workunit.client.1.smithi192.stdout:9/599: truncate d3/d5d/fd4 669186 0 2024-04-24T22:45:32.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/613: getdents d4 0 2024-04-24T22:45:32.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/640: rename d6/d8 to d6/d2f/dcd 0 2024-04-24T22:45:32.980 INFO:tasks.workunit.client.1.smithi192.stdout:7/578: unlink d2/db/d21/l40 0 2024-04-24T22:45:32.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/635: fdatasync d1/d22/f5b 0 2024-04-24T22:45:32.982 INFO:tasks.workunit.client.1.smithi192.stdout:6/437: creat d0/f94 x:0 0 0 2024-04-24T22:45:32.985 INFO:tasks.workunit.client.0.smithi063.stdout:4/596: rmdir d2/d8 39 2024-04-24T22:45:32.985 INFO:tasks.workunit.client.0.smithi063.stdout:4/597: dread - d2/d80/d81/d90/fb9 zero size 2024-04-24T22:45:32.989 INFO:tasks.workunit.client.0.smithi063.stdout:6/647: truncate db/d1d/f4c 827134 0 2024-04-24T22:45:32.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/636: dread d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:32.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/652: rename f7 to da/d18/d47/d4d/d9b/fe1 0 2024-04-24T22:45:32.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/653: chown da/d18/d47/d4d/c5b 35631352 1 2024-04-24T22:45:32.996 INFO:tasks.workunit.client.1.smithi192.stdout:0/530: symlink d3/d40/lb9 0 2024-04-24T22:45:32.997 INFO:tasks.workunit.client.0.smithi063.stdout:3/654: truncate da/d18/d3a/da1/fbd 907909 0 2024-04-24T22:45:32.998 INFO:tasks.workunit.client.0.smithi063.stdout:5/614: truncate d5/d24/d36/f6c 2820478 0 2024-04-24T22:45:32.998 INFO:tasks.workunit.client.1.smithi192.stdout:9/600: dread - d3/dd/de/d17/f8f zero size 2024-04-24T22:45:33.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/701: sync 2024-04-24T22:45:33.001 INFO:tasks.workunit.client.1.smithi192.stdout:2/614: rename d4/d9/d37/f4a to d4/d9/d10/d6c/d95/db7/fc3 0 2024-04-24T22:45:33.003 INFO:tasks.workunit.client.0.smithi063.stdout:0/564: dwrite d6/d64/d21/d25/d62/d63/f6e [0,4194304] 0 2024-04-24T22:45:33.004 INFO:tasks.workunit.client.1.smithi192.stdout:4/434: getdents d2/d9/d25/d2e/d31/d36/d5b 0 2024-04-24T22:45:33.004 INFO:tasks.workunit.client.0.smithi063.stdout:0/565: fdatasync d6/d74/fab 0 2024-04-24T22:45:33.008 INFO:tasks.workunit.client.1.smithi192.stdout:8/538: write db/dc/d29/f89 [2781045,86232] 0 2024-04-24T22:45:33.009 INFO:tasks.workunit.client.1.smithi192.stdout:1/427: mknod d9/d15/d2d/c8e 0 2024-04-24T22:45:33.013 INFO:tasks.workunit.client.0.smithi063.stdout:8/637: fsync d1/d21/f4f 0 2024-04-24T22:45:33.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/413: creat d0/f8b x:0 0 0 2024-04-24T22:45:33.021 INFO:tasks.workunit.client.1.smithi192.stdout:8/539: dread db/dc/d22/f66 [0,4194304] 0 2024-04-24T22:45:33.021 INFO:tasks.workunit.client.1.smithi192.stdout:0/531: dread d3/d27/d28/d2e/d4a/f4b [0,4194304] 0 2024-04-24T22:45:33.022 INFO:tasks.workunit.client.0.smithi063.stdout:9/641: chown d6/f30 18465 1 2024-04-24T22:45:33.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/414: read - d0/d13/d19/f4b zero size 2024-04-24T22:45:33.023 INFO:tasks.workunit.client.1.smithi192.stdout:9/601: write d3/dd/d53/f5c [394406,58258] 0 2024-04-24T22:45:33.024 INFO:tasks.workunit.client.1.smithi192.stdout:2/615: rename d4/d9/d9f/dba/fc2 to d4/d9/d10/da7/fc4 0 2024-04-24T22:45:33.026 INFO:tasks.workunit.client.1.smithi192.stdout:3/500: dwrite d2/f1e [0,4194304] 0 2024-04-24T22:45:33.026 INFO:tasks.workunit.client.1.smithi192.stdout:5/415: dread - d0/f8b zero size 2024-04-24T22:45:33.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/566: dread - d6/d64/da0/f84 zero size 2024-04-24T22:45:33.027 INFO:tasks.workunit.client.1.smithi192.stdout:4/435: creat d2/db/d10/f96 x:0 0 0 2024-04-24T22:45:33.027 INFO:tasks.workunit.client.0.smithi063.stdout:0/567: dread d6/d64/da0/d33/d87/f96 [0,4194304] 0 2024-04-24T22:45:33.028 INFO:tasks.workunit.client.1.smithi192.stdout:2/616: stat d4/d29/d60/c88 0 2024-04-24T22:45:33.029 INFO:tasks.workunit.client.0.smithi063.stdout:0/568: truncate d6/d64/da0/d33/fb8 405590 0 2024-04-24T22:45:33.029 INFO:tasks.workunit.client.1.smithi192.stdout:6/438: unlink d0/d8/d29/d4c/d4d/d83/f86 0 2024-04-24T22:45:33.030 INFO:tasks.workunit.client.0.smithi063.stdout:0/569: write d6/d3a/f7a [3937840,3914] 0 2024-04-24T22:45:33.030 INFO:tasks.workunit.client.0.smithi063.stdout:2/709: link d0/d2/d2d/d31/d89/l8b d0/d2/d35/ld3 0 2024-04-24T22:45:33.031 INFO:tasks.workunit.client.1.smithi192.stdout:6/439: readlink d0/d8/d3d/d3e/d40/l65 0 2024-04-24T22:45:33.033 INFO:tasks.workunit.client.1.smithi192.stdout:6/440: chown d0/d8/d29/c50 58946 1 2024-04-24T22:45:33.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/648: mkdir db/d85/dd2/dd7 0 2024-04-24T22:45:33.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/649: fsync db/d1d/d20/d3f/fc5 0 2024-04-24T22:45:33.038 INFO:tasks.workunit.client.1.smithi192.stdout:6/441: read d0/d8/d29/f30 [407883,31784] 0 2024-04-24T22:45:33.038 INFO:tasks.workunit.client.0.smithi063.stdout:1/702: dread d3/da/f6a [0,4194304] 0 2024-04-24T22:45:33.039 INFO:tasks.workunit.client.1.smithi192.stdout:6/442: chown d0/d8/d29/c50 607930 1 2024-04-24T22:45:33.043 INFO:tasks.workunit.client.1.smithi192.stdout:8/540: dread db/d43/f4e [0,4194304] 0 2024-04-24T22:45:33.044 INFO:tasks.workunit.client.1.smithi192.stdout:8/541: fdatasync db/f34 0 2024-04-24T22:45:33.044 INFO:tasks.workunit.client.0.smithi063.stdout:5/615: dread d5/d24/d49/f84 [0,4194304] 0 2024-04-24T22:45:33.047 INFO:tasks.workunit.client.0.smithi063.stdout:4/598: mknod d2/d8/d3c/cc0 0 2024-04-24T22:45:33.048 INFO:tasks.workunit.client.0.smithi063.stdout:9/642: chown d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e/f98 181024516 1 2024-04-24T22:45:33.049 INFO:tasks.workunit.client.1.smithi192.stdout:9/602: creat d3/dd/d53/fd5 x:0 0 0 2024-04-24T22:45:33.049 INFO:tasks.workunit.client.0.smithi063.stdout:7/561: sync 2024-04-24T22:45:33.050 INFO:tasks.workunit.client.0.smithi063.stdout:2/710: symlink d0/d2/d2f/ld4 0 2024-04-24T22:45:33.050 INFO:tasks.workunit.client.1.smithi192.stdout:4/436: readlink d2/d9/l2a 0 2024-04-24T22:45:33.052 INFO:tasks.workunit.client.0.smithi063.stdout:5/616: dwrite d5/d24/d36/fd2 [0,4194304] 0 2024-04-24T22:45:33.055 INFO:tasks.workunit.client.0.smithi063.stdout:0/570: dread d6/d64/d21/d9c/f9d [0,4194304] 0 2024-04-24T22:45:33.055 INFO:tasks.workunit.client.1.smithi192.stdout:3/501: creat d2/d4/d12/d39/d86/fa1 x:0 0 0 2024-04-24T22:45:33.059 INFO:tasks.workunit.client.0.smithi063.stdout:1/703: fdatasync d3/da/d34/d6d/fb6 0 2024-04-24T22:45:33.061 INFO:tasks.workunit.client.0.smithi063.stdout:8/638: symlink d1/d2/d17/dd1/ld2 0 2024-04-24T22:45:33.063 INFO:tasks.workunit.client.1.smithi192.stdout:9/603: dread d3/f21 [0,4194304] 0 2024-04-24T22:45:33.065 INFO:tasks.workunit.client.0.smithi063.stdout:3/655: creat da/fe2 x:0 0 0 2024-04-24T22:45:33.068 INFO:tasks.workunit.client.1.smithi192.stdout:8/542: creat db/d43/d7c/d85/fa7 x:0 0 0 2024-04-24T22:45:33.070 INFO:tasks.workunit.client.1.smithi192.stdout:8/543: write db/d5d/f84 [3059833,26152] 0 2024-04-24T22:45:33.072 INFO:tasks.workunit.client.1.smithi192.stdout:9/604: dwrite d3/dd/de/dba/f96 [0,4194304] 0 2024-04-24T22:45:33.073 INFO:tasks.workunit.client.1.smithi192.stdout:0/532: mkdir d3/d27/d28/dba 0 2024-04-24T22:45:33.077 INFO:tasks.workunit.client.1.smithi192.stdout:9/605: read d3/d24/f44 [1384673,4882] 0 2024-04-24T22:45:33.080 INFO:tasks.workunit.client.0.smithi063.stdout:2/711: rename d0/d2/l6 to d0/dd/d72/ld5 0 2024-04-24T22:45:33.083 INFO:tasks.workunit.client.1.smithi192.stdout:5/416: rename d0/d1b/d50/c54 to d0/d13/d3b/d79/c8c 0 2024-04-24T22:45:33.087 INFO:tasks.workunit.client.1.smithi192.stdout:2/617: creat d4/d9/d10/d6c/d95/db7/dbf/fc5 x:0 0 0 2024-04-24T22:45:33.088 INFO:tasks.workunit.client.0.smithi063.stdout:6/650: write db/d1d/d2d/d69/d82/d96/fb4 [4755732,18375] 0 2024-04-24T22:45:33.089 INFO:tasks.workunit.client.0.smithi063.stdout:0/571: dread d6/d64/da0/d33/d87/f96 [0,4194304] 0 2024-04-24T22:45:33.093 INFO:tasks.workunit.client.0.smithi063.stdout:8/639: mkdir d1/d11/d32/dd3 0 2024-04-24T22:45:33.095 INFO:tasks.workunit.client.0.smithi063.stdout:0/572: dwrite d6/d64/d24/f59 [4194304,4194304] 0 2024-04-24T22:45:33.095 INFO:tasks.workunit.client.0.smithi063.stdout:8/640: write d1/d22/d33/fc4 [105885,2553] 0 2024-04-24T22:45:33.098 INFO:tasks.workunit.client.0.smithi063.stdout:9/643: mknod d6/d2f/dcd/db/dca/cce 0 2024-04-24T22:45:33.102 INFO:tasks.workunit.client.0.smithi063.stdout:2/712: write d0/d2/d2d/d31/d89/d7/f64 [3522957,70799] 0 2024-04-24T22:45:33.103 INFO:tasks.workunit.client.0.smithi063.stdout:7/562: creat d0/d4/de/d17/d35/d69/db1/fc1 x:0 0 0 2024-04-24T22:45:33.106 INFO:tasks.workunit.client.1.smithi192.stdout:4/437: symlink d2/db/l97 0 2024-04-24T22:45:33.109 INFO:tasks.workunit.client.1.smithi192.stdout:2/618: dread d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:33.110 INFO:tasks.workunit.client.1.smithi192.stdout:5/417: unlink d0/l8 0 2024-04-24T22:45:33.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/418: stat d0/d1b/f6d 0 2024-04-24T22:45:33.113 INFO:tasks.workunit.client.1.smithi192.stdout:5/419: fsync d0/d13/d19/d83/f87 0 2024-04-24T22:45:33.113 INFO:tasks.workunit.client.1.smithi192.stdout:4/438: dwrite d2/d9/f8a [0,4194304] 0 2024-04-24T22:45:33.118 INFO:tasks.workunit.client.1.smithi192.stdout:6/443: creat d0/d8/d29/f95 x:0 0 0 2024-04-24T22:45:33.123 INFO:tasks.workunit.client.0.smithi063.stdout:0/573: creat d6/d64/d21/d25/d62/fc8 x:0 0 0 2024-04-24T22:45:33.126 INFO:tasks.workunit.client.0.smithi063.stdout:2/713: symlink d0/dd/dcd/ld6 0 2024-04-24T22:45:33.127 INFO:tasks.workunit.client.0.smithi063.stdout:2/714: chown d0/d2/d2d/d31/d89/d7/d86/l8d 14 1 2024-04-24T22:45:33.127 INFO:tasks.workunit.client.1.smithi192.stdout:9/606: creat d3/dd/de/da7/dae/fd6 x:0 0 0 2024-04-24T22:45:33.130 INFO:tasks.workunit.client.0.smithi063.stdout:1/704: creat d3/db5/d21/fea x:0 0 0 2024-04-24T22:45:33.132 INFO:tasks.workunit.client.0.smithi063.stdout:4/599: getdents d2/d80/d8d 0 2024-04-24T22:45:33.136 INFO:tasks.workunit.client.1.smithi192.stdout:1/428: link d9/c44 d9/d15/d2d/c8f 0 2024-04-24T22:45:33.138 INFO:tasks.workunit.client.0.smithi063.stdout:5/617: rename d5/d1d/dc0/dbb/f50 to d5/d11/d23/fd5 0 2024-04-24T22:45:33.139 INFO:tasks.workunit.client.0.smithi063.stdout:5/618: readlink d5/d1d/dc0/dbb/l34 0 2024-04-24T22:45:33.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/439: write d2/d9/d25/d2e/d31/d36/d5b/d64/f91 [2415549,32549] 0 2024-04-24T22:45:33.142 INFO:tasks.workunit.client.0.smithi063.stdout:6/651: creat db/d1d/d20/d3f/d5f/fd8 x:0 0 0 2024-04-24T22:45:33.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/641: dwrite d1/d22/f77 [4194304,4194304] 0 2024-04-24T22:45:33.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/444: chown d0/d8/c2d 7014 1 2024-04-24T22:45:33.144 INFO:tasks.workunit.client.1.smithi192.stdout:8/544: dwrite db/dc/d22/d31/f45 [0,4194304] 0 2024-04-24T22:45:33.145 INFO:tasks.workunit.client.1.smithi192.stdout:2/619: write d4/d9/d37/f4e [804459,17866] 0 2024-04-24T22:45:33.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/656: rename da/d18/d3a/d63/fda to da/db6/fe3 0 2024-04-24T22:45:33.153 INFO:tasks.workunit.client.1.smithi192.stdout:3/502: rename d2/d4/d12/d39/d97/f78 to d2/fa2 0 2024-04-24T22:45:33.156 INFO:tasks.workunit.client.0.smithi063.stdout:2/715: unlink d0/dd/d72/ld5 0 2024-04-24T22:45:33.156 INFO:tasks.workunit.client.1.smithi192.stdout:9/607: read d3/dd/de/dba/f2e [3936796,122458] 0 2024-04-24T22:45:33.159 INFO:tasks.workunit.client.1.smithi192.stdout:8/545: dwrite db/dc/f6e [0,4194304] 0 2024-04-24T22:45:33.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/608: chown d3/d59/d7d/d88/dc2 0 1 2024-04-24T22:45:33.160 INFO:tasks.workunit.client.1.smithi192.stdout:9/609: stat d3/dd/de/d17/f48 0 2024-04-24T22:45:33.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/716: dwrite d0/d2/d2d/d31/d89/d7/d86/fa2 [0,4194304] 0 2024-04-24T22:45:33.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/420: mknod d0/d13/d19/d71/d88/c8d 0 2024-04-24T22:45:33.163 INFO:tasks.workunit.client.1.smithi192.stdout:1/429: fdatasync d9/f14 0 2024-04-24T22:45:33.164 INFO:tasks.workunit.client.1.smithi192.stdout:3/503: dread d2/d4/d12/d65/f66 [0,4194304] 0 2024-04-24T22:45:33.164 INFO:tasks.workunit.client.1.smithi192.stdout:1/430: write d9/d15/d2d/d31/d2f/f73 [901352,71181] 0 2024-04-24T22:45:33.165 INFO:tasks.workunit.client.1.smithi192.stdout:0/533: getdents d3/d27/d28/d2e/d4a/d5c 0 2024-04-24T22:45:33.166 INFO:tasks.workunit.client.0.smithi063.stdout:8/642: mknod d1/d22/d33/da6/d54/d79/cd4 0 2024-04-24T22:45:33.168 INFO:tasks.workunit.client.0.smithi063.stdout:0/574: creat d6/da/fc9 x:0 0 0 2024-04-24T22:45:33.170 INFO:tasks.workunit.client.1.smithi192.stdout:4/440: truncate d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/f3c 1369851 0 2024-04-24T22:45:33.170 INFO:tasks.workunit.client.1.smithi192.stdout:0/534: read d3/d27/d28/d2e/d4a/d5c/f3d [3105143,73698] 0 2024-04-24T22:45:33.174 INFO:tasks.workunit.client.0.smithi063.stdout:9/644: rename d6/d2f/dcd/db/d6b/d4f/dad/dc0 to d6/dbd/dcf 0 2024-04-24T22:45:33.177 INFO:tasks.workunit.client.1.smithi192.stdout:2/620: dwrite d4/d9/d37/fa3 [0,4194304] 0 2024-04-24T22:45:33.179 INFO:tasks.workunit.client.1.smithi192.stdout:2/621: chown d4/d29/c71 124 1 2024-04-24T22:45:33.181 INFO:tasks.workunit.client.1.smithi192.stdout:0/535: dwrite d3/d6d/f91 [0,4194304] 0 2024-04-24T22:45:33.184 INFO:tasks.workunit.client.0.smithi063.stdout:7/563: sync 2024-04-24T22:45:33.186 INFO:tasks.workunit.client.0.smithi063.stdout:1/705: truncate d3/da/da2/dba/d11/d5e/d6f/f90 2384759 0 2024-04-24T22:45:33.188 INFO:tasks.workunit.client.1.smithi192.stdout:7/579: sync 2024-04-24T22:45:33.188 INFO:tasks.workunit.client.0.smithi063.stdout:2/717: mknod d0/d2/d2d/d31/d89/d1c/cd7 0 2024-04-24T22:45:33.192 INFO:tasks.workunit.client.0.smithi063.stdout:8/643: symlink d1/d22/d33/da6/d54/d79/ld5 0 2024-04-24T22:45:33.192 INFO:tasks.workunit.client.1.smithi192.stdout:5/421: dread d0/d13/d19/f60 [0,4194304] 0 2024-04-24T22:45:33.194 INFO:tasks.workunit.client.0.smithi063.stdout:6/652: dwrite db/d1d/d20/d3f/d5f/d93/f62 [0,4194304] 0 2024-04-24T22:45:33.195 INFO:tasks.workunit.client.0.smithi063.stdout:0/575: rmdir d6/d64/d21/d25/d62/d63/d89/d95 39 2024-04-24T22:45:33.197 INFO:tasks.workunit.client.1.smithi192.stdout:3/504: creat d2/d4/d4e/fa3 x:0 0 0 2024-04-24T22:45:33.197 INFO:tasks.workunit.client.1.smithi192.stdout:3/505: chown d2 658485658 1 2024-04-24T22:45:33.198 INFO:tasks.workunit.client.1.smithi192.stdout:3/506: chown d2/d4/d12/d39/d86/fa1 1530 1 2024-04-24T22:45:33.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/507: write d2/d4/d7/f60 [717613,25761] 0 2024-04-24T22:45:33.200 INFO:tasks.workunit.client.0.smithi063.stdout:3/657: write da/f15 [3325022,114450] 0 2024-04-24T22:45:33.203 INFO:tasks.workunit.client.1.smithi192.stdout:3/508: stat d2/d4/d12/l46 0 2024-04-24T22:45:33.205 INFO:tasks.workunit.client.1.smithi192.stdout:3/509: chown d2/d27/d28/d38/l8b 3 1 2024-04-24T22:45:33.207 INFO:tasks.workunit.client.0.smithi063.stdout:4/600: rename d2/d8/d36/d3a to d2/d19/d33/dc1 0 2024-04-24T22:45:33.208 INFO:tasks.workunit.client.1.smithi192.stdout:1/431: dwrite d9/d51/f82 [0,4194304] 0 2024-04-24T22:45:33.210 INFO:tasks.workunit.client.0.smithi063.stdout:9/645: readlink d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/lc9 0 2024-04-24T22:45:33.211 INFO:tasks.workunit.client.0.smithi063.stdout:4/601: dread d2/d19/d33/dc1/f9b [0,4194304] 0 2024-04-24T22:45:33.213 INFO:tasks.workunit.client.0.smithi063.stdout:4/602: read d2/d19/f37 [1685480,87372] 0 2024-04-24T22:45:33.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/564: mknod d0/d4/de/d17/d28/d31/d94/cc2 0 2024-04-24T22:45:33.216 INFO:tasks.workunit.client.1.smithi192.stdout:2/622: creat d4/d9/d10/d6c/d8c/fc6 x:0 0 0 2024-04-24T22:45:33.219 INFO:tasks.workunit.client.1.smithi192.stdout:8/546: mknod db/dc/d22/d31/d33/d8f/da6/ca8 0 2024-04-24T22:45:33.220 INFO:tasks.workunit.client.1.smithi192.stdout:8/547: fsync db/d43/d7c/f9d 0 2024-04-24T22:45:33.220 INFO:tasks.workunit.client.1.smithi192.stdout:7/580: mknod d2/db/d49/cb4 0 2024-04-24T22:45:33.221 INFO:tasks.workunit.client.0.smithi063.stdout:0/576: mknod d6/d64/d21/d25/d62/cca 0 2024-04-24T22:45:33.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/706: dwrite d3/db5/d21/d49/fd2 [0,4194304] 0 2024-04-24T22:45:33.223 INFO:tasks.workunit.client.1.smithi192.stdout:9/610: link d3/dd/d53/f56 d3/d5d/fd7 0 2024-04-24T22:45:33.223 INFO:tasks.workunit.client.0.smithi063.stdout:0/577: write d6/d64/da0/d33/fb8 [155452,31251] 0 2024-04-24T22:45:33.224 INFO:tasks.workunit.client.1.smithi192.stdout:2/623: dwrite d4/d9/d10/d6c/d8c/fc6 [0,4194304] 0 2024-04-24T22:45:33.225 INFO:tasks.workunit.client.0.smithi063.stdout:3/658: mkdir da/d39/d94/de4 0 2024-04-24T22:45:33.225 INFO:tasks.workunit.client.0.smithi063.stdout:0/578: readlink d6/d64/d24/l5a 0 2024-04-24T22:45:33.225 INFO:tasks.workunit.client.0.smithi063.stdout:8/644: dwrite d1/d11/d32/fc7 [0,4194304] 0 2024-04-24T22:45:33.228 INFO:tasks.workunit.client.0.smithi063.stdout:0/579: truncate d6/d64/d21/d25/d62/fc8 295761 0 2024-04-24T22:45:33.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/603: readlink d2/d80/l95 0 2024-04-24T22:45:33.237 INFO:tasks.workunit.client.0.smithi063.stdout:0/580: dwrite d6/d64/da0/d33/fb8 [0,4194304] 0 2024-04-24T22:45:33.238 INFO:tasks.workunit.client.1.smithi192.stdout:1/432: mkdir d9/d15/d5e/d90 0 2024-04-24T22:45:33.239 INFO:tasks.workunit.client.0.smithi063.stdout:4/604: fdatasync d2/d19/d33/d9d/fbf 0 2024-04-24T22:45:33.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/707: dread d3/f3c [0,4194304] 0 2024-04-24T22:45:33.241 INFO:tasks.workunit.client.0.smithi063.stdout:1/708: chown d3/da/da2/dba/d39/d5a/c7a 10405 1 2024-04-24T22:45:33.242 INFO:tasks.workunit.client.0.smithi063.stdout:1/709: truncate d3/db5/d21/f91 158921 0 2024-04-24T22:45:33.245 INFO:tasks.workunit.client.1.smithi192.stdout:6/445: rename d0/d8/d18/c7a to d0/d8/c96 0 2024-04-24T22:45:33.247 INFO:tasks.workunit.client.1.smithi192.stdout:1/433: dwrite d9/d15/d2d/d31/d2f/f85 [0,4194304] 0 2024-04-24T22:45:33.251 INFO:tasks.workunit.client.1.smithi192.stdout:4/441: creat d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/d7e/f98 x:0 0 0 2024-04-24T22:45:33.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/442: fdatasync d2/d9/d25/d2e/d31/d36/d5b/d28/f61 0 2024-04-24T22:45:33.254 INFO:tasks.workunit.client.1.smithi192.stdout:4/443: readlink d2/l69 0 2024-04-24T22:45:33.256 INFO:tasks.workunit.client.0.smithi063.stdout:3/659: rmdir da/d18 39 2024-04-24T22:45:33.259 INFO:tasks.workunit.client.0.smithi063.stdout:8/645: mkdir d1/d11/d8b/dd6 0 2024-04-24T22:45:33.259 INFO:tasks.workunit.client.1.smithi192.stdout:5/422: write d0/d13/d19/d64/d5f/d2c/d3c/f5a [606304,44632] 0 2024-04-24T22:45:33.263 INFO:tasks.workunit.client.0.smithi063.stdout:3/660: dwrite da/d40/f9c [0,4194304] 0 2024-04-24T22:45:33.264 INFO:tasks.workunit.client.1.smithi192.stdout:9/611: fdatasync d3/dd/f3a 0 2024-04-24T22:45:33.267 INFO:tasks.workunit.client.1.smithi192.stdout:5/423: dwrite d0/d1b/f89 [0,4194304] 0 2024-04-24T22:45:33.274 INFO:tasks.workunit.client.0.smithi063.stdout:9/646: dwrite d6/fd [0,4194304] 0 2024-04-24T22:45:33.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/446: mknod d0/d8/d3d/d3e/d7e/c97 0 2024-04-24T22:45:33.275 INFO:tasks.workunit.client.0.smithi063.stdout:6/653: dwrite db/d1d/d20/d29/d5d/fb6 [0,4194304] 0 2024-04-24T22:45:33.277 INFO:tasks.workunit.client.0.smithi063.stdout:0/581: rmdir d6/d64/da0/d33/d87 39 2024-04-24T22:45:33.282 INFO:tasks.workunit.client.1.smithi192.stdout:8/548: write db/dc/d22/d31/f4d [61569,55889] 0 2024-04-24T22:45:33.282 INFO:tasks.workunit.client.0.smithi063.stdout:0/582: write d6/da/d2b/d60/faf [663794,95029] 0 2024-04-24T22:45:33.283 INFO:tasks.workunit.client.1.smithi192.stdout:4/444: creat d2/d5c/f99 x:0 0 0 2024-04-24T22:45:33.287 INFO:tasks.workunit.client.1.smithi192.stdout:0/536: creat d3/d27/d28/d2e/d4a/fbb x:0 0 0 2024-04-24T22:45:33.293 INFO:tasks.workunit.client.1.smithi192.stdout:7/581: mkdir d2/db/d21/d42/d67/d9b/d6e/db5 0 2024-04-24T22:45:33.299 INFO:tasks.workunit.client.1.smithi192.stdout:2/624: link d4/d9/f8b d4/d9/d10/d6c/d95/db7/dbf/fc7 0 2024-04-24T22:45:33.302 INFO:tasks.workunit.client.1.smithi192.stdout:9/612: rename d3/d5d/fcc to d3/d59/d7d/d88/dc2/fd8 0 2024-04-24T22:45:33.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/625: dwrite d4/f13 [0,4194304] 0 2024-04-24T22:45:33.307 INFO:tasks.workunit.client.1.smithi192.stdout:6/447: rmdir d0/d8/d3d/d3e 39 2024-04-24T22:45:33.309 INFO:tasks.workunit.client.1.smithi192.stdout:1/434: symlink d9/d15/d2d/l91 0 2024-04-24T22:45:33.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/710: sync 2024-04-24T22:45:33.321 INFO:tasks.workunit.client.1.smithi192.stdout:7/582: dread d2/db/d21/f84 [0,4194304] 0 2024-04-24T22:45:33.324 INFO:tasks.workunit.client.1.smithi192.stdout:9/613: stat d3/dd/de/f12 0 2024-04-24T22:45:33.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/614: readlink d3/d5d/da1/d5e/d98/la0 0 2024-04-24T22:45:33.327 INFO:tasks.workunit.client.1.smithi192.stdout:3/510: link d2/d4/d12/d1a/d1d/d55/c2b d2/d4/d12/ca4 0 2024-04-24T22:45:33.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/537: dread d3/d27/d28/d2e/d4a/f4d [0,4194304] 0 2024-04-24T22:45:33.329 INFO:tasks.workunit.client.1.smithi192.stdout:8/549: dread db/dc/f1b [0,4194304] 0 2024-04-24T22:45:33.330 INFO:tasks.workunit.client.1.smithi192.stdout:2/626: mknod d4/d9/d10/da7/cc8 0 2024-04-24T22:45:33.338 INFO:tasks.workunit.client.1.smithi192.stdout:4/445: symlink d2/d9/d6b/l9a 0 2024-04-24T22:45:33.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/424: creat d0/f8e x:0 0 0 2024-04-24T22:45:33.347 INFO:tasks.workunit.client.1.smithi192.stdout:0/538: mkdir d3/d27/d28/d31/dbc 0 2024-04-24T22:45:33.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/539: chown d3/d27/d28/d31/fb5 3267617 1 2024-04-24T22:45:33.349 INFO:tasks.workunit.client.1.smithi192.stdout:0/540: chown d3/d6d/d79/f8b 241084573 1 2024-04-24T22:45:33.352 INFO:tasks.workunit.client.1.smithi192.stdout:9/615: rename d3/d59/d7d/d9b/fac to d3/d59/dbf/fd9 0 2024-04-24T22:45:33.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/616: dread - d3/dd/de/da7/dae/fd6 zero size 2024-04-24T22:45:33.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/511: truncate d2/d4/d12/f49 200812 0 2024-04-24T22:45:33.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/512: chown d2/d4/f16 9 1 2024-04-24T22:45:33.360 INFO:tasks.workunit.client.1.smithi192.stdout:6/448: mknod d0/d8/d3d/d3e/d40/d44/c98 0 2024-04-24T22:45:33.361 INFO:tasks.workunit.client.0.smithi063.stdout:8/646: mknod d1/d2/d3/d86/cd7 0 2024-04-24T22:45:33.362 INFO:tasks.workunit.client.1.smithi192.stdout:1/435: symlink d9/d15/d5e/d90/l92 0 2024-04-24T22:45:33.364 INFO:tasks.workunit.client.1.smithi192.stdout:7/583: dwrite d2/db/d21/d42/f8c [0,4194304] 0 2024-04-24T22:45:33.364 INFO:tasks.workunit.client.0.smithi063.stdout:5/619: rename d5/d11/d23/d2d/l6e to d5/d24/ld6 0 2024-04-24T22:45:33.365 INFO:tasks.workunit.client.1.smithi192.stdout:7/584: fdatasync d2/db/d21/d42/d5b/f9a 0 2024-04-24T22:45:33.369 INFO:tasks.workunit.client.0.smithi063.stdout:3/661: fdatasync da/d18/d9e/fc4 0 2024-04-24T22:45:33.374 INFO:tasks.workunit.client.0.smithi063.stdout:9/647: dread - d6/d2f/dcd/d53/f7a zero size 2024-04-24T22:45:33.375 INFO:tasks.workunit.client.0.smithi063.stdout:9/648: chown d6/d2f/dcd/f5b 608672888 1 2024-04-24T22:45:33.381 INFO:tasks.workunit.client.1.smithi192.stdout:9/617: mknod d3/dd/de/d17/d89/db4/cda 0 2024-04-24T22:45:33.384 INFO:tasks.workunit.client.0.smithi063.stdout:8/647: symlink d1/d2/d17/dd1/ld8 0 2024-04-24T22:45:33.385 INFO:tasks.workunit.client.1.smithi192.stdout:8/550: dread f5 [4194304,4194304] 0 2024-04-24T22:45:33.385 INFO:tasks.workunit.client.0.smithi063.stdout:4/605: truncate d2/f35 433246 0 2024-04-24T22:45:33.386 INFO:tasks.workunit.client.0.smithi063.stdout:4/606: chown d2/d19/d33/dc1/lb2 25698709 1 2024-04-24T22:45:33.386 INFO:tasks.workunit.client.1.smithi192.stdout:8/551: chown db/d5d/d74/d96 2 1 2024-04-24T22:45:33.388 INFO:tasks.workunit.client.1.smithi192.stdout:4/446: dwrite d2/d9/d25/d2e/d31/d36/d5b/d28/f2b [0,4194304] 0 2024-04-24T22:45:33.388 INFO:tasks.workunit.client.1.smithi192.stdout:6/449: fdatasync d0/d8/d3d/d3e/d40/f47 0 2024-04-24T22:45:33.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/450: chown d0/d8/d3d/d3e/d40/c8f 46022088 1 2024-04-24T22:45:33.390 INFO:tasks.workunit.client.0.smithi063.stdout:7/565: rename d0/d4/de/d6f/d2e/d55/faa to d0/d4/de/d17/d35/d6c/fc3 0 2024-04-24T22:45:33.391 INFO:tasks.workunit.client.1.smithi192.stdout:4/447: write d2/d9/d25/d2e/d31/d36/d5b/d28/d2f/d7e/f98 [8095,130828] 0 2024-04-24T22:45:33.392 INFO:tasks.workunit.client.1.smithi192.stdout:1/436: mkdir d9/d15/d2d/d48/d93 0 2024-04-24T22:45:33.393 INFO:tasks.workunit.client.1.smithi192.stdout:4/448: chown d2/d9/d25/d2e/d31/d36/d5b/d64/l81 73839903 1 2024-04-24T22:45:33.394 INFO:tasks.workunit.client.1.smithi192.stdout:4/449: fsync d2/d9/d25/f71 0 2024-04-24T22:45:33.396 INFO:tasks.workunit.client.1.smithi192.stdout:4/450: fdatasync d2/d9/d25/d2e/d31/d36/d5b/d64/f83 0 2024-04-24T22:45:33.400 INFO:tasks.workunit.client.0.smithi063.stdout:6/654: symlink db/d85/dd2/dd7/ld9 0 2024-04-24T22:45:33.403 INFO:tasks.workunit.client.1.smithi192.stdout:0/541: rename d3/d27/d28/d2e/d4a/d5c/d80/dac/lb4 to d3/d40/lbd 0 2024-04-24T22:45:33.405 INFO:tasks.workunit.client.0.smithi063.stdout:0/583: link d6/d64/d21/d80/f9e d6/d3a/fcb 0 2024-04-24T22:45:33.405 INFO:tasks.workunit.client.0.smithi063.stdout:0/584: stat d6/d64/da0/d33/db5 0 2024-04-24T22:45:33.406 INFO:tasks.workunit.client.1.smithi192.stdout:2/627: rmdir d4/d9/d9f/dba 0 2024-04-24T22:45:33.406 INFO:tasks.workunit.client.0.smithi063.stdout:0/585: write d6/d64/d21/d25/d62/fc8 [279012,121574] 0 2024-04-24T22:45:33.407 INFO:tasks.workunit.client.1.smithi192.stdout:2/628: chown d4/d9/d10/d6c/d8c/fab 150 1 2024-04-24T22:45:33.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/648: symlink d1/d11/d32/d38/db7/ld9 0 2024-04-24T22:45:33.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/649: chown d1/d22/d33/da6 2274 1 2024-04-24T22:45:33.410 INFO:tasks.workunit.client.0.smithi063.stdout:5/620: mknod d5/d1d/cd7 0 2024-04-24T22:45:33.410 INFO:tasks.workunit.client.0.smithi063.stdout:3/662: dread da/d18/d3a/f3f [0,4194304] 0 2024-04-24T22:45:33.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/618: write d3/dd/de/d17/f1d [469190,83137] 0 2024-04-24T22:45:33.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/619: stat d3/d5d/fb2 0 2024-04-24T22:45:33.414 INFO:tasks.workunit.client.0.smithi063.stdout:2/718: rename d0/d2/d27/d3a/f84 to d0/d2/d6c/fd8 0 2024-04-24T22:45:33.415 INFO:tasks.workunit.client.0.smithi063.stdout:2/719: write d0/d2/d2d/d31/d89/d7/d86/fa2 [1031571,120106] 0 2024-04-24T22:45:33.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/566: unlink d0/d4/de/d17/f42 0 2024-04-24T22:45:33.419 INFO:tasks.workunit.client.0.smithi063.stdout:7/567: write d0/d4/de/d17/d28/fb8 [784271,41162] 0 2024-04-24T22:45:33.420 INFO:tasks.workunit.client.1.smithi192.stdout:6/451: rmdir d0/d8/d3d/d3e/d40 39 2024-04-24T22:45:33.428 INFO:tasks.workunit.client.1.smithi192.stdout:9/620: dread d3/f31 [0,4194304] 0 2024-04-24T22:45:33.429 INFO:tasks.workunit.client.1.smithi192.stdout:5/425: link d0/d13/d19/d64/d4f/d52/f7e d0/d1b/f8f 0 2024-04-24T22:45:33.433 INFO:tasks.workunit.client.1.smithi192.stdout:3/513: rename d2/d4/d12/d65 to d2/d27/d28/da5 0 2024-04-24T22:45:33.439 INFO:tasks.workunit.client.0.smithi063.stdout:0/586: dwrite d6/d64/d21/d25/d62/d63/d89/d95/da4/f8f [0,4194304] 0 2024-04-24T22:45:33.440 INFO:tasks.workunit.client.1.smithi192.stdout:3/514: dwrite d2/d4/d12/d1a/d1d/d55/d5f/f80 [0,4194304] 0 2024-04-24T22:45:33.440 INFO:tasks.workunit.client.0.smithi063.stdout:8/650: unlink d1/d2/d17/l1a 0 2024-04-24T22:45:33.442 INFO:tasks.workunit.client.0.smithi063.stdout:5/621: fsync d5/d11/d18/f19 0 2024-04-24T22:45:33.447 INFO:tasks.workunit.client.0.smithi063.stdout:4/607: link d2/d19/d33/dc1/d50/f9e d2/d8/d40/da0/fc2 0 2024-04-24T22:45:33.449 INFO:tasks.workunit.client.1.smithi192.stdout:8/552: symlink db/dc/d22/d31/la9 0 2024-04-24T22:45:33.453 INFO:tasks.workunit.client.1.smithi192.stdout:4/451: creat d2/d9/d25/d2e/d31/d7d/f9b x:0 0 0 2024-04-24T22:45:33.453 INFO:tasks.workunit.client.1.smithi192.stdout:0/542: write d3/d27/d28/d31/f3a [382953,120647] 0 2024-04-24T22:45:33.456 INFO:tasks.workunit.client.0.smithi063.stdout:1/711: rename d3/da/da2/dba/d39/fda to d3/db5/d21/d9f/daa/feb 0 2024-04-24T22:45:33.458 INFO:tasks.workunit.client.1.smithi192.stdout:9/621: rmdir d3/dd/de 39 2024-04-24T22:45:33.459 INFO:tasks.workunit.client.1.smithi192.stdout:9/622: write d3/d24/f7e [867554,16223] 0 2024-04-24T22:45:33.461 INFO:tasks.workunit.client.1.smithi192.stdout:5/426: unlink d0/d13/f86 0 2024-04-24T22:45:33.463 INFO:tasks.workunit.client.1.smithi192.stdout:7/585: rename d2/db/d21/d42/d67/d9b/db2 to d2/db/d21/d42/d67/d9b/d6e/db5/db6 0 2024-04-24T22:45:33.464 INFO:tasks.workunit.client.0.smithi063.stdout:2/720: mkdir d0/d2/d2d/d31/d89/d7/d86/da4/dd9 0 2024-04-24T22:45:33.465 INFO:tasks.workunit.client.0.smithi063.stdout:2/721: chown d0/dd/d62/d7c/fb9 71755617 1 2024-04-24T22:45:33.468 INFO:tasks.workunit.client.0.smithi063.stdout:8/651: truncate d1/d2/d3/f56 1716529 0 2024-04-24T22:45:33.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/608: truncate d2/d8/d36/f7d 887660 0 2024-04-24T22:45:33.476 INFO:tasks.workunit.client.0.smithi063.stdout:7/568: creat d0/d4/de/d17/d35/d6c/d8e/fc4 x:0 0 0 2024-04-24T22:45:33.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/587: symlink d6/d64/da0/lcc 0 2024-04-24T22:45:33.478 INFO:tasks.workunit.client.0.smithi063.stdout:0/588: dread - d6/d64/d21/d80/f9e zero size 2024-04-24T22:45:33.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/609: mkdir d2/d19/d33/dc1/d50/dc3 0 2024-04-24T22:45:33.479 INFO:tasks.workunit.client.0.smithi063.stdout:1/712: symlink d3/db5/d21/d9f/dd5/lec 0 2024-04-24T22:45:33.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/655: link db/d1d/d20/d3f/d7f/d68/d70/l79 db/d1d/d20/d3f/lda 0 2024-04-24T22:45:33.480 INFO:tasks.workunit.client.0.smithi063.stdout:8/652: dread d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:33.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/622: creat d5/d11/d23/fd8 x:0 0 0 2024-04-24T22:45:33.481 INFO:tasks.workunit.client.0.smithi063.stdout:6/656: chown db/d1d/d20/d3e/f45 5 1 2024-04-24T22:45:33.482 INFO:tasks.workunit.client.0.smithi063.stdout:9/649: rename d6/d2f/dcd/db/d6b/d4f/dad/c75 to d6/d2f/dcd/d1f/d20/d22/cd0 0 2024-04-24T22:45:33.483 INFO:tasks.workunit.client.0.smithi063.stdout:6/657: truncate db/d1d/d20/d3e/f87 748632 0 2024-04-24T22:45:33.486 INFO:tasks.workunit.client.0.smithi063.stdout:4/610: creat d2/d8/d40/d51/fc4 x:0 0 0 2024-04-24T22:45:33.486 INFO:tasks.workunit.client.0.smithi063.stdout:4/611: fdatasync d2/d80/fb0 0 2024-04-24T22:45:33.486 INFO:tasks.workunit.client.0.smithi063.stdout:3/663: rename da/d39/d94/lb2 to da/d39/d94/dbc/le5 0 2024-04-24T22:45:33.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/658: mkdir db/d1c/ddb 0 2024-04-24T22:45:33.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/612: dwrite d2/d19/d82/fb8 [0,4194304] 0 2024-04-24T22:45:33.490 INFO:tasks.workunit.client.0.smithi063.stdout:2/722: rename d0/dd/d62 to d0/dd/d72/d77/dda 0 2024-04-24T22:45:33.491 INFO:tasks.workunit.client.0.smithi063.stdout:4/613: readlink d2/d8/d36/da3/lbc 0 2024-04-24T22:45:33.491 INFO:tasks.workunit.client.0.smithi063.stdout:6/659: fdatasync db/d1d/f4c 0 2024-04-24T22:45:33.493 INFO:tasks.workunit.client.0.smithi063.stdout:2/723: symlink d0/d2/d2f/dba/ldb 0 2024-04-24T22:45:33.494 INFO:tasks.workunit.client.0.smithi063.stdout:4/614: rename d2/f29 to d2/d19/fc5 0 2024-04-24T22:45:33.497 INFO:tasks.workunit.client.0.smithi063.stdout:6/660: dread db/d1d/d20/d3f/d5f/d93/f62 [0,4194304] 0 2024-04-24T22:45:33.498 INFO:tasks.workunit.client.0.smithi063.stdout:1/713: read d3/db5/d21/d9f/daa/db7/f58 [2697038,108339] 0 2024-04-24T22:45:33.498 INFO:tasks.workunit.client.1.smithi192.stdout:7/586: chown d2/db/d36 0 1 2024-04-24T22:45:33.499 INFO:tasks.workunit.client.1.smithi192.stdout:0/543: creat d3/d27/d28/d2e/d4a/d5c/d80/d94/fbe x:0 0 0 2024-04-24T22:45:33.499 INFO:tasks.workunit.client.0.smithi063.stdout:4/615: mkdir d2/d8/d36/da3/dc6 0 2024-04-24T22:45:33.499 INFO:tasks.workunit.client.1.smithi192.stdout:4/452: mkdir d2/d9/d25/d2e/d31/d7d/d9c 0 2024-04-24T22:45:33.501 INFO:tasks.workunit.client.1.smithi192.stdout:9/623: dwrite d3/dd/fbe [0,4194304] 0 2024-04-24T22:45:33.505 INFO:tasks.workunit.client.1.smithi192.stdout:9/624: chown d3/dd/fbe 4602 1 2024-04-24T22:45:33.507 INFO:tasks.workunit.client.1.smithi192.stdout:2/629: getdents d4/d9/d10/d6c/d95 0 2024-04-24T22:45:33.511 INFO:tasks.workunit.client.1.smithi192.stdout:3/515: dread d2/d4/d12/d1a/d1d/d55/f32 [0,4194304] 0 2024-04-24T22:45:33.511 INFO:tasks.workunit.client.1.smithi192.stdout:4/453: symlink d2/db/d10/l9d 0 2024-04-24T22:45:33.512 INFO:tasks.workunit.client.1.smithi192.stdout:3/516: chown d2/d4/d12/f1c 257834 1 2024-04-24T22:45:33.515 INFO:tasks.workunit.client.1.smithi192.stdout:1/437: rename d9/d15/d2d/d31/f32 to d9/d51/d70/f94 0 2024-04-24T22:45:33.516 INFO:tasks.workunit.client.1.smithi192.stdout:3/517: dread d2/d4/d7/f47 [0,4194304] 0 2024-04-24T22:45:33.517 INFO:tasks.workunit.client.0.smithi063.stdout:1/714: symlink d3/led 0 2024-04-24T22:45:33.523 INFO:tasks.workunit.client.1.smithi192.stdout:0/544: unlink d3/d27/d28/d2e/d4a/d5c/d80/d83/l98 0 2024-04-24T22:45:33.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/715: dwrite d3/da/da2/dba/d11/d5e/d6f/fbb [0,4194304] 0 2024-04-24T22:45:33.526 INFO:tasks.workunit.client.0.smithi063.stdout:6/661: rename db/d1d/d20/d3f/lda to db/d1d/d20/ldc 0 2024-04-24T22:45:33.530 INFO:tasks.workunit.client.0.smithi063.stdout:4/616: dread d2/d8/d36/fba [0,4194304] 0 2024-04-24T22:45:33.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/454: creat d2/d9/d25/d2e/d31/d36/d5b/d28/f9e x:0 0 0 2024-04-24T22:45:33.536 INFO:tasks.workunit.client.0.smithi063.stdout:6/662: rename db/da0/dbd to db/d1c/dc3/ddd 0 2024-04-24T22:45:33.537 INFO:tasks.workunit.client.0.smithi063.stdout:6/663: chown db/d1d/d20/d84/f9e 3 1 2024-04-24T22:45:33.537 INFO:tasks.workunit.client.1.smithi192.stdout:6/452: rename d0/d8/c88 to d0/d8/d18/c99 0 2024-04-24T22:45:33.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/438: creat d9/d51/d70/f95 x:0 0 0 2024-04-24T22:45:33.543 INFO:tasks.workunit.client.0.smithi063.stdout:7/569: sync 2024-04-24T22:45:33.543 INFO:tasks.workunit.client.0.smithi063.stdout:5/623: write d5/d1d/fba [629232,65945] 0 2024-04-24T22:45:33.544 INFO:tasks.workunit.client.0.smithi063.stdout:0/589: truncate d6/d64/d24/f73 3053186 0 2024-04-24T22:45:33.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/625: mkdir d3/dd/de/dba/dbc/ddb 0 2024-04-24T22:45:33.546 INFO:tasks.workunit.client.0.smithi063.stdout:9/650: write d6/d2f/d42/d81/f96 [154931,73534] 0 2024-04-24T22:45:33.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/617: rename d2/d8/d36/da3/lb7 to d2/d8/d36/da3/dc6/lc7 0 2024-04-24T22:45:33.550 INFO:tasks.workunit.client.0.smithi063.stdout:6/664: symlink db/d85/dd2/lde 0 2024-04-24T22:45:33.550 INFO:tasks.workunit.client.0.smithi063.stdout:0/590: dwrite d6/d64/d21/d25/d62/fc8 [0,4194304] 0 2024-04-24T22:45:33.552 INFO:tasks.workunit.client.0.smithi063.stdout:8/653: creat d1/d22/d33/da6/d54/d79/fda x:0 0 0 2024-04-24T22:45:33.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/545: stat d3/d27/d28/d2e/l82 0 2024-04-24T22:45:33.554 INFO:tasks.workunit.client.0.smithi063.stdout:3/664: truncate da/d40/f81 3036773 0 2024-04-24T22:45:33.555 INFO:tasks.workunit.client.0.smithi063.stdout:7/570: chown d0/d4/de/d6f/d5c/lac 1900865 1 2024-04-24T22:45:33.555 INFO:tasks.workunit.client.1.smithi192.stdout:2/630: mkdir d4/d9/d10/d6c/d6f/dc9 0 2024-04-24T22:45:33.557 INFO:tasks.workunit.client.0.smithi063.stdout:7/571: write d0/d4/de/d17/f9b [3035120,107440] 0 2024-04-24T22:45:33.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/654: dwrite d1/d2/f84 [0,4194304] 0 2024-04-24T22:45:33.559 INFO:tasks.workunit.client.0.smithi063.stdout:7/572: fdatasync d0/d4/de/f10 0 2024-04-24T22:45:33.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/453: creat d0/d8/d22/f9a x:0 0 0 2024-04-24T22:45:33.562 INFO:tasks.workunit.client.0.smithi063.stdout:2/724: write d0/d2/d2d/d31/d89/d7/d86/da0/d94/fc0 [3970016,14628] 0 2024-04-24T22:45:33.564 INFO:tasks.workunit.client.1.smithi192.stdout:1/439: creat d9/d15/d2d/d4a/f96 x:0 0 0 2024-04-24T22:45:33.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/651: stat d6/d2f/d42/d81/fc2 0 2024-04-24T22:45:33.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/573: dwrite d0/d4/de/d17/d35/f54 [0,4194304] 0 2024-04-24T22:45:33.567 INFO:tasks.workunit.client.0.smithi063.stdout:9/652: write d6/d2f/d42/d81/fc6 [434799,16806] 0 2024-04-24T22:45:33.569 INFO:tasks.workunit.client.0.smithi063.stdout:9/653: fdatasync d6/d2f/dcd/f71 0 2024-04-24T22:45:33.571 INFO:tasks.workunit.client.1.smithi192.stdout:9/626: fsync d3/dd/d53/f56 0 2024-04-24T22:45:33.575 INFO:tasks.workunit.client.1.smithi192.stdout:6/454: dread d0/d8/d22/f27 [0,4194304] 0 2024-04-24T22:45:33.575 INFO:tasks.workunit.client.1.smithi192.stdout:7/587: dwrite d2/db/d49/f70 [0,4194304] 0 2024-04-24T22:45:33.576 INFO:tasks.workunit.client.0.smithi063.stdout:0/591: mknod d6/dd/ccd 0 2024-04-24T22:45:33.578 INFO:tasks.workunit.client.1.smithi192.stdout:1/440: dwrite d9/dd/d74/f88 [0,4194304] 0 2024-04-24T22:45:33.579 INFO:tasks.workunit.client.1.smithi192.stdout:2/631: creat d4/d9/d10/da7/fca x:0 0 0 2024-04-24T22:45:33.582 INFO:tasks.workunit.client.0.smithi063.stdout:1/716: write d3/da/da2/dba/d11/f41 [8846274,79874] 0 2024-04-24T22:45:33.584 INFO:tasks.workunit.client.0.smithi063.stdout:3/665: creat da/d18/d47/d4d/d9b/fe6 x:0 0 0 2024-04-24T22:45:33.594 INFO:tasks.workunit.client.0.smithi063.stdout:5/624: mknod d5/d24/d36/d65/cd9 0 2024-04-24T22:45:33.595 INFO:tasks.workunit.client.1.smithi192.stdout:9/627: fsync d3/d5d/da1/f79 0 2024-04-24T22:45:33.596 INFO:tasks.workunit.client.1.smithi192.stdout:5/427: truncate d0/d13/d53/f70 478584 0 2024-04-24T22:45:33.597 INFO:tasks.workunit.client.1.smithi192.stdout:9/628: write d3/fbd [211464,70557] 0 2024-04-24T22:45:33.598 INFO:tasks.workunit.client.1.smithi192.stdout:9/629: chown d3/dd/de/d17/d89/db4/db9 95616071 1 2024-04-24T22:45:33.605 INFO:tasks.workunit.client.0.smithi063.stdout:2/725: mkdir d0/d2/d2f/ddc 0 2024-04-24T22:45:33.605 INFO:tasks.workunit.client.0.smithi063.stdout:2/726: chown d0/dd/dcd 1888254127 1 2024-04-24T22:45:33.605 INFO:tasks.workunit.client.0.smithi063.stdout:2/727: chown d0/d2/f6a 1737881 1 2024-04-24T22:45:33.606 INFO:tasks.workunit.client.0.smithi063.stdout:2/728: chown d0/d2/d2d/d31/d89/d7/d86/da4 103945 1 2024-04-24T22:45:33.607 INFO:tasks.workunit.client.0.smithi063.stdout:7/574: unlink d0/l2 0 2024-04-24T22:45:33.608 INFO:tasks.workunit.client.1.smithi192.stdout:0/546: write d3/d27/d28/d31/f85 [998206,106975] 0 2024-04-24T22:45:33.608 INFO:tasks.workunit.client.1.smithi192.stdout:2/632: creat d4/d9/d10/da7/fcb x:0 0 0 2024-04-24T22:45:33.610 INFO:tasks.workunit.client.1.smithi192.stdout:0/547: fdatasync d3/d27/d28/d2e/d4a/fa1 0 2024-04-24T22:45:33.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/633: write d4/d9/f90 [588982,45315] 0 2024-04-24T22:45:33.611 INFO:tasks.workunit.client.1.smithi192.stdout:7/588: dread d2/db/d21/d42/d67/f7e [0,4194304] 0 2024-04-24T22:45:33.612 INFO:tasks.workunit.client.1.smithi192.stdout:2/634: chown d4/d9/d10/d6c/f85 30982432 1 2024-04-24T22:45:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:4/618: symlink d2/d8/d40/da0/lc8 0 2024-04-24T22:45:33.613 INFO:tasks.workunit.client.1.smithi192.stdout:4/455: write d2/d9/d25/d2e/d31/f4c [4600435,88235] 0 2024-04-24T22:45:33.613 INFO:tasks.workunit.client.1.smithi192.stdout:3/518: getdents d2/d4/d7/d4b/d54 0 2024-04-24T22:45:33.614 INFO:tasks.workunit.client.1.smithi192.stdout:2/635: truncate d4/d9/f90 972921 0 2024-04-24T22:45:33.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/654: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fd1 x:0 0 0 2024-04-24T22:45:33.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/655: write d6/d2f/dcd/d53/f99 [569003,7917] 0 2024-04-24T22:45:33.621 INFO:tasks.workunit.client.1.smithi192.stdout:9/630: rename d3/dd/de/d17/f23 to d3/d5d/fdc 0 2024-04-24T22:45:33.622 INFO:tasks.workunit.client.0.smithi063.stdout:0/592: symlink d6/d3a/lce 0 2024-04-24T22:45:33.623 INFO:tasks.workunit.client.0.smithi063.stdout:5/625: dread d5/d1d/faa [0,4194304] 0 2024-04-24T22:45:33.625 INFO:tasks.workunit.client.1.smithi192.stdout:1/441: write d9/d6f/f77 [1130155,106760] 0 2024-04-24T22:45:33.626 INFO:tasks.workunit.client.1.smithi192.stdout:1/442: stat d9/d15/d2d/d63/d68/f76 0 2024-04-24T22:45:33.633 INFO:tasks.workunit.client.0.smithi063.stdout:2/729: mknod d0/d2/d2d/d31/d68/cdd 0 2024-04-24T22:45:33.633 INFO:tasks.workunit.client.1.smithi192.stdout:4/456: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:33.634 INFO:tasks.workunit.client.1.smithi192.stdout:7/589: mknod d2/db/d49/cb7 0 2024-04-24T22:45:33.635 INFO:tasks.workunit.client.1.smithi192.stdout:6/455: dwrite d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:33.636 INFO:tasks.workunit.client.0.smithi063.stdout:7/575: creat d0/d4/de/d17/d35/d69/db1/fc5 x:0 0 0 2024-04-24T22:45:33.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/576: chown d0/d4/de/d17/d28/c2b 210 1 2024-04-24T22:45:33.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/577: fdatasync d0/d2f/f37 0 2024-04-24T22:45:33.638 INFO:tasks.workunit.client.0.smithi063.stdout:4/619: symlink d2/d19/d33/dc1/lc9 0 2024-04-24T22:45:33.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/578: chown d0/d4/de/d17/d28/d31/l82 4553 1 2024-04-24T22:45:33.640 INFO:tasks.workunit.client.1.smithi192.stdout:3/519: mknod d2/d4/d7/d4b/d54/ca6 0 2024-04-24T22:45:33.641 INFO:tasks.workunit.client.0.smithi063.stdout:9/656: creat d6/d2f/dcd/d1f/d20/d2a/d4b/fd2 x:0 0 0 2024-04-24T22:45:33.644 INFO:tasks.workunit.client.0.smithi063.stdout:6/665: creat db/fdf x:0 0 0 2024-04-24T22:45:33.647 INFO:tasks.workunit.client.1.smithi192.stdout:9/631: mkdir d3/dd/d53/ddd 0 2024-04-24T22:45:33.648 INFO:tasks.workunit.client.1.smithi192.stdout:9/632: dread - d3/dd/de/d17/f4f zero size 2024-04-24T22:45:33.648 INFO:tasks.workunit.client.0.smithi063.stdout:0/593: mkdir d6/d64/d21/d9c/dcf 0 2024-04-24T22:45:33.650 INFO:tasks.workunit.client.1.smithi192.stdout:1/443: symlink d9/d15/d2d/d63/l97 0 2024-04-24T22:45:33.650 INFO:tasks.workunit.client.0.smithi063.stdout:0/594: fsync d6/dd/f4d 0 2024-04-24T22:45:33.650 INFO:tasks.workunit.client.1.smithi192.stdout:0/548: write d3/d27/d3c/d64/f84 [4223801,118786] 0 2024-04-24T22:45:33.652 INFO:tasks.workunit.client.1.smithi192.stdout:0/549: write d3/d27/d3c/d64/f84 [3676100,35511] 0 2024-04-24T22:45:33.653 INFO:tasks.workunit.client.0.smithi063.stdout:6/666: dwrite db/d1d/d20/fd6 [0,4194304] 0 2024-04-24T22:45:33.655 INFO:tasks.workunit.client.0.smithi063.stdout:1/717: dwrite d3/da/da2/dba/f3a [0,4194304] 0 2024-04-24T22:45:33.655 INFO:tasks.workunit.client.1.smithi192.stdout:4/457: mkdir d2/d9/d25/d2e/d31/d36/d5b/d28/d9f 0 2024-04-24T22:45:33.656 INFO:tasks.workunit.client.0.smithi063.stdout:8/655: rmdir d1/d2/d3/dac 0 2024-04-24T22:45:33.660 INFO:tasks.workunit.client.0.smithi063.stdout:2/730: unlink d0/d2/d2d/fb6 0 2024-04-24T22:45:33.660 INFO:tasks.workunit.client.1.smithi192.stdout:5/428: write d0/d13/f39 [2221884,34650] 0 2024-04-24T22:45:33.672 INFO:tasks.workunit.client.0.smithi063.stdout:9/657: unlink d6/d2f/dcd/d1f/f61 0 2024-04-24T22:45:33.673 INFO:tasks.workunit.client.0.smithi063.stdout:9/658: chown d6/d2f/d42/f6f 162 1 2024-04-24T22:45:33.673 INFO:tasks.workunit.client.1.smithi192.stdout:9/633: creat d3/d59/d7d/d88/fde x:0 0 0 2024-04-24T22:45:33.675 INFO:tasks.workunit.client.1.smithi192.stdout:9/634: truncate d3/d59/dbf/fd9 818872 0 2024-04-24T22:45:33.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/666: creat da/d18/d9e/fe7 x:0 0 0 2024-04-24T22:45:33.677 INFO:tasks.workunit.client.1.smithi192.stdout:0/550: unlink d3/d27/d28/d2e/d4a/d5c/f51 0 2024-04-24T22:45:33.681 INFO:tasks.workunit.client.1.smithi192.stdout:7/590: unlink d2/db/d21/d42/d67/d9b/d6e/da2/faa 0 2024-04-24T22:45:33.681 INFO:tasks.workunit.client.1.smithi192.stdout:7/591: read - d2/db/d49/f6c zero size 2024-04-24T22:45:33.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/718: mknod d3/db5/d21/d9f/dd5/cee 0 2024-04-24T22:45:33.682 INFO:tasks.workunit.client.0.smithi063.stdout:3/667: dwrite da/fa2 [0,4194304] 0 2024-04-24T22:45:33.684 INFO:tasks.workunit.client.0.smithi063.stdout:8/656: unlink d1/d11/f9f 0 2024-04-24T22:45:33.685 INFO:tasks.workunit.client.0.smithi063.stdout:8/657: chown d1/d2/f2f 2101206157 1 2024-04-24T22:45:33.687 INFO:tasks.workunit.client.1.smithi192.stdout:2/636: link d4/d9/d37/f91 d4/d9/fcc 0 2024-04-24T22:45:33.690 INFO:tasks.workunit.client.1.smithi192.stdout:1/444: fdatasync d9/f4f 0 2024-04-24T22:45:33.690 INFO:tasks.workunit.client.0.smithi063.stdout:2/731: stat d0/dd/dcd/lbd 0 2024-04-24T22:45:33.696 INFO:tasks.workunit.client.0.smithi063.stdout:4/620: dwrite d2/d19/d33/dc1/d50/f9e [0,4194304] 0 2024-04-24T22:45:33.696 INFO:tasks.workunit.client.1.smithi192.stdout:3/520: dwrite d2/d4/d12/d1a/d1d/d55/f32 [0,4194304] 0 2024-04-24T22:45:33.702 INFO:tasks.workunit.client.0.smithi063.stdout:0/595: symlink d6/d64/da0/d98/dc7/ld0 0 2024-04-24T22:45:33.715 INFO:tasks.workunit.client.0.smithi063.stdout:7/579: creat d0/d4/de/d17/fc6 x:0 0 0 2024-04-24T22:45:33.717 INFO:tasks.workunit.client.0.smithi063.stdout:5/626: getdents d5/d11/d18 0 2024-04-24T22:45:33.718 INFO:tasks.workunit.client.0.smithi063.stdout:6/667: rename db/d1d/la3 to db/d1d/d20/d3f/le0 0 2024-04-24T22:45:33.718 INFO:tasks.workunit.client.0.smithi063.stdout:9/659: dwrite d6/d2f/dcd/d53/f7a [0,4194304] 0 2024-04-24T22:45:33.724 INFO:tasks.workunit.client.0.smithi063.stdout:7/580: unlink d0/d4/de/d17/d35/d6c/fc3 0 2024-04-24T22:45:33.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/658: write d1/d11/f63 [1733163,76888] 0 2024-04-24T22:45:33.727 INFO:tasks.workunit.client.0.smithi063.stdout:8/659: write d1/f5 [2643609,60389] 0 2024-04-24T22:45:33.727 INFO:tasks.workunit.client.0.smithi063.stdout:5/627: mkdir d5/d24/d49/d3b/dc5/dda 0 2024-04-24T22:45:33.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/660: symlink d6/d2f/dcd/db/d6b/d4f/d59/ld3 0 2024-04-24T22:45:33.731 INFO:tasks.workunit.client.0.smithi063.stdout:1/719: write d3/de0/d79/f2e [4395719,130500] 0 2024-04-24T22:45:33.731 INFO:tasks.workunit.client.0.smithi063.stdout:3/668: truncate da/f28 5736238 0 2024-04-24T22:45:33.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/661: dread - d6/d2f/dcd/d1f/d20/d22/fb2 zero size 2024-04-24T22:45:33.733 INFO:tasks.workunit.client.0.smithi063.stdout:1/720: write d3/da/da2/dba/f3a [1108414,31007] 0 2024-04-24T22:45:33.733 INFO:tasks.workunit.client.0.smithi063.stdout:4/621: rename d2/d19/d33/d9d/lb4 to d2/d8/d40/da0/lca 0 2024-04-24T22:45:33.734 INFO:tasks.workunit.client.0.smithi063.stdout:4/622: readlink d2/d80/l85 0 2024-04-24T22:45:33.734 INFO:tasks.workunit.client.0.smithi063.stdout:1/721: chown d3/da/da2/dba/d39/dbf/lc8 235873062 1 2024-04-24T22:45:33.735 INFO:tasks.workunit.client.0.smithi063.stdout:4/623: write d2/d8/d36/fb6 [461863,33897] 0 2024-04-24T22:45:33.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/732: getdents d0/d2/d2d/d31/d89/d1c 0 2024-04-24T22:45:33.740 INFO:tasks.workunit.client.0.smithi063.stdout:2/733: truncate d0/d2/d2d/d31/d89/d7/d86/d8e/fbe 804455 0 2024-04-24T22:45:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/734: write d0/d2/d39/fb7 [303843,71938] 0 2024-04-24T22:45:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:1/722: fsync d3/da/da2/dba/fc4 0 2024-04-24T22:45:33.746 INFO:tasks.workunit.client.0.smithi063.stdout:5/628: mknod d5/cdb 0 2024-04-24T22:45:33.748 INFO:tasks.workunit.client.0.smithi063.stdout:7/581: link d0/d4/de/d17/d35/d6c/d7a/laf d0/d4/de/d6f/d5c/d8b/dc0/lc7 0 2024-04-24T22:45:33.749 INFO:tasks.workunit.client.0.smithi063.stdout:6/668: truncate db/f2a 4066558 0 2024-04-24T22:45:33.750 INFO:tasks.workunit.client.0.smithi063.stdout:7/582: chown d0/d4/de/d17/d35/f79 2495 1 2024-04-24T22:45:33.752 INFO:tasks.workunit.client.0.smithi063.stdout:1/723: symlink d3/db5/d21/d49/lef 0 2024-04-24T22:45:33.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/624: rename d2/d8/l30 to d2/d80/lcb 0 2024-04-24T22:45:33.755 INFO:tasks.workunit.client.0.smithi063.stdout:5/629: symlink d5/d11/d23/d2d/d62/d81/ldc 0 2024-04-24T22:45:33.759 INFO:tasks.workunit.client.0.smithi063.stdout:8/660: creat d1/d22/d33/da6/fdb x:0 0 0 2024-04-24T22:45:33.760 INFO:tasks.workunit.client.0.smithi063.stdout:7/583: dwrite d0/d4/de/d17/d35/d69/fb0 [0,4194304] 0 2024-04-24T22:45:33.763 INFO:tasks.workunit.client.0.smithi063.stdout:8/661: chown d1/d11/d3c/l52 928902590 1 2024-04-24T22:45:33.765 INFO:tasks.workunit.client.0.smithi063.stdout:5/630: unlink d5/d1d/dc0/dbb/f89 0 2024-04-24T22:45:33.766 INFO:tasks.workunit.client.0.smithi063.stdout:9/662: dwrite d6/d2f/dcd/d1f/d20/d22/f3b [0,4194304] 0 2024-04-24T22:45:33.768 INFO:tasks.workunit.client.0.smithi063.stdout:8/662: fsync d1/d2/fcf 0 2024-04-24T22:45:33.769 INFO:tasks.workunit.client.0.smithi063.stdout:1/724: rename d3/da/da2/dba/d11/d5e/d9c/dcb/fcc to d3/de0/d79/ff0 0 2024-04-24T22:45:33.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/625: dwrite d2/d9f/fbd [0,4194304] 0 2024-04-24T22:45:33.773 INFO:tasks.workunit.client.0.smithi063.stdout:2/735: dwrite d0/d2/d35/f52 [0,4194304] 0 2024-04-24T22:45:33.776 INFO:tasks.workunit.client.0.smithi063.stdout:4/626: mknod d2/d19/ccc 0 2024-04-24T22:45:33.780 INFO:tasks.workunit.client.0.smithi063.stdout:5/631: rename d5/d1d/dc0/dbb/l34 to d5/d11/d18/ldd 0 2024-04-24T22:45:33.783 INFO:tasks.workunit.client.0.smithi063.stdout:9/663: creat d6/d2f/dcd/db/d1c/d78/d9a/daa/fd4 x:0 0 0 2024-04-24T22:45:33.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/725: dwrite d3/da/da2/dba/d11/f5f [0,4194304] 0 2024-04-24T22:45:33.786 INFO:tasks.workunit.client.0.smithi063.stdout:7/584: dread d0/d4/de/d6f/d5c/d63/f84 [0,4194304] 0 2024-04-24T22:45:33.787 INFO:tasks.workunit.client.0.smithi063.stdout:9/664: mkdir d6/d2f/dcd/d53/dd5 0 2024-04-24T22:45:33.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/669: write db/d1d/d20/d3f/d5f/d93/f54 [626459,44782] 0 2024-04-24T22:45:33.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/736: read d0/d2/d2f/f58 [51240,558] 0 2024-04-24T22:45:33.793 INFO:tasks.workunit.client.0.smithi063.stdout:4/627: dwrite d2/d8/d40/f68 [0,4194304] 0 2024-04-24T22:45:33.794 INFO:tasks.workunit.client.0.smithi063.stdout:7/585: unlink d0/d2f/l58 0 2024-04-24T22:45:33.797 INFO:tasks.workunit.client.0.smithi063.stdout:9/665: rmdir d6/d2f/dcd/db/d93 39 2024-04-24T22:45:33.799 INFO:tasks.workunit.client.0.smithi063.stdout:6/670: creat db/d1d/d20/d3e/fe1 x:0 0 0 2024-04-24T22:45:33.803 INFO:tasks.workunit.client.0.smithi063.stdout:8/663: rename d1/d22/d33/da6/d54/l98 to d1/d2/d3/db6/ldc 0 2024-04-24T22:45:33.803 INFO:tasks.workunit.client.0.smithi063.stdout:8/664: readlink d1/d22/d33/da6/d54/d79/lc2 0 2024-04-24T22:45:33.804 INFO:tasks.workunit.client.0.smithi063.stdout:7/586: mknod d0/db5/cc8 0 2024-04-24T22:45:33.805 INFO:tasks.workunit.client.0.smithi063.stdout:7/587: write d0/d4/de/f10 [8153169,66301] 0 2024-04-24T22:45:33.806 INFO:tasks.workunit.client.0.smithi063.stdout:4/628: fsync d2/d19/fa6 0 2024-04-24T22:45:33.806 INFO:tasks.workunit.client.0.smithi063.stdout:2/737: dread d0/f69 [0,4194304] 0 2024-04-24T22:45:33.808 INFO:tasks.workunit.client.0.smithi063.stdout:8/665: truncate d1/d11/d32/d38/f42 3452072 0 2024-04-24T22:45:33.809 INFO:tasks.workunit.client.0.smithi063.stdout:2/738: read d0/d2/d2d/d31/d89/f25 [4378745,125940] 0 2024-04-24T22:45:33.810 INFO:tasks.workunit.client.0.smithi063.stdout:8/666: dread - d1/d22/d33/dbd/fbe zero size 2024-04-24T22:45:33.811 INFO:tasks.workunit.client.0.smithi063.stdout:9/666: mknod d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/db9/cd6 0 2024-04-24T22:45:33.812 INFO:tasks.workunit.client.0.smithi063.stdout:7/588: dwrite d0/fa [4194304,4194304] 0 2024-04-24T22:45:33.814 INFO:tasks.workunit.client.0.smithi063.stdout:1/726: truncate d3/db5/fb2 3170108 0 2024-04-24T22:45:33.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/629: creat d2/d19/d33/dc1/d50/fcd x:0 0 0 2024-04-24T22:45:33.817 INFO:tasks.workunit.client.0.smithi063.stdout:8/667: rmdir d1/d22/d33/da6/d54/d79 39 2024-04-24T22:45:33.818 INFO:tasks.workunit.client.0.smithi063.stdout:4/630: truncate d2/d19/d33/dc1/f6d 919492 0 2024-04-24T22:45:33.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/596: sync 2024-04-24T22:45:33.820 INFO:tasks.workunit.client.0.smithi063.stdout:9/667: mknod d6/d2f/dcd/db/dca/cd7 0 2024-04-24T22:45:33.823 INFO:tasks.workunit.client.0.smithi063.stdout:2/739: mknod d0/d2/d27/cde 0 2024-04-24T22:45:33.824 INFO:tasks.workunit.client.0.smithi063.stdout:4/631: mkdir d2/d19/dce 0 2024-04-24T22:45:33.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/597: mkdir d6/d64/d24/dd1 0 2024-04-24T22:45:33.830 INFO:tasks.workunit.client.0.smithi063.stdout:1/727: symlink d3/da/da2/dba/d11/lf1 0 2024-04-24T22:45:33.831 INFO:tasks.workunit.client.0.smithi063.stdout:2/740: rename d0/d2/d2d/d31/d89/d1c/cd7 to d0/d2/d2d/d31/d68/cdf 0 2024-04-24T22:45:33.833 INFO:tasks.workunit.client.0.smithi063.stdout:2/741: write d0/d2/d2d/d31/d89/d7/f64 [2757451,84164] 0 2024-04-24T22:45:33.835 INFO:tasks.workunit.client.0.smithi063.stdout:0/598: mkdir d6/d64/d24/d42/dd2 0 2024-04-24T22:45:33.836 INFO:tasks.workunit.client.0.smithi063.stdout:0/599: write d6/d64/d21/d25/d62/d63/d89/d95/da4/fc2 [846319,3086] 0 2024-04-24T22:45:33.836 INFO:tasks.workunit.client.0.smithi063.stdout:8/668: link d1/d22/d33/da6/d54/d7b/fb9 d1/d2/d17/dd1/fdd 0 2024-04-24T22:45:33.837 INFO:tasks.workunit.client.0.smithi063.stdout:0/600: read - d6/d64/f92 zero size 2024-04-24T22:45:33.841 INFO:tasks.workunit.client.0.smithi063.stdout:7/589: dwrite d0/d4/fb [0,4194304] 0 2024-04-24T22:45:33.843 INFO:tasks.workunit.client.0.smithi063.stdout:8/669: symlink d1/d11/d8b/lde 0 2024-04-24T22:45:33.844 INFO:tasks.workunit.client.0.smithi063.stdout:4/632: link d2/d80/l8c d2/d19/dce/lcf 0 2024-04-24T22:45:33.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/633: write d2/d80/f8a [3319876,74369] 0 2024-04-24T22:45:33.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/601: mknod d6/d64/da0/d98/cd3 0 2024-04-24T22:45:33.848 INFO:tasks.workunit.client.0.smithi063.stdout:9/668: creat d6/d2f/dcd/db/d93/fd8 x:0 0 0 2024-04-24T22:45:33.850 INFO:tasks.workunit.client.0.smithi063.stdout:8/670: dwrite d1/d22/d33/da6/d54/f71 [4194304,4194304] 0 2024-04-24T22:45:33.851 INFO:tasks.workunit.client.0.smithi063.stdout:7/590: dread - d0/d4/de/d17/f8a zero size 2024-04-24T22:45:33.853 INFO:tasks.workunit.client.0.smithi063.stdout:8/671: dread - d1/d2/d17/dd1/fdd zero size 2024-04-24T22:45:33.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/458: link d2/db/d5d/d77/f92 d2/d9/d25/d2e/d31/d36/d5b/d28/d9f/fa0 0 2024-04-24T22:45:33.854 INFO:tasks.workunit.client.1.smithi192.stdout:5/429: mkdir d0/d90 0 2024-04-24T22:45:33.854 INFO:tasks.workunit.client.0.smithi063.stdout:8/672: readlink d1/d22/d33/da6/d54/d7b/la4 0 2024-04-24T22:45:33.857 INFO:tasks.workunit.client.0.smithi063.stdout:9/669: dwrite d6/d2f/dcd/d53/f7a [0,4194304] 0 2024-04-24T22:45:33.858 INFO:tasks.workunit.client.1.smithi192.stdout:6/456: getdents d0/d8 0 2024-04-24T22:45:33.860 INFO:tasks.workunit.client.1.smithi192.stdout:1/445: write d9/d15/d2d/d31/d2f/f4c [978125,78290] 0 2024-04-24T22:45:33.862 INFO:tasks.workunit.client.1.smithi192.stdout:1/446: fdatasync d9/d15/f2e 0 2024-04-24T22:45:33.864 INFO:tasks.workunit.client.1.smithi192.stdout:1/447: fsync d9/d15/d2d/d31/d2f/f73 0 2024-04-24T22:45:33.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/551: fdatasync d3/d27/d28/f1d 0 2024-04-24T22:45:33.866 INFO:tasks.workunit.client.0.smithi063.stdout:2/742: dwrite d0/d2/f1e [0,4194304] 0 2024-04-24T22:45:33.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/521: dwrite d2/fa2 [0,4194304] 0 2024-04-24T22:45:33.867 INFO:tasks.workunit.client.1.smithi192.stdout:0/552: chown d3/d40/f9b 1 1 2024-04-24T22:45:33.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/591: mkdir d0/d4/de/d17/d35/d69/dc9 0 2024-04-24T22:45:33.869 INFO:tasks.workunit.client.0.smithi063.stdout:1/728: dread d3/da/d34/d68/fa8 [0,4194304] 0 2024-04-24T22:45:33.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/430: mkdir d0/d13/d19/d64/d5f/d2c/d3c/d91 0 2024-04-24T22:45:33.870 INFO:tasks.workunit.client.0.smithi063.stdout:1/729: chown d3/db5/d21/d9f/daa/db7/c4a 21676789 1 2024-04-24T22:45:33.871 INFO:tasks.workunit.client.0.smithi063.stdout:1/730: dread - d3/da/da2/dba/fc4 zero size 2024-04-24T22:45:33.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/731: chown d3/de0/d79/le3 51650 1 2024-04-24T22:45:33.875 INFO:tasks.workunit.client.1.smithi192.stdout:4/459: rename d2/d9/d25/d2e/d31 to d2/db/d75/da1 0 2024-04-24T22:45:33.876 INFO:tasks.workunit.client.1.smithi192.stdout:5/431: chown d0/d13/d19/d64/d4f 0 1 2024-04-24T22:45:33.879 INFO:tasks.workunit.client.0.smithi063.stdout:2/743: dread d0/d2/fa1 [0,4194304] 0 2024-04-24T22:45:33.882 INFO:tasks.workunit.client.0.smithi063.stdout:8/673: unlink d1/d22/d33/da6/d54/d7b/l85 0 2024-04-24T22:45:33.883 INFO:tasks.workunit.client.1.smithi192.stdout:6/457: mkdir d0/d8/d3d/d3e/d7e/d9b 0 2024-04-24T22:45:33.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/635: link d3/dd/de/lf d3/dd/de/d17/d89/db4/db9/ldf 0 2024-04-24T22:45:33.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/670: symlink d6/d2f/dcd/db/d1c/d78/d9a/ld9 0 2024-04-24T22:45:33.886 INFO:tasks.workunit.client.1.smithi192.stdout:6/458: write d0/d8/d3d/f49 [679859,60344] 0 2024-04-24T22:45:33.887 INFO:tasks.workunit.client.1.smithi192.stdout:1/448: dwrite d9/d15/d2d/d63/f79 [0,4194304] 0 2024-04-24T22:45:33.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/459: fdatasync d0/d8/d29/d58/f5b 0 2024-04-24T22:45:33.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/602: rename d6/d64/da0/d33/d87 to d6/d64/d24/d42/dd4 0 2024-04-24T22:45:33.891 INFO:tasks.workunit.client.0.smithi063.stdout:0/603: write d6/da/d2b/d60/fa1 [933248,106114] 0 2024-04-24T22:45:33.892 INFO:tasks.workunit.client.0.smithi063.stdout:9/671: dwrite d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fa2 [0,4194304] 0 2024-04-24T22:45:33.894 INFO:tasks.workunit.client.0.smithi063.stdout:9/672: dread - d6/d2f/dcd/db/d93/da0/fae zero size 2024-04-24T22:45:33.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/592: unlink d0/d4/de/d17/d28/d31/c62 0 2024-04-24T22:45:33.900 INFO:tasks.workunit.client.1.smithi192.stdout:3/522: rmdir d2/d27/d28/da5/d93 39 2024-04-24T22:45:33.902 INFO:tasks.workunit.client.0.smithi063.stdout:2/744: creat d0/d2/d39/fe0 x:0 0 0 2024-04-24T22:45:33.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/669: sync 2024-04-24T22:45:33.902 INFO:tasks.workunit.client.0.smithi063.stdout:6/671: sync 2024-04-24T22:45:33.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/632: sync 2024-04-24T22:45:33.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/634: dwrite d2/f7c [0,4194304] 0 2024-04-24T22:45:33.908 INFO:tasks.workunit.client.0.smithi063.stdout:9/673: mkdir d6/d2f/d42/dda 0 2024-04-24T22:45:33.908 INFO:tasks.workunit.client.0.smithi063.stdout:7/593: rmdir d0/d2f 39 2024-04-24T22:45:33.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/674: write d6/d2f/dcd/db/f62 [5384518,23056] 0 2024-04-24T22:45:33.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/594: write d0/d4/de/d17/fc6 [542316,72125] 0 2024-04-24T22:45:33.912 INFO:tasks.workunit.client.0.smithi063.stdout:8/674: mkdir d1/d2/ddf 0 2024-04-24T22:45:33.914 INFO:tasks.workunit.client.0.smithi063.stdout:5/633: dwrite d5/d1d/dc7/fd3 [0,4194304] 0 2024-04-24T22:45:33.914 INFO:tasks.workunit.client.0.smithi063.stdout:3/670: symlink da/d18/d47/d4d/d62/le8 0 2024-04-24T22:45:33.915 INFO:tasks.workunit.client.0.smithi063.stdout:5/634: stat d5/d1d/d7d/d67/dbe 0 2024-04-24T22:45:33.918 INFO:tasks.workunit.client.0.smithi063.stdout:2/745: creat d0/d2/d2d/d31/d89/d7/d86/da0/fe1 x:0 0 0 2024-04-24T22:45:33.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/604: mkdir d6/d64/d24/dd1/dd5 0 2024-04-24T22:45:33.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/675: dread d6/d2f/dcd/db/d6b/f46 [0,4194304] 0 2024-04-24T22:45:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:33 smithi192 ceph-mon[33991]: pgmap v14: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 300 MiB/s wr, 536 op/s 2024-04-24T22:45:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:33 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:33 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:33.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/676: dread d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fa2 [0,4194304] 0 2024-04-24T22:45:33.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/432: symlink d0/d13/d19/d64/l92 0 2024-04-24T22:45:33.936 INFO:tasks.workunit.client.0.smithi063.stdout:8/675: fsync d1/d11/d8b/f8d 0 2024-04-24T22:45:33.937 INFO:tasks.workunit.client.0.smithi063.stdout:7/595: creat d0/d4/de/d17/d35/d69/db1/fca x:0 0 0 2024-04-24T22:45:33.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/636: rmdir d3/dd/de/da7 39 2024-04-24T22:45:33.939 INFO:tasks.workunit.client.0.smithi063.stdout:6/672: getdents db/d1c/dc3/ddd 0 2024-04-24T22:45:33.941 INFO:tasks.workunit.client.0.smithi063.stdout:3/671: creat da/d18/d47/fe9 x:0 0 0 2024-04-24T22:45:33.942 INFO:tasks.workunit.client.0.smithi063.stdout:3/672: dread - da/d18/d9e/fe7 zero size 2024-04-24T22:45:33.943 INFO:tasks.workunit.client.0.smithi063.stdout:3/673: dread da/d48/f5c [0,4194304] 0 2024-04-24T22:45:33.945 INFO:tasks.workunit.client.0.smithi063.stdout:2/746: stat d0/d2/d2d/l3e 0 2024-04-24T22:45:33.946 INFO:tasks.workunit.client.0.smithi063.stdout:0/605: dread d6/d64/f3b [0,4194304] 0 2024-04-24T22:45:33.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/606: chown d6/dd/c30 425209 1 2024-04-24T22:45:33.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/635: mknod d2/d8/d3c/cd0 0 2024-04-24T22:45:33.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/607: chown d6/da/d2b/f40 18396 1 2024-04-24T22:45:33.951 INFO:tasks.workunit.client.1.smithi192.stdout:0/553: mknod d3/d27/d28/cbf 0 2024-04-24T22:45:33.952 INFO:tasks.workunit.client.0.smithi063.stdout:9/677: mkdir d6/d2f/dcd/d1f/ddb 0 2024-04-24T22:45:33.953 INFO:tasks.workunit.client.0.smithi063.stdout:4/636: dwrite d2/d19/d33/d9d/fab [0,4194304] 0 2024-04-24T22:45:33.953 INFO:tasks.workunit.client.1.smithi192.stdout:3/523: unlink d2/d27/f9d 0 2024-04-24T22:45:33.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/637: link d4/d29/d60/c46 d4/d9/d10/d6c/d95/db7/dbf/ccd 0 2024-04-24T22:45:33.958 INFO:tasks.workunit.client.1.smithi192.stdout:9/637: dread - d3/dd/d62/f6b zero size 2024-04-24T22:45:33.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/732: truncate d3/f19 2799117 0 2024-04-24T22:45:33.961 INFO:tasks.workunit.client.1.smithi192.stdout:1/449: creat d9/d15/d2d/d48/d93/f98 x:0 0 0 2024-04-24T22:45:33.962 INFO:tasks.workunit.client.1.smithi192.stdout:1/450: readlink d9/dd/l5a 0 2024-04-24T22:45:33.965 INFO:tasks.workunit.client.0.smithi063.stdout:3/674: readlink da/d18/d3a/l8f 0 2024-04-24T22:45:33.966 INFO:tasks.workunit.client.1.smithi192.stdout:1/451: dread d9/dd/d74/f88 [0,4194304] 0 2024-04-24T22:45:33.967 INFO:tasks.workunit.client.0.smithi063.stdout:2/747: chown d0/d2/d2d/f4d 238 1 2024-04-24T22:45:33.970 INFO:tasks.workunit.client.0.smithi063.stdout:5/635: write d5/ff [804483,123602] 0 2024-04-24T22:45:33.971 INFO:tasks.workunit.client.0.smithi063.stdout:0/608: dread d6/d64/d21/d25/f55 [0,4194304] 0 2024-04-24T22:45:33.975 INFO:tasks.workunit.client.1.smithi192.stdout:9/638: mknod d3/d5d/da1/ce0 0 2024-04-24T22:45:33.976 INFO:tasks.workunit.client.0.smithi063.stdout:9/678: rmdir d6/d2f/dcd/db 39 2024-04-24T22:45:33.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/637: truncate d2/f1c 787398 0 2024-04-24T22:45:33.981 INFO:tasks.workunit.client.1.smithi192.stdout:6/460: link d0/d8/d22/f32 d0/d8/d3d/f9c 0 2024-04-24T22:45:33.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/676: write d1/d21/f83 [372430,42387] 0 2024-04-24T22:45:33.986 INFO:tasks.workunit.client.1.smithi192.stdout:3/524: write d2/d4/d7/f2c [3002247,71012] 0 2024-04-24T22:45:33.986 INFO:tasks.workunit.client.0.smithi063.stdout:2/748: dread d0/d2/d2d/d31/d89/d7/f13 [0,4194304] 0 2024-04-24T22:45:33.987 INFO:tasks.workunit.client.1.smithi192.stdout:3/525: read - d2/d4/d4e/f8f zero size 2024-04-24T22:45:33.988 INFO:tasks.workunit.client.1.smithi192.stdout:3/526: chown d2/d27/d28/d38/l64 428226 1 2024-04-24T22:45:33.988 INFO:tasks.workunit.client.1.smithi192.stdout:2/638: mkdir d4/d9/d10/d6c/d95/dc0/dce 0 2024-04-24T22:45:33.989 INFO:tasks.workunit.client.1.smithi192.stdout:5/433: write d0/d13/d3b/f3a [585366,5925] 0 2024-04-24T22:45:33.992 INFO:tasks.workunit.client.0.smithi063.stdout:5/636: unlink d5/d11/d76/d92/f9d 0 2024-04-24T22:45:33.995 INFO:tasks.workunit.client.0.smithi063.stdout:0/609: mkdir d6/d64/da0/d98/dd6 0 2024-04-24T22:45:33.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/677: dread d1/d21/f6f [0,4194304] 0 2024-04-24T22:45:33.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/639: dwrite d4/d9/d10/d6c/d6f/da8/fb4 [0,4194304] 0 2024-04-24T22:45:33.997 INFO:tasks.workunit.client.0.smithi063.stdout:8/678: readlink d1/d22/l66 0 2024-04-24T22:45:33.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/640: write d4/d9/fa4 [83146,98019] 0 2024-04-24T22:45:33.997 INFO:tasks.workunit.client.0.smithi063.stdout:9/679: readlink d6/d2f/dcd/l29 0 2024-04-24T22:45:33.999 INFO:tasks.workunit.client.0.smithi063.stdout:0/610: dread d6/da/d2b/d60/fa1 [0,4194304] 0 2024-04-24T22:45:34.000 INFO:tasks.workunit.client.1.smithi192.stdout:6/461: unlink d0/f19 0 2024-04-24T22:45:34.004 INFO:tasks.workunit.client.1.smithi192.stdout:0/554: dwrite d3/d27/d28/f8e [0,4194304] 0 2024-04-24T22:45:34.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/596: creat d0/d4/de/d6f/fcb x:0 0 0 2024-04-24T22:45:34.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/597: dread - d0/d4/de/d17/d35/d69/db1/fc5 zero size 2024-04-24T22:45:34.008 INFO:tasks.workunit.client.0.smithi063.stdout:7/598: stat d0/d4/de/d6f/f47 0 2024-04-24T22:45:34.009 INFO:tasks.workunit.client.1.smithi192.stdout:3/527: rmdir d2/d4/d12/d39/d97 39 2024-04-24T22:45:34.011 INFO:tasks.workunit.client.1.smithi192.stdout:3/528: write d2/d4/d7/d4b/f8d [695436,23864] 0 2024-04-24T22:45:34.012 INFO:tasks.workunit.client.1.smithi192.stdout:5/434: symlink d0/d13/d3b/d2f/l93 0 2024-04-24T22:45:34.012 INFO:tasks.workunit.client.1.smithi192.stdout:3/529: stat d2/f33 0 2024-04-24T22:45:34.014 INFO:tasks.workunit.client.1.smithi192.stdout:3/530: read d2/d4/d7/f9 [346931,12250] 0 2024-04-24T22:45:34.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/673: link db/c3a db/d1d/d20/d3f/d7f/d68/d70/ce2 0 2024-04-24T22:45:34.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/639: mkdir d3/dd/de/d17/d89/de1 0 2024-04-24T22:45:34.019 INFO:tasks.workunit.client.1.smithi192.stdout:3/531: dread d2/d4/d12/d1a/d1d/d55/d5f/f80 [0,4194304] 0 2024-04-24T22:45:34.026 INFO:tasks.workunit.client.0.smithi063.stdout:2/749: write d0/dd/d72/d77/dda/d7c/f61 [2139614,27087] 0 2024-04-24T22:45:34.026 INFO:tasks.workunit.client.1.smithi192.stdout:0/555: dread d3/d27/d3c/d6c/db3/f86 [0,4194304] 0 2024-04-24T22:45:34.027 INFO:tasks.workunit.client.0.smithi063.stdout:7/599: dread d0/d4/de/d6f/f53 [4194304,4194304] 0 2024-04-24T22:45:34.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/452: rename d9/dd/f29 to d9/d15/d2d/d31/f99 0 2024-04-24T22:45:34.029 INFO:tasks.workunit.client.1.smithi192.stdout:1/453: fsync d9/dd/d74/f7a 0 2024-04-24T22:45:34.030 INFO:tasks.workunit.client.0.smithi063.stdout:5/637: write f0 [3110614,17736] 0 2024-04-24T22:45:34.030 INFO:tasks.workunit.client.1.smithi192.stdout:5/435: creat d0/d13/d53/f94 x:0 0 0 2024-04-24T22:45:34.031 INFO:tasks.workunit.client.0.smithi063.stdout:6/674: dread db/d1d/d20/d84/fae [0,4194304] 0 2024-04-24T22:45:34.032 INFO:tasks.workunit.client.0.smithi063.stdout:0/611: rename d6/da/f29 to d6/d64/d21/d80/fd7 0 2024-04-24T22:45:34.033 INFO:tasks.workunit.client.0.smithi063.stdout:5/638: dread - d5/d11/d23/d2d/d62/d81/da4/da6/fc6 zero size 2024-04-24T22:45:34.034 INFO:tasks.workunit.client.1.smithi192.stdout:9/640: mknod d3/d5d/da1/ce2 0 2024-04-24T22:45:34.034 INFO:tasks.workunit.client.0.smithi063.stdout:4/638: symlink d2/d8/d36/ld1 0 2024-04-24T22:45:34.036 INFO:tasks.workunit.client.0.smithi063.stdout:9/680: dwrite d6/d2f/dcd/db/d6b/f8f [0,4194304] 0 2024-04-24T22:45:34.038 INFO:tasks.workunit.client.0.smithi063.stdout:9/681: truncate d6/d2f/dcd/d1f/f28 805855 0 2024-04-24T22:45:34.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/682: read d6/f30 [2771633,35954] 0 2024-04-24T22:45:34.046 INFO:tasks.workunit.client.0.smithi063.stdout:9/683: dwrite d6/d2f/dcd/db/d93/fd8 [0,4194304] 0 2024-04-24T22:45:34.047 INFO:tasks.workunit.client.1.smithi192.stdout:1/454: mknod d9/d15/d2d/d78/c9a 0 2024-04-24T22:45:34.047 INFO:tasks.workunit.client.1.smithi192.stdout:5/436: creat d0/d13/d19/d83/f95 x:0 0 0 2024-04-24T22:45:34.050 INFO:tasks.workunit.client.0.smithi063.stdout:3/675: creat da/d18/fea x:0 0 0 2024-04-24T22:45:34.050 INFO:tasks.workunit.client.1.smithi192.stdout:9/641: rmdir d3/d59/d7d 39 2024-04-24T22:45:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:33 smithi063 ceph-mon[26912]: pgmap v14: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 300 MiB/s wr, 536 op/s 2024-04-24T22:45:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:33 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:33 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:34.054 INFO:tasks.workunit.client.1.smithi192.stdout:3/532: truncate d2/d4/d7/d4f/f7a 75410 0 2024-04-24T22:45:34.055 INFO:tasks.workunit.client.1.smithi192.stdout:3/533: chown d2/d4/d7/d4f/c57 14747854 1 2024-04-24T22:45:34.056 INFO:tasks.workunit.client.0.smithi063.stdout:8/679: creat d1/dc0/fe0 x:0 0 0 2024-04-24T22:45:34.057 INFO:tasks.workunit.client.1.smithi192.stdout:2/641: creat d4/d9/d10/fcf x:0 0 0 2024-04-24T22:45:34.059 INFO:tasks.workunit.client.1.smithi192.stdout:6/462: truncate d0/d8/fe 982345 0 2024-04-24T22:45:34.062 INFO:tasks.workunit.client.0.smithi063.stdout:2/750: rename d0/d2/d2d/d31/d68 to d0/d2/d35/dc6/de2 0 2024-04-24T22:45:34.062 INFO:tasks.workunit.client.0.smithi063.stdout:2/751: readlink d0/d2/d2f/dba/ldb 0 2024-04-24T22:45:34.063 INFO:tasks.workunit.client.1.smithi192.stdout:1/455: creat d9/d6f/f9b x:0 0 0 2024-04-24T22:45:34.070 INFO:tasks.workunit.client.0.smithi063.stdout:2/752: dread d0/d2/d2d/f4d [0,4194304] 0 2024-04-24T22:45:34.071 INFO:tasks.workunit.client.0.smithi063.stdout:0/612: truncate d6/d64/d21/d80/f9e 646563 0 2024-04-24T22:45:34.071 INFO:tasks.workunit.client.1.smithi192.stdout:2/642: truncate d4/d9/f2c 373123 0 2024-04-24T22:45:34.072 INFO:tasks.workunit.client.1.smithi192.stdout:2/643: readlink d4/d9/d10/d6c/d6f/l96 0 2024-04-24T22:45:34.073 INFO:tasks.workunit.client.1.smithi192.stdout:0/556: creat d3/fc0 x:0 0 0 2024-04-24T22:45:34.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/733: getdents d3/da 0 2024-04-24T22:45:34.077 INFO:tasks.workunit.client.0.smithi063.stdout:5/639: dread d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:34.078 INFO:tasks.workunit.client.1.smithi192.stdout:9/642: rename d3/dd/c25 to d3/d59/d7d/ce3 0 2024-04-24T22:45:34.080 INFO:tasks.workunit.client.0.smithi063.stdout:9/684: creat d6/d2f/dcd/d1f/d20/d2a/d4b/fdc x:0 0 0 2024-04-24T22:45:34.081 INFO:tasks.workunit.client.1.smithi192.stdout:2/644: stat d4/d9/l72 0 2024-04-24T22:45:34.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/676: mknod da/d18/d47/d4d/ceb 0 2024-04-24T22:45:34.084 INFO:tasks.workunit.client.0.smithi063.stdout:8/680: fdatasync d1/d22/d33/da6/d54/d7b/fb9 0 2024-04-24T22:45:34.085 INFO:tasks.workunit.client.0.smithi063.stdout:8/681: fsync d1/d2/fbc 0 2024-04-24T22:45:34.086 INFO:tasks.workunit.client.1.smithi192.stdout:3/534: write d2/d27/d28/f63 [1276667,88956] 0 2024-04-24T22:45:34.087 INFO:tasks.workunit.client.1.smithi192.stdout:3/535: fsync d2/fa2 0 2024-04-24T22:45:34.088 INFO:tasks.workunit.client.1.smithi192.stdout:6/463: mkdir d0/d8/d3d/d9d 0 2024-04-24T22:45:34.092 INFO:tasks.workunit.client.0.smithi063.stdout:6/675: creat db/d1d/d20/d29/dab/dd0/fe3 x:0 0 0 2024-04-24T22:45:34.093 INFO:tasks.workunit.client.1.smithi192.stdout:9/643: chown d3/dd/d62/d6d/l83 10871 1 2024-04-24T22:45:34.098 INFO:tasks.workunit.client.0.smithi063.stdout:1/734: write d3/f6c [1887818,56053] 0 2024-04-24T22:45:34.101 INFO:tasks.workunit.client.1.smithi192.stdout:3/536: truncate d2/d4/d7/f17 4811072 0 2024-04-24T22:45:34.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/464: creat d0/d8/d29/d4c/d4d/f9e x:0 0 0 2024-04-24T22:45:34.102 INFO:tasks.workunit.client.1.smithi192.stdout:0/557: write d3/d27/f43 [1822161,98730] 0 2024-04-24T22:45:34.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/685: fdatasync d6/f7 0 2024-04-24T22:45:34.104 INFO:tasks.workunit.client.1.smithi192.stdout:1/456: creat d9/d51/f9c x:0 0 0 2024-04-24T22:45:34.106 INFO:tasks.workunit.client.1.smithi192.stdout:6/465: dread d0/d8/d29/d5a/f60 [0,4194304] 0 2024-04-24T22:45:34.108 INFO:tasks.workunit.client.0.smithi063.stdout:3/677: dread da/d18/d47/d4d/f8c [0,4194304] 0 2024-04-24T22:45:34.109 INFO:tasks.workunit.client.0.smithi063.stdout:9/686: dwrite d6/d2f/dcd/db/d93/da0/fae [0,4194304] 0 2024-04-24T22:45:34.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/676: rename db/da0 to db/d1d/d2d/d69/dd4/de4 0 2024-04-24T22:45:34.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/677: chown db/d1d/d8d 75112 1 2024-04-24T22:45:34.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/558: mknod d3/d27/d6a/cc1 0 2024-04-24T22:45:34.119 INFO:tasks.workunit.client.0.smithi063.stdout:8/682: write d1/d11/f47 [1149518,30821] 0 2024-04-24T22:45:34.119 INFO:tasks.workunit.client.0.smithi063.stdout:1/735: dread - d3/da/da2/dba/d39/d5a/f97 zero size 2024-04-24T22:45:34.122 INFO:tasks.workunit.client.0.smithi063.stdout:5/640: write d5/d1d/d7d/d67/f78 [398073,37692] 0 2024-04-24T22:45:34.123 INFO:tasks.workunit.client.1.smithi192.stdout:6/466: creat d0/d8/d29/d58/f9f x:0 0 0 2024-04-24T22:45:34.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/753: truncate d0/d2/f6a 1661009 0 2024-04-24T22:45:34.124 INFO:tasks.workunit.client.1.smithi192.stdout:1/457: write d9/d15/f1f [1376090,2834] 0 2024-04-24T22:45:34.125 INFO:tasks.workunit.client.1.smithi192.stdout:3/537: mknod d2/d4/d12/d39/ca7 0 2024-04-24T22:45:34.127 INFO:tasks.workunit.client.1.smithi192.stdout:0/559: mkdir d3/d40/dc2 0 2024-04-24T22:45:34.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/644: rename d3/d59/d7d/dc6/ccd to d3/dd/ce4 0 2024-04-24T22:45:34.131 INFO:tasks.workunit.client.1.smithi192.stdout:6/467: creat d0/d8/d18/d2e/fa0 x:0 0 0 2024-04-24T22:45:34.132 INFO:tasks.workunit.client.0.smithi063.stdout:3/678: unlink da/d18/d9e/da3/dc1/ccc 0 2024-04-24T22:45:34.133 INFO:tasks.workunit.client.1.smithi192.stdout:1/458: mkdir d9/d15/d2d/d63/d68/d9d 0 2024-04-24T22:45:34.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/459: readlink d9/d15/d2d/l91 0 2024-04-24T22:45:34.134 INFO:tasks.workunit.client.0.smithi063.stdout:9/687: mkdir d6/d2f/dcd/db/d1c/d78/d9a/ddd 0 2024-04-24T22:45:34.135 INFO:tasks.workunit.client.1.smithi192.stdout:1/460: chown d9/d15/d2d/d31/f99 478928732 1 2024-04-24T22:45:34.136 INFO:tasks.workunit.client.0.smithi063.stdout:6/678: mkdir db/d1d/d2d/d48/de5 0 2024-04-24T22:45:34.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/679: chown l9 3 1 2024-04-24T22:45:34.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/468: creat d0/d8/d3d/d3e/d7e/fa1 x:0 0 0 2024-04-24T22:45:34.141 INFO:tasks.workunit.client.1.smithi192.stdout:1/461: dread - d9/d15/d2d/d31/f62 zero size 2024-04-24T22:45:34.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/469: fdatasync d0/d8/f20 0 2024-04-24T22:45:34.145 INFO:tasks.workunit.client.0.smithi063.stdout:9/688: dread d6/d2f/dcd/db/d1c/d78/d9a/daa/fab [0,4194304] 0 2024-04-24T22:45:34.146 INFO:tasks.workunit.client.0.smithi063.stdout:6/680: mkdir db/d85/dd2/de6 0 2024-04-24T22:45:34.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/681: write db/d1c/dc3/fc8 [722570,111017] 0 2024-04-24T22:45:34.147 INFO:tasks.workunit.client.1.smithi192.stdout:0/560: creat d3/d27/d28/fc3 x:0 0 0 2024-04-24T22:45:34.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/538: dread d2/d4/d7/f9 [0,4194304] 0 2024-04-24T22:45:34.149 INFO:tasks.workunit.client.0.smithi063.stdout:5/641: link d5/d24/f51 d5/d24/d36/d65/fde 0 2024-04-24T22:45:34.150 INFO:tasks.workunit.client.1.smithi192.stdout:1/462: fdatasync d9/dd/f41 0 2024-04-24T22:45:34.150 INFO:tasks.workunit.client.1.smithi192.stdout:1/463: dread - d9/d15/d2d/d48/d93/f98 zero size 2024-04-24T22:45:34.211 INFO:tasks.workunit.client.0.smithi063.stdout:4/639: sync 2024-04-24T22:45:34.212 INFO:tasks.workunit.client.0.smithi063.stdout:0/613: sync 2024-04-24T22:45:34.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/600: sync 2024-04-24T22:45:34.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/470: chown d0/c92 110250237 1 2024-04-24T22:45:34.217 INFO:tasks.workunit.client.0.smithi063.stdout:6/682: mknod db/d1d/d2d/d69/d82/d96/ce7 0 2024-04-24T22:45:34.218 INFO:tasks.workunit.client.1.smithi192.stdout:0/561: mkdir d3/d27/d6a/dc4 0 2024-04-24T22:45:34.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/539: symlink d2/d27/la8 0 2024-04-24T22:45:34.220 INFO:tasks.workunit.client.0.smithi063.stdout:8/683: dread d1/f2c [0,4194304] 0 2024-04-24T22:45:34.220 INFO:tasks.workunit.client.0.smithi063.stdout:2/754: link d0/d2/d6c/c8a d0/d2/d2d/d31/d89/d1c/d3f/dca/ce3 0 2024-04-24T22:45:34.221 INFO:tasks.workunit.client.1.smithi192.stdout:1/464: fdatasync d9/d51/f54 0 2024-04-24T22:45:34.222 INFO:tasks.workunit.client.1.smithi192.stdout:1/465: dread - d9/d51/d70/f95 zero size 2024-04-24T22:45:34.223 INFO:tasks.workunit.client.1.smithi192.stdout:1/466: chown d9/d15/d2d/d31/d2f/f85 1994251 1 2024-04-24T22:45:34.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/736: dread d3/da/f5c [0,4194304] 0 2024-04-24T22:45:34.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/684: read d1/d22/d33/da6/d54/d7b/f7f [19612,82505] 0 2024-04-24T22:45:34.225 INFO:tasks.workunit.client.0.smithi063.stdout:6/683: dwrite db/d1c/dc3/fc8 [0,4194304] 0 2024-04-24T22:45:34.225 INFO:tasks.workunit.client.0.smithi063.stdout:4/640: creat d2/d19/d33/dc1/fd2 x:0 0 0 2024-04-24T22:45:34.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/614: read d6/da/d2b/d60/fa1 [191961,22490] 0 2024-04-24T22:45:34.232 INFO:tasks.workunit.client.0.smithi063.stdout:8/685: dwrite d1/d22/d33/da6/d54/f8e [0,4194304] 0 2024-04-24T22:45:34.233 INFO:tasks.workunit.client.1.smithi192.stdout:3/540: truncate d2/fa2 3736230 0 2024-04-24T22:45:34.236 INFO:tasks.workunit.client.0.smithi063.stdout:2/755: mkdir d0/d2/d35/dc6/de2/de4 0 2024-04-24T22:45:34.239 INFO:tasks.workunit.client.1.smithi192.stdout:6/471: getdents d0/d8/d3d/d3e/d7e 0 2024-04-24T22:45:34.241 INFO:tasks.workunit.client.1.smithi192.stdout:3/541: dwrite d2/d4/d7/d4f/f5a [0,4194304] 0 2024-04-24T22:45:34.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/641: truncate d2/d8/d40/d51/f64 41566 0 2024-04-24T22:45:34.249 INFO:tasks.workunit.client.1.smithi192.stdout:6/472: symlink d0/d8/d3d/la2 0 2024-04-24T22:45:34.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/473: mknod d0/d8/d18/ca3 0 2024-04-24T22:45:34.255 INFO:tasks.workunit.client.0.smithi063.stdout:9/689: link d6/d2f/dcd/db/d6b/d4f/dad/l64 d6/d2f/dcd/d1f/d20/d2a/lde 0 2024-04-24T22:45:34.255 INFO:tasks.workunit.client.0.smithi063.stdout:4/642: chown d2/d80 400 1 2024-04-24T22:45:34.255 INFO:tasks.workunit.client.0.smithi063.stdout:0/615: rename d6/d64/d21/lb2 to d6/d64/d21/d25/d62/d63/d89/d95/da4/ld8 0 2024-04-24T22:45:34.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/643: chown d2/d8/d40 12375799 1 2024-04-24T22:45:34.258 INFO:tasks.workunit.client.0.smithi063.stdout:5/642: link d5/d11/d23/d2d/d62/d81/da4/da6/fc6 d5/d24/d49/fdf 0 2024-04-24T22:45:34.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/684: dread db/d1d/d20/d3f/fc5 [0,4194304] 0 2024-04-24T22:45:34.258 INFO:tasks.workunit.client.0.smithi063.stdout:1/737: write f1 [1404609,64156] 0 2024-04-24T22:45:34.264 INFO:tasks.workunit.client.0.smithi063.stdout:7/601: dwrite d0/d4/de/f7c [0,4194304] 0 2024-04-24T22:45:34.266 INFO:tasks.workunit.client.0.smithi063.stdout:2/756: dread d0/d2/d27/fc4 [0,4194304] 0 2024-04-24T22:45:34.266 INFO:tasks.workunit.client.1.smithi192.stdout:3/542: dwrite d2/d4/d7/f2e [0,4194304] 0 2024-04-24T22:45:34.267 INFO:tasks.workunit.client.0.smithi063.stdout:2/757: read - d0/dd/d72/d77/d87/fc8 zero size 2024-04-24T22:45:34.268 INFO:tasks.workunit.client.0.smithi063.stdout:3/679: sync 2024-04-24T22:45:34.275 INFO:tasks.workunit.client.1.smithi192.stdout:3/543: rmdir d2/d4/d12 39 2024-04-24T22:45:34.276 INFO:tasks.workunit.client.1.smithi192.stdout:3/544: dread - d2/d4/d7/d4f/f8c zero size 2024-04-24T22:45:34.285 INFO:tasks.workunit.client.1.smithi192.stdout:3/545: dwrite d2/d4/d12/f88 [0,4194304] 0 2024-04-24T22:45:34.286 INFO:tasks.workunit.client.1.smithi192.stdout:3/546: chown d2/d4/d7/f77 2592342 1 2024-04-24T22:45:34.287 INFO:tasks.workunit.client.1.smithi192.stdout:3/547: dread - d2/d4/d12/d1a/f81 zero size 2024-04-24T22:45:34.288 INFO:tasks.workunit.client.1.smithi192.stdout:3/548: fdatasync d2/f33 0 2024-04-24T22:45:34.294 INFO:tasks.workunit.client.1.smithi192.stdout:3/549: dwrite d2/d4/f3e [0,4194304] 0 2024-04-24T22:45:34.294 INFO:tasks.workunit.client.0.smithi063.stdout:8/686: mkdir d1/d22/d33/da6/d54/d79/de1 0 2024-04-24T22:45:34.295 INFO:tasks.workunit.client.1.smithi192.stdout:3/550: chown d2/d4/d12/d1a/l99 2 1 2024-04-24T22:45:34.296 INFO:tasks.workunit.client.0.smithi063.stdout:8/687: read d1/d22/f5b [163440,59837] 0 2024-04-24T22:45:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/690: creat d6/d2f/dcd/db/d93/da0/fdf x:0 0 0 2024-04-24T22:45:34.300 INFO:tasks.workunit.client.0.smithi063.stdout:4/644: truncate d2/d19/f2b 4287758 0 2024-04-24T22:45:34.301 INFO:tasks.workunit.client.0.smithi063.stdout:4/645: fsync d2/ff 0 2024-04-24T22:45:34.301 INFO:tasks.workunit.client.1.smithi192.stdout:3/551: readlink d2/d4/d12/d39/d97/l45 0 2024-04-24T22:45:34.306 INFO:tasks.workunit.client.1.smithi192.stdout:3/552: truncate d2/d4/f14 259026 0 2024-04-24T22:45:34.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/758: symlink d0/d2/d35/dc6/de2/le5 0 2024-04-24T22:45:34.306 INFO:tasks.workunit.client.0.smithi063.stdout:3/680: rename da/d39/l9d to da/db6/lec 0 2024-04-24T22:45:34.308 INFO:tasks.workunit.client.1.smithi192.stdout:3/553: rename d2/d4/l21 to d2/d4/d4e/la9 0 2024-04-24T22:45:34.309 INFO:tasks.workunit.client.1.smithi192.stdout:3/554: write d2/d4/d7/f87 [128176,101455] 0 2024-04-24T22:45:34.309 INFO:tasks.workunit.client.0.smithi063.stdout:4/646: dwrite d2/d8/d36/faf [0,4194304] 0 2024-04-24T22:45:34.311 INFO:tasks.workunit.client.0.smithi063.stdout:6/685: dread db/d1d/d2d/f4d [0,4194304] 0 2024-04-24T22:45:34.314 INFO:tasks.workunit.client.1.smithi192.stdout:3/555: creat d2/d27/d28/da5/d93/faa x:0 0 0 2024-04-24T22:45:34.318 INFO:tasks.workunit.client.1.smithi192.stdout:3/556: mkdir d2/d27/d28/dab 0 2024-04-24T22:45:34.319 INFO:tasks.workunit.client.0.smithi063.stdout:6/686: dwrite db/d1d/fc9 [0,4194304] 0 2024-04-24T22:45:34.324 INFO:tasks.workunit.client.1.smithi192.stdout:3/557: mkdir d2/d4/d7/d41/dac 0 2024-04-24T22:45:34.327 INFO:tasks.workunit.client.0.smithi063.stdout:9/691: dread d6/d2f/dcd/db/f62 [0,4194304] 0 2024-04-24T22:45:34.327 INFO:tasks.workunit.client.0.smithi063.stdout:8/688: rename d1/d11/d8b/l89 to d1/d11/d32/dd3/le2 0 2024-04-24T22:45:34.330 INFO:tasks.workunit.client.0.smithi063.stdout:8/689: write d1/d11/d32/d38/fa0 [353271,66147] 0 2024-04-24T22:45:34.330 INFO:tasks.workunit.client.1.smithi192.stdout:3/558: mkdir d2/d4/d7/d41/dad 0 2024-04-24T22:45:34.330 INFO:tasks.workunit.client.0.smithi063.stdout:6/687: chown db/d1d/d20/d3e/c8c 768 1 2024-04-24T22:45:34.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/738: getdents d3/db5/d21/d9f/dd5 0 2024-04-24T22:45:34.333 INFO:tasks.workunit.client.0.smithi063.stdout:5/643: write d5/d24/d36/d65/fde [36377,3637] 0 2024-04-24T22:45:34.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/681: creat da/d18/d9e/fed x:0 0 0 2024-04-24T22:45:34.336 INFO:tasks.workunit.client.0.smithi063.stdout:7/602: dread d0/d4/de/d6f/f53 [0,4194304] 0 2024-04-24T22:45:34.338 INFO:tasks.workunit.client.0.smithi063.stdout:7/603: read d0/d4/de/d17/d28/f4a [2774826,113538] 0 2024-04-24T22:45:34.338 INFO:tasks.workunit.client.0.smithi063.stdout:4/647: rename d2/d19/d82/f84 to d2/d8/d36/da3/fd3 0 2024-04-24T22:45:34.339 INFO:tasks.workunit.client.0.smithi063.stdout:7/604: readlink d0/d4/de/d6f/lba 0 2024-04-24T22:45:34.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/692: write d6/d2f/dcd/db/f62 [3219770,18642] 0 2024-04-24T22:45:34.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/559: dwrite d2/d4/d7/d4f/f52 [0,4194304] 0 2024-04-24T22:45:34.341 INFO:tasks.workunit.client.1.smithi192.stdout:3/560: fdatasync d2/d4/d12/d39/d86/fa1 0 2024-04-24T22:45:34.342 INFO:tasks.workunit.client.1.smithi192.stdout:3/561: fsync d2/d27/d28/f63 0 2024-04-24T22:45:34.343 INFO:tasks.workunit.client.0.smithi063.stdout:7/605: dread d0/d4/de/f7c [0,4194304] 0 2024-04-24T22:45:34.346 INFO:tasks.workunit.client.0.smithi063.stdout:0/616: write d6/d64/d21/d80/fd7 [2776015,5342] 0 2024-04-24T22:45:34.347 INFO:tasks.workunit.client.1.smithi192.stdout:3/562: mknod d2/d4/d12/d1a/d1d/d71/cae 0 2024-04-24T22:45:34.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/617: truncate d6/d64/da0/d33/f99 203904 0 2024-04-24T22:45:34.351 INFO:tasks.workunit.client.1.smithi192.stdout:3/563: truncate d2/d4/d7/f47 670872 0 2024-04-24T22:45:34.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/564: write d2/d4/d12/d1a/d1d/d55/f24 [4133049,72759] 0 2024-04-24T22:45:34.356 INFO:tasks.workunit.client.0.smithi063.stdout:4/648: dread d2/d8/d40/d51/f64 [0,4194304] 0 2024-04-24T22:45:34.359 INFO:tasks.workunit.client.0.smithi063.stdout:8/690: dread d1/d2/d3/f56 [0,4194304] 0 2024-04-24T22:45:34.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/693: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/de0 0 2024-04-24T22:45:34.362 INFO:tasks.workunit.client.0.smithi063.stdout:7/606: symlink d0/d4/de/d17/d28/d31/lcc 0 2024-04-24T22:45:34.364 INFO:tasks.workunit.client.0.smithi063.stdout:6/688: truncate db/f2a 835273 0 2024-04-24T22:45:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:7/607: dread d0/d4/de/d17/fc6 [0,4194304] 0 2024-04-24T22:45:34.366 INFO:tasks.workunit.client.0.smithi063.stdout:5/644: fdatasync d5/d24/d44/f46 0 2024-04-24T22:45:34.369 INFO:tasks.workunit.client.0.smithi063.stdout:0/618: rename d6/d64/d21/d25/d62/d63/d89/d95/da4/c93 to d6/d64/da0/d98/cd9 0 2024-04-24T22:45:34.376 INFO:tasks.workunit.client.1.smithi192.stdout:3/565: dread d2/d4/f16 [0,4194304] 0 2024-04-24T22:45:34.379 INFO:tasks.workunit.client.0.smithi063.stdout:1/739: creat d3/ff2 x:0 0 0 2024-04-24T22:45:34.382 INFO:tasks.workunit.client.0.smithi063.stdout:9/694: write d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e/f98 [1648133,103154] 0 2024-04-24T22:45:34.385 INFO:tasks.workunit.client.0.smithi063.stdout:5/645: unlink d5/d11/ca0 0 2024-04-24T22:45:34.385 INFO:tasks.workunit.client.0.smithi063.stdout:8/691: dread d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:34.386 INFO:tasks.workunit.client.0.smithi063.stdout:8/692: fsync d1/d22/d33/fc4 0 2024-04-24T22:45:34.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/759: sync 2024-04-24T22:45:34.387 INFO:tasks.workunit.client.0.smithi063.stdout:4/649: mkdir d2/d8/dd4 0 2024-04-24T22:45:34.388 INFO:tasks.workunit.client.0.smithi063.stdout:8/693: chown d1/d21/f6f 3466 1 2024-04-24T22:45:34.392 INFO:tasks.workunit.client.1.smithi192.stdout:3/566: dwrite d2/d4/d7/d4f/f7a [0,4194304] 0 2024-04-24T22:45:34.392 INFO:tasks.workunit.client.0.smithi063.stdout:6/689: dwrite db/d1c/f90 [4194304,4194304] 0 2024-04-24T22:45:34.398 INFO:tasks.workunit.client.0.smithi063.stdout:2/760: dwrite d0/fa6 [0,4194304] 0 2024-04-24T22:45:34.399 INFO:tasks.workunit.client.0.smithi063.stdout:3/682: dwrite da/d39/f98 [4194304,4194304] 0 2024-04-24T22:45:34.425 INFO:tasks.workunit.client.0.smithi063.stdout:9/695: truncate d6/d2f/dcd/db/d1c/d78/d9a/daa/fab 1078396 0 2024-04-24T22:45:34.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/740: mknod d3/cf3 0 2024-04-24T22:45:34.429 INFO:tasks.workunit.client.0.smithi063.stdout:4/650: mkdir d2/d8/d40/d51/dd5 0 2024-04-24T22:45:34.430 INFO:tasks.workunit.client.1.smithi192.stdout:3/567: dread d2/d4/d12/f49 [0,4194304] 0 2024-04-24T22:45:34.431 INFO:tasks.workunit.client.0.smithi063.stdout:5/646: dwrite d5/d11/d23/fd5 [0,4194304] 0 2024-04-24T22:45:34.432 INFO:tasks.workunit.client.0.smithi063.stdout:7/608: link d0/d2f/f5a d0/d4/de/d6f/d5c/fcd 0 2024-04-24T22:45:34.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/609: write d0/d4/de/d6f/d5c/d8b/fae [4026229,20220] 0 2024-04-24T22:45:34.434 INFO:tasks.workunit.client.1.smithi192.stdout:3/568: unlink d2/d4/d7/d4b/d54/f6a 0 2024-04-24T22:45:34.438 INFO:tasks.workunit.client.0.smithi063.stdout:9/696: rename d6/d2f/dcd/d1f/d20/d2a/d4b/fd2 to d6/d2f/dcd/d53/dd5/fe1 0 2024-04-24T22:45:34.439 INFO:tasks.workunit.client.0.smithi063.stdout:1/741: mknod d3/da/cf4 0 2024-04-24T22:45:34.444 INFO:tasks.workunit.client.0.smithi063.stdout:8/694: write d1/d11/f1b [201155,23788] 0 2024-04-24T22:45:34.445 INFO:tasks.workunit.client.0.smithi063.stdout:2/761: creat d0/d2/d2d/d31/d89/d7/d86/da4/dd9/fe6 x:0 0 0 2024-04-24T22:45:34.448 INFO:tasks.workunit.client.0.smithi063.stdout:0/619: getdents d6/d64/da0/d98/dc7 0 2024-04-24T22:45:34.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/651: symlink d2/d8/ld6 0 2024-04-24T22:45:34.450 INFO:tasks.workunit.client.0.smithi063.stdout:5/647: write d5/d1d/d7d/f52 [1467175,102737] 0 2024-04-24T22:45:34.451 INFO:tasks.workunit.client.0.smithi063.stdout:6/690: dwrite db/d1d/d20/d3e/f4f [0,4194304] 0 2024-04-24T22:45:34.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/697: rename d6/d2f/dcd/d1f/d20/d2a/dcb/lcc to d6/d2f/dcd/db/d6b/d4f/dad/le2 0 2024-04-24T22:45:34.455 INFO:tasks.workunit.client.0.smithi063.stdout:2/762: creat d0/d2/d39/fe7 x:0 0 0 2024-04-24T22:45:34.460 INFO:tasks.workunit.client.0.smithi063.stdout:1/742: dread d3/da/da2/dba/d11/f41 [4194304,4194304] 0 2024-04-24T22:45:34.460 INFO:tasks.workunit.client.0.smithi063.stdout:7/610: truncate d0/f7 516505 0 2024-04-24T22:45:34.461 INFO:tasks.workunit.client.0.smithi063.stdout:3/683: write da/d39/d94/db3/fc8 [1501346,108760] 0 2024-04-24T22:45:34.462 INFO:tasks.workunit.client.0.smithi063.stdout:0/620: symlink d6/d64/d21/d25/d62/d63/d89/d95/lda 0 2024-04-24T22:45:34.462 INFO:tasks.workunit.client.0.smithi063.stdout:4/652: write d2/d8/d3c/f62 [464273,18263] 0 2024-04-24T22:45:34.463 INFO:tasks.workunit.client.0.smithi063.stdout:2/763: mkdir d0/d2/d2f/de8 0 2024-04-24T22:45:34.466 INFO:tasks.workunit.client.0.smithi063.stdout:8/695: rmdir d1/d22/d33/da6/d54/d79/de1 0 2024-04-24T22:45:34.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/684: creat da/d18/d3a/da1/fee x:0 0 0 2024-04-24T22:45:34.469 INFO:tasks.workunit.client.0.smithi063.stdout:0/621: dwrite d6/d64/d21/f36 [0,4194304] 0 2024-04-24T22:45:34.472 INFO:tasks.workunit.client.0.smithi063.stdout:9/698: link d6/d2f/dcd/db/d1c/d78/d9a/daa/fd4 d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fe3 0 2024-04-24T22:45:34.475 INFO:tasks.workunit.client.0.smithi063.stdout:8/696: dwrite d1/f62 [0,4194304] 0 2024-04-24T22:45:34.477 INFO:tasks.workunit.client.0.smithi063.stdout:5/648: write d5/d24/d44/fbf [197787,115984] 0 2024-04-24T22:45:34.482 INFO:tasks.workunit.client.0.smithi063.stdout:6/691: truncate db/d1d/d2d/f36 3823575 0 2024-04-24T22:45:34.482 INFO:tasks.workunit.client.0.smithi063.stdout:5/649: chown d5/d11/d18/c1e 187 1 2024-04-24T22:45:34.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/743: write d3/da/d34/d68/fa8 [3361107,68713] 0 2024-04-24T22:45:34.485 INFO:tasks.workunit.client.0.smithi063.stdout:0/622: dwrite d6/d64/da0/d33/fc3 [0,4194304] 0 2024-04-24T22:45:34.485 INFO:tasks.workunit.client.0.smithi063.stdout:6/692: chown db/d85/dd2/de6 5381075 1 2024-04-24T22:45:34.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/697: mkdir d1/d11/d32/d38/db1/de3 0 2024-04-24T22:45:34.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/693: dread - db/d1c/dc2/fd3 zero size 2024-04-24T22:45:34.488 INFO:tasks.workunit.client.0.smithi063.stdout:3/685: link da/d18/d47/l6d da/d18/d9e/da3/lef 0 2024-04-24T22:45:34.490 INFO:tasks.workunit.client.0.smithi063.stdout:6/694: stat db/d1c 0 2024-04-24T22:45:34.491 INFO:tasks.workunit.client.0.smithi063.stdout:9/699: dwrite d6/d2f/dcd/f71 [0,4194304] 0 2024-04-24T22:45:34.491 INFO:tasks.workunit.client.0.smithi063.stdout:5/650: rename d5/f5b to d5/d24/db6/fe0 0 2024-04-24T22:45:34.494 INFO:tasks.workunit.client.0.smithi063.stdout:0/623: symlink d6/d64/d21/ldb 0 2024-04-24T22:45:34.494 INFO:tasks.workunit.client.0.smithi063.stdout:3/686: chown da/d18/d47/d4d/d62/f6f 3182 1 2024-04-24T22:45:34.495 INFO:tasks.workunit.client.0.smithi063.stdout:4/653: dread d2/d8/fe [0,4194304] 0 2024-04-24T22:45:34.496 INFO:tasks.workunit.client.0.smithi063.stdout:1/744: mkdir d3/da/da2/dba/df5 0 2024-04-24T22:45:34.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/764: dwrite d0/d2/d2d/d31/d89/d7/f19 [0,4194304] 0 2024-04-24T22:45:34.500 INFO:tasks.workunit.client.0.smithi063.stdout:0/624: write d6/d64/da0/d33/f99 [626677,31276] 0 2024-04-24T22:45:34.501 INFO:tasks.workunit.client.0.smithi063.stdout:6/695: fsync db/d1d/d20/d3e/f74 0 2024-04-24T22:45:34.504 INFO:tasks.workunit.client.0.smithi063.stdout:7/611: truncate d0/d4/f19 725037 0 2024-04-24T22:45:34.505 INFO:tasks.workunit.client.0.smithi063.stdout:2/765: write d0/d2/d39/fe7 [812728,50498] 0 2024-04-24T22:45:34.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/687: unlink da/d18/d3a/d63/f64 0 2024-04-24T22:45:34.510 INFO:tasks.workunit.client.0.smithi063.stdout:1/745: mknod d3/da/d34/d6d/cf6 0 2024-04-24T22:45:34.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/698: creat d1/d11/fe4 x:0 0 0 2024-04-24T22:45:34.510 INFO:tasks.workunit.client.0.smithi063.stdout:0/625: write d6/d64/d21/d25/d62/d63/f6e [1682508,7173] 0 2024-04-24T22:45:34.510 INFO:tasks.workunit.client.0.smithi063.stdout:9/700: rename d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/fb4 to d6/d2f/dcd/db/d6b/d4f/dad/d36/fe4 0 2024-04-24T22:45:34.511 INFO:tasks.workunit.client.0.smithi063.stdout:3/688: chown da/d18/d3a/l8e 261251 1 2024-04-24T22:45:34.511 INFO:tasks.workunit.client.0.smithi063.stdout:5/651: dwrite d5/d11/d23/f79 [0,4194304] 0 2024-04-24T22:45:34.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/696: mknod db/d1d/d20/d3f/d7f/d68/d70/ce8 0 2024-04-24T22:45:34.516 INFO:tasks.workunit.client.0.smithi063.stdout:8/699: chown d1/d11/d32/l7a 75 1 2024-04-24T22:45:34.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/746: creat d3/db5/d21/d49/ff7 x:0 0 0 2024-04-24T22:45:34.520 INFO:tasks.workunit.client.0.smithi063.stdout:5/652: creat d5/d24/d49/d3b/fe1 x:0 0 0 2024-04-24T22:45:34.521 INFO:tasks.workunit.client.0.smithi063.stdout:6/697: rmdir db/d1d/d20/d3e 39 2024-04-24T22:45:34.521 INFO:tasks.workunit.client.0.smithi063.stdout:2/766: creat d0/d2/d35/dc6/de2/de4/fe9 x:0 0 0 2024-04-24T22:45:34.522 INFO:tasks.workunit.client.0.smithi063.stdout:9/701: rename d6/d2f/dcd/db/d6b/d4f/d59/f76 to d6/d2f/dcd/db/d6b/d4f/dad/d36/fe5 0 2024-04-24T22:45:34.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/747: readlink d3/da/da2/dba/d39/dbf/lc8 0 2024-04-24T22:45:34.528 INFO:tasks.workunit.client.0.smithi063.stdout:3/689: dwrite da/d39/d94/db3/fdf [0,4194304] 0 2024-04-24T22:45:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:0/626: dwrite d6/d3a/fc5 [0,4194304] 0 2024-04-24T22:45:34.530 INFO:tasks.workunit.client.0.smithi063.stdout:3/690: dread - da/d18/d9e/fe7 zero size 2024-04-24T22:45:34.531 INFO:tasks.workunit.client.0.smithi063.stdout:0/627: write d6/d64/d24/f8b [1811232,98611] 0 2024-04-24T22:45:34.531 INFO:tasks.workunit.client.0.smithi063.stdout:8/700: mkdir d1/d22/d33/da6/d91/de5 0 2024-04-24T22:45:34.532 INFO:tasks.workunit.client.0.smithi063.stdout:9/702: unlink d6/d2f/dcd/db/f62 0 2024-04-24T22:45:34.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/698: creat db/d1d/d20/d3f/d7f/d68/fe9 x:0 0 0 2024-04-24T22:45:34.534 INFO:tasks.workunit.client.0.smithi063.stdout:3/691: dread da/d18/d3a/da1/fbd [0,4194304] 0 2024-04-24T22:45:34.536 INFO:tasks.workunit.client.0.smithi063.stdout:1/748: mkdir d3/de0/d79/d96/df8 0 2024-04-24T22:45:34.537 INFO:tasks.workunit.client.0.smithi063.stdout:9/703: truncate d6/d2f/dcd/db/d6b/d4f/d59/f9b 952234 0 2024-04-24T22:45:34.537 INFO:tasks.workunit.client.0.smithi063.stdout:1/749: stat d3/da/da2/dba/d39 0 2024-04-24T22:45:34.539 INFO:tasks.workunit.client.0.smithi063.stdout:2/767: creat d0/d2/d2d/d31/d89/d7/d86/da4/dd9/fea x:0 0 0 2024-04-24T22:45:34.539 INFO:tasks.workunit.client.0.smithi063.stdout:9/704: chown d6/d2f/dcd/d1f/d20/fbc 28 1 2024-04-24T22:45:34.540 INFO:tasks.workunit.client.0.smithi063.stdout:4/654: dread d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:34.547 INFO:tasks.workunit.client.0.smithi063.stdout:3/692: mknod da/d18/d9e/da3/cf0 0 2024-04-24T22:45:34.549 INFO:tasks.workunit.client.0.smithi063.stdout:1/750: rmdir d3/da/da2/dba 39 2024-04-24T22:45:34.550 INFO:tasks.workunit.client.0.smithi063.stdout:1/751: fsync d3/f6c 0 2024-04-24T22:45:34.552 INFO:tasks.workunit.client.0.smithi063.stdout:2/768: mkdir d0/d2/d2d/d31/d89/d7/d86/da0/deb 0 2024-04-24T22:45:34.554 INFO:tasks.workunit.client.0.smithi063.stdout:3/693: dread - da/d18/d3a/f96 zero size 2024-04-24T22:45:34.556 INFO:tasks.workunit.client.0.smithi063.stdout:9/705: mkdir d6/d2f/dcd/d1f/d20/de6 0 2024-04-24T22:45:34.558 INFO:tasks.workunit.client.0.smithi063.stdout:4/655: mknod d2/d8/d36/cd7 0 2024-04-24T22:45:34.558 INFO:tasks.workunit.client.0.smithi063.stdout:7/612: dwrite d0/d4/de/d17/d35/d69/f6b [0,4194304] 0 2024-04-24T22:45:34.559 INFO:tasks.workunit.client.0.smithi063.stdout:7/613: chown d0/d4/de/d6f/d5c/d8b 157187 1 2024-04-24T22:45:34.560 INFO:tasks.workunit.client.0.smithi063.stdout:0/628: rename d6/d64/f3b to d6/d64/d21/d25/d62/d63/d89/d95/fdc 0 2024-04-24T22:45:34.560 INFO:tasks.workunit.client.0.smithi063.stdout:7/614: chown d0/d4/de/c50 5065 1 2024-04-24T22:45:34.561 INFO:tasks.workunit.client.0.smithi063.stdout:2/769: mkdir d0/d2/d2f/dec 0 2024-04-24T22:45:34.563 INFO:tasks.workunit.client.0.smithi063.stdout:7/615: write d0/fa [7538468,111575] 0 2024-04-24T22:45:34.564 INFO:tasks.workunit.client.0.smithi063.stdout:8/701: rmdir d1/d22/d33 39 2024-04-24T22:45:34.566 INFO:tasks.workunit.client.0.smithi063.stdout:1/752: fdatasync d3/da/da2/dba/d39/f66 0 2024-04-24T22:45:34.567 INFO:tasks.workunit.client.0.smithi063.stdout:5/653: dread d5/d11/d76/f7b [0,4194304] 0 2024-04-24T22:45:34.568 INFO:tasks.workunit.client.0.smithi063.stdout:5/654: fsync d5/d11/f6a 0 2024-04-24T22:45:34.568 INFO:tasks.workunit.client.0.smithi063.stdout:4/656: mknod d2/d9f/cd8 0 2024-04-24T22:45:34.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/706: creat d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/fe7 x:0 0 0 2024-04-24T22:45:34.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/699: rename db/d1d/d20/d29/d5d/da6 to db/d1c/dea 0 2024-04-24T22:45:34.573 INFO:tasks.workunit.client.0.smithi063.stdout:8/702: mkdir d1/d21/de6 0 2024-04-24T22:45:34.574 INFO:tasks.workunit.client.0.smithi063.stdout:8/703: write d1/d11/d8b/fa9 [1811601,85564] 0 2024-04-24T22:45:34.575 INFO:tasks.workunit.client.0.smithi063.stdout:7/616: creat d0/d4/de/d6f/d2e/d76/fce x:0 0 0 2024-04-24T22:45:34.596 INFO:tasks.workunit.client.1.smithi192.stdout:8/553: sync 2024-04-24T22:45:34.608 INFO:tasks.workunit.client.0.smithi063.stdout:1/753: creat d3/da/d34/d99/ff9 x:0 0 0 2024-04-24T22:45:34.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/629: rename d6/da/fc9 to d6/d64/d21/d25/d31/fdd 0 2024-04-24T22:45:34.610 INFO:tasks.workunit.client.0.smithi063.stdout:0/630: chown d6/d64/da0/d33/fc3 7262 1 2024-04-24T22:45:34.670 INFO:tasks.workunit.client.0.smithi063.stdout:6/700: truncate db/d1d/d20/d29/f9f 850413 0 2024-04-24T22:45:34.670 INFO:tasks.workunit.client.0.smithi063.stdout:5/655: sync 2024-04-24T22:45:34.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/657: mkdir d2/d8/d36/dd9 0 2024-04-24T22:45:34.677 INFO:tasks.workunit.client.1.smithi192.stdout:7/592: sync 2024-04-24T22:45:34.681 INFO:tasks.workunit.client.0.smithi063.stdout:0/631: readlink d6/da/d2b/l34 0 2024-04-24T22:45:34.682 INFO:tasks.workunit.client.0.smithi063.stdout:3/694: link da/d18/c44 da/d48/cf1 0 2024-04-24T22:45:34.685 INFO:tasks.workunit.client.0.smithi063.stdout:6/701: creat db/d1c/dc2/feb x:0 0 0 2024-04-24T22:45:34.692 INFO:tasks.workunit.client.0.smithi063.stdout:8/704: dread d1/d21/f3a [0,4194304] 0 2024-04-24T22:45:34.699 INFO:tasks.workunit.client.0.smithi063.stdout:6/702: mkdir db/d1d/d2d/d48/dec 0 2024-04-24T22:45:34.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/770: getdents d0/d2/d27/d3a 0 2024-04-24T22:45:34.702 INFO:tasks.workunit.client.0.smithi063.stdout:4/658: getdents d2/d19/d33/dc1/d50/dc3 0 2024-04-24T22:45:34.703 INFO:tasks.workunit.client.0.smithi063.stdout:4/659: chown d2/d19/f4d 0 1 2024-04-24T22:45:34.703 INFO:tasks.workunit.client.0.smithi063.stdout:0/632: dread f5 [0,4194304] 0 2024-04-24T22:45:34.704 INFO:tasks.workunit.client.0.smithi063.stdout:8/705: fsync d1/d11/d8b/f8d 0 2024-04-24T22:45:34.705 INFO:tasks.workunit.client.0.smithi063.stdout:7/617: link d0/d4/de/d17/d35/f79 d0/d4/de/d6f/d2e/d76/d9f/dbd/fcf 0 2024-04-24T22:45:34.705 INFO:tasks.workunit.client.0.smithi063.stdout:8/706: fsync d1/d2/f2f 0 2024-04-24T22:45:34.705 INFO:tasks.workunit.client.0.smithi063.stdout:4/660: chown d2/d19/d33/dc1/f4b 773 1 2024-04-24T22:45:34.706 INFO:tasks.workunit.client.0.smithi063.stdout:4/661: stat d2/d19/fc5 0 2024-04-24T22:45:34.706 INFO:tasks.workunit.client.0.smithi063.stdout:7/618: chown d0/d4/de/d6f/d2e/f85 15 1 2024-04-24T22:45:34.714 INFO:tasks.workunit.client.0.smithi063.stdout:4/662: dwrite d2/d80/d8d/f89 [0,4194304] 0 2024-04-24T22:45:34.714 INFO:tasks.workunit.client.0.smithi063.stdout:4/663: truncate d2/d8/d36/da3/fd3 4646324 0 2024-04-24T22:45:34.717 INFO:tasks.workunit.client.0.smithi063.stdout:8/707: dread d1/d11/f93 [0,4194304] 0 2024-04-24T22:45:34.720 INFO:tasks.workunit.client.0.smithi063.stdout:1/754: sync 2024-04-24T22:45:34.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/755: stat d3/de0/d79/d96/df8 0 2024-04-24T22:45:34.722 INFO:tasks.workunit.client.0.smithi063.stdout:1/756: write d3/da/d34/d6d/fb6 [1118097,30814] 0 2024-04-24T22:45:34.725 INFO:tasks.workunit.client.0.smithi063.stdout:4/664: rmdir d2/d19/dce 39 2024-04-24T22:45:34.726 INFO:tasks.workunit.client.0.smithi063.stdout:6/703: dread db/d1d/d20/d3f/d7f/d68/f6d [0,4194304] 0 2024-04-24T22:45:34.727 INFO:tasks.workunit.client.0.smithi063.stdout:6/704: fsync db/d1d/fc9 0 2024-04-24T22:45:34.728 INFO:tasks.workunit.client.0.smithi063.stdout:6/705: chown db/d1d/d20/d3f/fc5 28817 1 2024-04-24T22:45:34.730 INFO:tasks.workunit.client.0.smithi063.stdout:7/619: creat d0/d4/de/d6f/d5c/d8b/dc0/fd0 x:0 0 0 2024-04-24T22:45:34.734 INFO:tasks.workunit.client.0.smithi063.stdout:6/706: rmdir db/d1d/d2d/d48 39 2024-04-24T22:45:34.735 INFO:tasks.workunit.client.0.smithi063.stdout:6/707: write db/d1c/f90 [8066829,10039] 0 2024-04-24T22:45:34.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/771: getdents d0/d2/d2d/d31/d89/d1c/d3f 0 2024-04-24T22:45:34.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/772: dread d0/d2/d2d/d31/d89/d1c/f2a [0,4194304] 0 2024-04-24T22:45:34.741 INFO:tasks.workunit.client.0.smithi063.stdout:7/620: mkdir d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1 0 2024-04-24T22:45:34.745 INFO:tasks.workunit.client.0.smithi063.stdout:8/708: mkdir d1/d22/d33/da6/dae/de7 0 2024-04-24T22:45:34.747 INFO:tasks.workunit.client.0.smithi063.stdout:7/621: dwrite d0/d4/de/d17/d35/d69/fb0 [0,4194304] 0 2024-04-24T22:45:34.747 INFO:tasks.workunit.client.0.smithi063.stdout:2/773: mknod d0/d2/d2d/d31/d89/d7/d86/da0/d94/ced 0 2024-04-24T22:45:34.749 INFO:tasks.workunit.client.0.smithi063.stdout:8/709: creat d1/d11/d32/dd3/fe8 x:0 0 0 2024-04-24T22:45:34.750 INFO:tasks.workunit.client.0.smithi063.stdout:7/622: symlink d0/d4/de/d6f/d5c/ld2 0 2024-04-24T22:45:34.751 INFO:tasks.workunit.client.0.smithi063.stdout:8/710: chown d1/d11/d8b/f8d 183698701 1 2024-04-24T22:45:34.751 INFO:tasks.workunit.client.0.smithi063.stdout:8/711: dread - d1/d2/fba zero size 2024-04-24T22:45:34.752 INFO:tasks.workunit.client.0.smithi063.stdout:2/774: link d0/d2/d6c/fd8 d0/d2/d2d/d31/d89/d48/fee 0 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:0/562: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/437: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:2/645: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:4/460: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:1/467: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:6/474: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:9/645: sync 2024-04-24T22:45:34.757 INFO:tasks.workunit.client.1.smithi192.stdout:3/569: sync 2024-04-24T22:45:34.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/775: dwrite d0/d2/d2d/d31/d89/d48/f5d [0,4194304] 0 2024-04-24T22:45:34.759 INFO:tasks.workunit.client.0.smithi063.stdout:6/708: symlink db/d1d/d2d/d69/d82/led 0 2024-04-24T22:45:34.760 INFO:tasks.workunit.client.0.smithi063.stdout:6/709: dread - db/d1d/d20/d84/f9e zero size 2024-04-24T22:45:34.760 INFO:tasks.workunit.client.1.smithi192.stdout:9/646: chown d3/dd/f38 12 1 2024-04-24T22:45:34.760 INFO:tasks.workunit.client.0.smithi063.stdout:2/776: truncate d0/d2/d2f/f6d 4016909 0 2024-04-24T22:45:34.763 INFO:tasks.workunit.client.0.smithi063.stdout:5/656: link d5/d24/d49/f97 d5/d1d/d7d/fe2 0 2024-04-24T22:45:34.764 INFO:tasks.workunit.client.1.smithi192.stdout:1/468: symlink d9/d15/d5e/d90/l9e 0 2024-04-24T22:45:34.764 INFO:tasks.workunit.client.1.smithi192.stdout:6/475: dread d0/d8/d22/f32 [4194304,4194304] 0 2024-04-24T22:45:34.768 INFO:tasks.workunit.client.0.smithi063.stdout:5/657: dwrite d5/d24/f86 [4194304,4194304] 0 2024-04-24T22:45:34.771 INFO:tasks.workunit.client.1.smithi192.stdout:6/476: symlink d0/d8/d18/d2e/la4 0 2024-04-24T22:45:34.775 INFO:tasks.workunit.client.1.smithi192.stdout:1/469: rmdir d9/d15/d2d/d48/d93 39 2024-04-24T22:45:34.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/646: dwrite d4/d29/d60/f89 [0,4194304] 0 2024-04-24T22:45:34.781 INFO:tasks.workunit.client.0.smithi063.stdout:6/710: dread db/d1d/d2d/f36 [0,4194304] 0 2024-04-24T22:45:34.782 INFO:tasks.workunit.client.1.smithi192.stdout:4/461: unlink d2/db/d75/da1/d36/d5b/d28/f61 0 2024-04-24T22:45:34.783 INFO:tasks.workunit.client.1.smithi192.stdout:4/462: stat d2/db/d10/f11 0 2024-04-24T22:45:34.790 INFO:tasks.workunit.client.1.smithi192.stdout:3/570: unlink d2/d27/d28/f63 0 2024-04-24T22:45:34.800 INFO:tasks.workunit.client.1.smithi192.stdout:0/563: dwrite d3/d27/d28/d2e/d4a/f4b [0,4194304] 0 2024-04-24T22:45:34.802 INFO:tasks.workunit.client.1.smithi192.stdout:0/564: chown d3/d27/d3c/d6c/fae 14569776 1 2024-04-24T22:45:34.803 INFO:tasks.workunit.client.0.smithi063.stdout:6/711: link db/d1c/c35 db/d85/db5/cee 0 2024-04-24T22:45:34.805 INFO:tasks.workunit.client.1.smithi192.stdout:2/647: symlink d4/d29/ld0 0 2024-04-24T22:45:34.807 INFO:tasks.workunit.client.0.smithi063.stdout:5/658: link d5/d1d/dc0/lb3 d5/d24/d49/d3b/dc5/dda/le3 0 2024-04-24T22:45:34.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/647: truncate d3/f71 396811 0 2024-04-24T22:45:34.808 INFO:tasks.workunit.client.0.smithi063.stdout:5/659: truncate d5/d11/d23/fd5 4686942 0 2024-04-24T22:45:34.809 INFO:tasks.workunit.client.1.smithi192.stdout:9/648: chown d3/dd/d53/fd5 35 1 2024-04-24T22:45:34.814 INFO:tasks.workunit.client.0.smithi063.stdout:5/660: dread - d5/d24/d36/d65/fa7 zero size 2024-04-24T22:45:34.815 INFO:tasks.workunit.client.0.smithi063.stdout:5/661: stat d5/d11/d18/c1e 0 2024-04-24T22:45:34.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/662: creat d5/d24/d44/fe4 x:0 0 0 2024-04-24T22:45:34.821 INFO:tasks.workunit.client.0.smithi063.stdout:5/663: write d5/d11/f6a [5233294,30841] 0 2024-04-24T22:45:34.823 INFO:tasks.workunit.client.0.smithi063.stdout:5/664: read d5/d24/d36/f5a [430779,79390] 0 2024-04-24T22:45:34.824 INFO:tasks.workunit.client.1.smithi192.stdout:6/477: mkdir d0/d8/d3d/d9d/da5 0 2024-04-24T22:45:34.824 INFO:tasks.workunit.client.1.smithi192.stdout:6/478: dread - d0/d8/d18/d2e/f8d zero size 2024-04-24T22:45:34.825 INFO:tasks.workunit.client.1.smithi192.stdout:1/470: symlink d9/d15/d2d/d48/l9f 0 2024-04-24T22:45:34.826 INFO:tasks.workunit.client.1.smithi192.stdout:9/649: dread d3/dd/f6c [4194304,4194304] 0 2024-04-24T22:45:34.828 INFO:tasks.workunit.client.0.smithi063.stdout:5/665: dwrite d5/d24/f91 [0,4194304] 0 2024-04-24T22:45:34.829 INFO:tasks.workunit.client.1.smithi192.stdout:5/438: creat d0/d13/d3b/d2f/f96 x:0 0 0 2024-04-24T22:45:34.829 INFO:tasks.workunit.client.1.smithi192.stdout:5/439: chown d0/d13/d3b/f3a 343587 1 2024-04-24T22:45:34.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/666: truncate d5/d24/d49/f97 115805 0 2024-04-24T22:45:34.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/667: chown d5/d1d/dc0/f72 6221 1 2024-04-24T22:45:34.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/565: read d3/f47 [1293205,25337] 0 2024-04-24T22:45:34.833 INFO:tasks.workunit.client.1.smithi192.stdout:4/463: fdatasync d2/db/d75/da1/d36/d5b/d28/d2f/f3c 0 2024-04-24T22:45:34.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/650: symlink d3/dd/d53/d60/le5 0 2024-04-24T22:45:34.841 INFO:tasks.workunit.client.1.smithi192.stdout:2/648: symlink d4/d9/d10/ld1 0 2024-04-24T22:45:34.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/649: write d4/d29/f2b [2031831,70291] 0 2024-04-24T22:45:34.844 INFO:tasks.workunit.client.1.smithi192.stdout:0/566: rmdir d3/d27/d28/d2e/d4a/d5c/d80/dac 39 2024-04-24T22:45:34.845 INFO:tasks.workunit.client.1.smithi192.stdout:0/567: fsync d3/d27/d3c/d64/faa 0 2024-04-24T22:45:34.849 INFO:tasks.workunit.client.1.smithi192.stdout:9/651: symlink d3/dd/de/dba/le6 0 2024-04-24T22:45:34.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/464: dread d2/db/f7a [0,4194304] 0 2024-04-24T22:45:34.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/650: symlink d4/d9/d10/d6c/d8c/da5/ld2 0 2024-04-24T22:45:34.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/651: readlink d4/d9/d37/l3a 0 2024-04-24T22:45:34.863 INFO:tasks.workunit.client.1.smithi192.stdout:6/479: link d0/d8/d29/l3c d0/d8/d29/d4c/d4d/d6a/la6 0 2024-04-24T22:45:34.868 INFO:tasks.workunit.client.1.smithi192.stdout:9/652: dwrite d3/dd/f6c [4194304,4194304] 0 2024-04-24T22:45:34.870 INFO:tasks.workunit.client.1.smithi192.stdout:2/652: symlink d4/d9/d10/ld3 0 2024-04-24T22:45:34.870 INFO:tasks.workunit.client.1.smithi192.stdout:6/480: write d0/d8/d18/d2e/f8d [237749,19717] 0 2024-04-24T22:45:34.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/554: dwrite db/dc/f18 [0,4194304] 0 2024-04-24T22:45:34.877 INFO:tasks.workunit.client.1.smithi192.stdout:7/593: write d2/db/d21/d42/d67/d9b/d6e/db5/db6/f8d [369041,7347] 0 2024-04-24T22:45:34.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/653: creat d3/dd/de/d17/d89/de1/fe7 x:0 0 0 2024-04-24T22:45:34.887 INFO:tasks.workunit.client.1.smithi192.stdout:7/594: unlink d2/f17 0 2024-04-24T22:45:34.889 INFO:tasks.workunit.client.1.smithi192.stdout:9/654: chown d3/d5d/fdc 395649972 1 2024-04-24T22:45:34.890 INFO:tasks.workunit.client.1.smithi192.stdout:9/655: stat d3/d5d/fdc 0 2024-04-24T22:45:34.891 INFO:tasks.workunit.client.1.smithi192.stdout:2/653: dwrite d4/d9/d37/f9c [4194304,4194304] 0 2024-04-24T22:45:34.892 INFO:tasks.workunit.client.1.smithi192.stdout:8/555: dwrite db/d43/d76/d92/f9b [0,4194304] 0 2024-04-24T22:45:34.894 INFO:tasks.workunit.client.1.smithi192.stdout:2/654: write d4/d9/f52 [1402254,97377] 0 2024-04-24T22:45:34.904 INFO:tasks.workunit.client.0.smithi063.stdout:9/707: rename d6/d2f/dcd/db/d6b/f8f to d6/d2f/dcd/fe8 0 2024-04-24T22:45:34.907 INFO:tasks.workunit.client.0.smithi063.stdout:1/757: rename d3/da/da2/dba/d39/dbf/lc8 to d3/da/da2/de6/lfa 0 2024-04-24T22:45:34.914 INFO:tasks.workunit.client.1.smithi192.stdout:9/656: dread d3/dd/de/dba/f2e [0,4194304] 0 2024-04-24T22:45:34.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/665: rename d2/d8/d36/da3 to d2/d8/dd4/dda 0 2024-04-24T22:45:34.922 INFO:tasks.workunit.client.0.smithi063.stdout:0/633: truncate d6/d64/d21/f86 2295458 0 2024-04-24T22:45:34.923 INFO:tasks.workunit.client.0.smithi063.stdout:1/758: rename d3/da/l85 to d3/da/da2/dba/d11/lfb 0 2024-04-24T22:45:34.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/695: symlink da/d39/lf2 0 2024-04-24T22:45:34.928 INFO:tasks.workunit.client.0.smithi063.stdout:0/634: dwrite d6/d64/d24/d42/fbe [0,4194304] 0 2024-04-24T22:45:34.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/696: chown da/d18/f30 3 1 2024-04-24T22:45:34.930 INFO:tasks.workunit.client.1.smithi192.stdout:6/481: dread - d0/d8/d3d/d3e/d40/d44/f73 zero size 2024-04-24T22:45:34.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/595: chown d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 651 1 2024-04-24T22:45:34.932 INFO:tasks.workunit.client.0.smithi063.stdout:3/697: mkdir da/d39/d94/db3/df3 0 2024-04-24T22:45:34.933 INFO:tasks.workunit.client.1.smithi192.stdout:7/596: chown d2/db/d21/d42/d67/l6a 6363 1 2024-04-24T22:45:34.938 INFO:tasks.workunit.client.1.smithi192.stdout:8/556: creat db/dc/d22/d31/d33/d81/d8a/faa x:0 0 0 2024-04-24T22:45:34.939 INFO:tasks.workunit.client.1.smithi192.stdout:9/657: symlink d3/d59/d7d/le8 0 2024-04-24T22:45:34.942 INFO:tasks.workunit.client.1.smithi192.stdout:2/655: mkdir d4/d9/dd4 0 2024-04-24T22:45:34.943 INFO:tasks.workunit.client.1.smithi192.stdout:6/482: mknod d0/d8/d29/ca7 0 2024-04-24T22:45:34.944 INFO:tasks.workunit.client.1.smithi192.stdout:6/483: read d0/d8/f31 [90119,55757] 0 2024-04-24T22:45:34.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/658: creat d3/d59/dbf/fe9 x:0 0 0 2024-04-24T22:45:34.948 INFO:tasks.workunit.client.1.smithi192.stdout:2/656: write d4/d9/d37/f94 [378564,110515] 0 2024-04-24T22:45:34.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/657: write d4/d9/d10/da7/fbc [202342,43013] 0 2024-04-24T22:45:34.951 INFO:tasks.workunit.client.0.smithi063.stdout:9/708: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/fe9 x:0 0 0 2024-04-24T22:45:34.951 INFO:tasks.workunit.client.0.smithi063.stdout:8/712: write d1/d22/d33/f57 [3737528,68103] 0 2024-04-24T22:45:34.952 INFO:tasks.workunit.client.1.smithi192.stdout:6/484: creat d0/d8/d18/d2e/fa8 x:0 0 0 2024-04-24T22:45:34.953 INFO:tasks.workunit.client.1.smithi192.stdout:7/597: creat d2/db/d21/d42/d67/d9b/d6e/fb8 x:0 0 0 2024-04-24T22:45:34.954 INFO:tasks.workunit.client.0.smithi063.stdout:7/623: dwrite d0/d4/de/d17/f8a [0,4194304] 0 2024-04-24T22:45:34.957 INFO:tasks.workunit.client.1.smithi192.stdout:8/557: link db/dc/ld db/d43/d7c/lab 0 2024-04-24T22:45:34.965 INFO:tasks.workunit.client.1.smithi192.stdout:9/659: mknod d3/d59/d7d/d9b/dd1/cea 0 2024-04-24T22:45:34.966 INFO:tasks.workunit.client.1.smithi192.stdout:7/598: creat d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb9 x:0 0 0 2024-04-24T22:45:34.967 INFO:tasks.workunit.client.0.smithi063.stdout:2/777: truncate d0/fa6 368397 0 2024-04-24T22:45:34.969 INFO:tasks.workunit.client.1.smithi192.stdout:8/558: truncate f8 1016473 0 2024-04-24T22:45:34.972 INFO:tasks.workunit.client.1.smithi192.stdout:9/660: dwrite d3/d5d/da1/d85/fb7 [0,4194304] 0 2024-04-24T22:45:34.973 INFO:tasks.workunit.client.1.smithi192.stdout:6/485: mknod d0/d8/d3d/ca9 0 2024-04-24T22:45:34.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/778: symlink d0/d2/d2d/d31/d89/d7/d86/da4/dd9/lef 0 2024-04-24T22:45:34.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/779: readlink d0/dd/l7b 0 2024-04-24T22:45:34.976 INFO:tasks.workunit.client.0.smithi063.stdout:8/713: getdents d1/d11/d32/d38/db7/dc1 0 2024-04-24T22:45:34.978 INFO:tasks.workunit.client.1.smithi192.stdout:2/658: creat d4/d9/d10/d6c/fd5 x:0 0 0 2024-04-24T22:45:34.979 INFO:tasks.workunit.client.0.smithi063.stdout:7/624: dread d0/d4/de/d17/d35/d69/f7b [0,4194304] 0 2024-04-24T22:45:34.979 INFO:tasks.workunit.client.0.smithi063.stdout:7/625: fdatasync d0/d4/de/f12 0 2024-04-24T22:45:34.981 INFO:tasks.workunit.client.0.smithi063.stdout:2/780: symlink d0/d2/d2d/d31/d89/d7/d86/da0/d94/lf0 0 2024-04-24T22:45:34.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/486: symlink d0/d8/d18/d2e/laa 0 2024-04-24T22:45:34.988 INFO:tasks.workunit.client.1.smithi192.stdout:2/659: dread d4/d29/f34 [0,4194304] 0 2024-04-24T22:45:34.988 INFO:tasks.workunit.client.1.smithi192.stdout:6/487: dread d0/d8/d22/f32 [0,4194304] 0 2024-04-24T22:45:34.988 INFO:tasks.workunit.client.0.smithi063.stdout:5/668: mkdir d5/de5 0 2024-04-24T22:45:34.989 INFO:tasks.workunit.client.0.smithi063.stdout:8/714: mknod d1/d22/d33/da6/dae/de7/ce9 0 2024-04-24T22:45:34.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/669: write d5/d11/d23/d2d/d62/d81/da4/fd4 [298751,51050] 0 2024-04-24T22:45:34.991 INFO:tasks.workunit.client.1.smithi192.stdout:9/661: fdatasync d3/dd/de/d17/d89/d8b/f3e 0 2024-04-24T22:45:34.997 INFO:tasks.workunit.client.1.smithi192.stdout:6/488: creat d0/d8/d29/d58/fab x:0 0 0 2024-04-24T22:45:34.998 INFO:tasks.workunit.client.1.smithi192.stdout:6/489: write d0/d8/d3d/d3e/d7e/f8e [928636,17424] 0 2024-04-24T22:45:34.998 INFO:tasks.workunit.client.0.smithi063.stdout:5/670: creat d5/d11/d23/d2d/d62/fe6 x:0 0 0 2024-04-24T22:45:35.002 INFO:tasks.workunit.client.0.smithi063.stdout:8/715: mknod d1/d21/de6/cea 0 2024-04-24T22:45:35.003 INFO:tasks.workunit.client.1.smithi192.stdout:2/660: getdents d4/d9/dd4 0 2024-04-24T22:45:35.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/571: write d2/d27/d28/f36 [120409,10571] 0 2024-04-24T22:45:35.005 INFO:tasks.workunit.client.0.smithi063.stdout:5/671: dread f1 [4194304,4194304] 0 2024-04-24T22:45:35.007 INFO:tasks.workunit.client.1.smithi192.stdout:2/661: unlink d4/d9/d10/d6c/fd5 0 2024-04-24T22:45:35.008 INFO:tasks.workunit.client.1.smithi192.stdout:3/572: mknod d2/d4/d7/d4f/caf 0 2024-04-24T22:45:35.009 INFO:tasks.workunit.client.1.smithi192.stdout:2/662: chown d4/d9/d10/d6c/d6f/da8/fb4 121 1 2024-04-24T22:45:35.009 INFO:tasks.workunit.client.1.smithi192.stdout:3/573: fdatasync d2/d4/d7/d4f/f8c 0 2024-04-24T22:45:35.010 INFO:tasks.workunit.client.0.smithi063.stdout:8/716: mknod d1/d22/d33/da6/dae/ceb 0 2024-04-24T22:45:35.010 INFO:tasks.workunit.client.0.smithi063.stdout:8/717: write d1/d11/f1b [2316834,76529] 0 2024-04-24T22:45:35.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/662: rmdir d3/d59/d7d/dc6 0 2024-04-24T22:45:35.012 INFO:tasks.workunit.client.1.smithi192.stdout:3/574: dread d2/d27/d28/f36 [0,4194304] 0 2024-04-24T22:45:35.015 INFO:tasks.workunit.client.1.smithi192.stdout:3/575: truncate d2/d4/d12/d39/d97/f9e 450812 0 2024-04-24T22:45:35.022 INFO:tasks.workunit.client.1.smithi192.stdout:5/440: write d0/f23 [4936589,102813] 0 2024-04-24T22:45:35.022 INFO:tasks.workunit.client.1.smithi192.stdout:3/576: dwrite d2/d4/d7/d4b/f8d [0,4194304] 0 2024-04-24T22:45:35.025 INFO:tasks.workunit.client.1.smithi192.stdout:6/490: getdents d0/d8 0 2024-04-24T22:45:35.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/718: dread d1/d11/d32/d38/fa0 [4194304,4194304] 0 2024-04-24T22:45:35.026 INFO:tasks.workunit.client.1.smithi192.stdout:6/491: stat d0/d8/d29/f95 0 2024-04-24T22:45:35.033 INFO:tasks.workunit.client.1.smithi192.stdout:3/577: dwrite d2/d4/d4e/f8f [0,4194304] 0 2024-04-24T22:45:35.035 INFO:tasks.workunit.client.1.smithi192.stdout:3/578: truncate d2/d4/d7/d4f/f8c 704125 0 2024-04-24T22:45:35.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/635: rename d6/d64/d5d to d6/d64/d21/d80/dde 0 2024-04-24T22:45:35.038 INFO:tasks.workunit.client.0.smithi063.stdout:8/719: creat d1/fec x:0 0 0 2024-04-24T22:45:35.039 INFO:tasks.workunit.client.0.smithi063.stdout:1/759: sync 2024-04-24T22:45:35.039 INFO:tasks.workunit.client.0.smithi063.stdout:1/760: chown d3/da 358989044 1 2024-04-24T22:45:35.040 INFO:tasks.workunit.client.1.smithi192.stdout:4/465: write d2/db/d5d/d77/f92 [103082,20260] 0 2024-04-24T22:45:35.041 INFO:tasks.workunit.client.1.smithi192.stdout:5/441: getdents d0/d13/d3b/d2f 0 2024-04-24T22:45:35.043 INFO:tasks.workunit.client.1.smithi192.stdout:3/579: dread d2/d4/d7/f60 [0,4194304] 0 2024-04-24T22:45:35.044 INFO:tasks.workunit.client.0.smithi063.stdout:3/698: rename da/d18/d47/d4d/d9b/fe6 to da/d39/d94/db3/dbe/ff4 0 2024-04-24T22:45:35.044 INFO:tasks.workunit.client.0.smithi063.stdout:0/636: rename d6/d3a/dba to d6/d3a/dba/ddf 22 2024-04-24T22:45:35.045 INFO:tasks.workunit.client.0.smithi063.stdout:6/712: rmdir db/d1d 39 2024-04-24T22:45:35.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/637: read d6/d64/d21/d25/f55 [32541,97043] 0 2024-04-24T22:45:35.048 INFO:tasks.workunit.client.1.smithi192.stdout:0/568: dwrite d3/d27/d28/f56 [0,4194304] 0 2024-04-24T22:45:35.050 INFO:tasks.workunit.client.1.smithi192.stdout:6/492: creat d0/d8/d3d/d3e/fac x:0 0 0 2024-04-24T22:45:35.051 INFO:tasks.workunit.client.1.smithi192.stdout:6/493: dread - d0/d8/d29/d4c/f6d zero size 2024-04-24T22:45:35.051 INFO:tasks.workunit.client.1.smithi192.stdout:5/442: mknod d0/d13/d19/d71/d7f/c97 0 2024-04-24T22:45:35.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/443: dread - d0/f8b zero size 2024-04-24T22:45:35.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/466: creat d2/db/d75/da1/d36/d5b/d28/d2f/fa2 x:0 0 0 2024-04-24T22:45:35.061 INFO:tasks.workunit.client.1.smithi192.stdout:0/569: readlink d3/d27/d3c/d6c/l7e 0 2024-04-24T22:45:35.068 INFO:tasks.workunit.client.0.smithi063.stdout:0/638: creat d6/d3a/dba/fe0 x:0 0 0 2024-04-24T22:45:35.073 INFO:tasks.workunit.client.1.smithi192.stdout:4/467: dread d2/db/d10/f4f [0,4194304] 0 2024-04-24T22:45:35.073 INFO:tasks.workunit.client.0.smithi063.stdout:1/761: link d3/da/d34/d6d/c8e d3/da/d34/d99/cfc 0 2024-04-24T22:45:35.074 INFO:tasks.workunit.client.1.smithi192.stdout:6/494: creat d0/d8/d3d/d3e/d7e/d9b/fad x:0 0 0 2024-04-24T22:45:35.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/471: rename d9/c11 to d9/d15/d2d/d63/d68/ca0 0 2024-04-24T22:45:35.077 INFO:tasks.workunit.client.0.smithi063.stdout:6/713: mknod db/d1d/d20/d3f/cef 0 2024-04-24T22:45:35.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/714: dread - db/d1d/d20/d3f/d5f/fd8 zero size 2024-04-24T22:45:35.079 INFO:tasks.workunit.client.0.smithi063.stdout:7/626: sync 2024-04-24T22:45:35.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/639: unlink d6/da/d2b/l4a 0 2024-04-24T22:45:35.082 INFO:tasks.workunit.client.1.smithi192.stdout:6/495: dwrite d0/d8/d29/d58/fab [0,4194304] 0 2024-04-24T22:45:35.084 INFO:tasks.workunit.client.0.smithi063.stdout:1/762: creat d3/da/d34/d6d/ffd x:0 0 0 2024-04-24T22:45:35.087 INFO:tasks.workunit.client.0.smithi063.stdout:7/627: rename d0/d4/de/f10 to d0/d4/de/d17/d35/d69/db1/fd3 0 2024-04-24T22:45:35.088 INFO:tasks.workunit.client.0.smithi063.stdout:6/715: dread db/fb7 [0,4194304] 0 2024-04-24T22:45:35.089 INFO:tasks.workunit.client.0.smithi063.stdout:7/628: chown d0/d4/de/d6f/lba 0 1 2024-04-24T22:45:35.089 INFO:tasks.workunit.client.1.smithi192.stdout:7/599: rename d2/db/d21/d42/l7f to d2/db/d2a/d57/da9/lba 0 2024-04-24T22:45:35.090 INFO:tasks.workunit.client.0.smithi063.stdout:1/763: mknod d3/da/da2/dba/d39/dbf/cfe 0 2024-04-24T22:45:35.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/666: write d2/d8/f1a [1698711,36329] 0 2024-04-24T22:45:35.093 INFO:tasks.workunit.client.0.smithi063.stdout:4/667: write d2/d80/fb0 [188907,89839] 0 2024-04-24T22:45:35.093 INFO:tasks.workunit.client.0.smithi063.stdout:0/640: dread d6/d64/d24/f72 [0,4194304] 0 2024-04-24T22:45:35.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/668: write d2/d8/f71 [355655,15384] 0 2024-04-24T22:45:35.096 INFO:tasks.workunit.client.0.smithi063.stdout:0/641: truncate d6/da/d2b/f40 3077864 0 2024-04-24T22:45:35.098 INFO:tasks.workunit.client.0.smithi063.stdout:6/716: symlink db/d1d/d20/d3f/d7f/d68/d70/lf0 0 2024-04-24T22:45:35.100 INFO:tasks.workunit.client.0.smithi063.stdout:7/629: unlink d0/d4/de/d17/c36 0 2024-04-24T22:45:35.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/496: creat d0/d8/d3d/d3e/d7e/fae x:0 0 0 2024-04-24T22:45:35.108 INFO:tasks.workunit.client.1.smithi192.stdout:2/663: rename d4/d9/d10/d6c/d6f/da8/fb4 to d4/d29/d60/d9a/db3/fd6 0 2024-04-24T22:45:35.112 INFO:tasks.workunit.client.0.smithi063.stdout:6/717: rmdir db/d1c 39 2024-04-24T22:45:35.113 INFO:tasks.workunit.client.1.smithi192.stdout:1/472: getdents d9/d15/d2d/d63/d68/d9d 0 2024-04-24T22:45:35.114 INFO:tasks.workunit.client.1.smithi192.stdout:1/473: write d9/d6f/f86 [432846,99007] 0 2024-04-24T22:45:35.115 INFO:tasks.workunit.client.1.smithi192.stdout:1/474: chown d9/d51/d70/f7f 1710959929 1 2024-04-24T22:45:35.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/630: mknod d0/d4/de/d17/d35/d69/cd4 0 2024-04-24T22:45:35.115 INFO:tasks.workunit.client.1.smithi192.stdout:1/475: stat d9/d51 0 2024-04-24T22:45:35.116 INFO:tasks.workunit.client.1.smithi192.stdout:6/497: creat d0/d8/d3d/d3e/d7e/faf x:0 0 0 2024-04-24T22:45:35.122 INFO:tasks.workunit.client.1.smithi192.stdout:2/664: creat d4/d9/d9f/fd7 x:0 0 0 2024-04-24T22:45:35.123 INFO:tasks.workunit.client.0.smithi063.stdout:7/631: write d0/d4/de/f25 [1815594,18261] 0 2024-04-24T22:45:35.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/498: dwrite d0/d8/d3d/d3e/d7e/f8e [0,4194304] 0 2024-04-24T22:45:35.124 INFO:tasks.workunit.client.0.smithi063.stdout:7/632: chown d0/d4/de/d17/d35/c71 17 1 2024-04-24T22:45:35.125 INFO:tasks.workunit.client.1.smithi192.stdout:6/499: stat d0/d8/d22/c23 0 2024-04-24T22:45:35.125 INFO:tasks.workunit.client.0.smithi063.stdout:6/718: dwrite db/d1d/d20/d3e/f43 [0,4194304] 0 2024-04-24T22:45:35.128 INFO:tasks.workunit.client.0.smithi063.stdout:7/633: truncate d0/d4/de/d17/d35/d69/db1/fc1 40069 0 2024-04-24T22:45:35.129 INFO:tasks.workunit.client.0.smithi063.stdout:7/634: stat d0/d4/de/d6f/lba 0 2024-04-24T22:45:35.130 INFO:tasks.workunit.client.0.smithi063.stdout:7/635: fdatasync d0/d4/de/d17/d28/fb8 0 2024-04-24T22:45:35.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/709: dwrite d6/d2f/dcd/db/d6b/f23 [0,4194304] 0 2024-04-24T22:45:35.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/500: dread - d0/d8/d3d/f62 zero size 2024-04-24T22:45:35.138 INFO:tasks.workunit.client.1.smithi192.stdout:1/476: mknod d9/dd/d74/d83/ca1 0 2024-04-24T22:45:35.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/719: symlink db/d1c/dc3/lf1 0 2024-04-24T22:45:35.141 INFO:tasks.workunit.client.1.smithi192.stdout:8/559: write db/dc/d29/f4f [4261512,109068] 0 2024-04-24T22:45:35.145 INFO:tasks.workunit.client.1.smithi192.stdout:6/501: mkdir d0/d8/d29/d4c/db0 0 2024-04-24T22:45:35.146 INFO:tasks.workunit.client.1.smithi192.stdout:8/560: dread db/dc/d22/d31/f4d [0,4194304] 0 2024-04-24T22:45:35.147 INFO:tasks.workunit.client.1.smithi192.stdout:1/477: stat d9/d15/d2d/d4a/l69 0 2024-04-24T22:45:35.150 INFO:tasks.workunit.client.0.smithi063.stdout:2/781: write d0/d2/d2d/d31/fab [1282749,43247] 0 2024-04-24T22:45:35.154 INFO:tasks.workunit.client.1.smithi192.stdout:6/502: creat d0/d8/fb1 x:0 0 0 2024-04-24T22:45:35.159 INFO:tasks.workunit.client.1.smithi192.stdout:1/478: fdatasync d9/d15/d2d/d48/d93/f98 0 2024-04-24T22:45:35.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/665: getdents d4/d29 0 2024-04-24T22:45:35.164 INFO:tasks.workunit.client.0.smithi063.stdout:9/710: getdents d6/d2f/dcd/db/d93/da0 0 2024-04-24T22:45:35.168 INFO:tasks.workunit.client.1.smithi192.stdout:6/503: creat d0/d8/d3d/d9d/fb2 x:0 0 0 2024-04-24T22:45:35.168 INFO:tasks.workunit.client.0.smithi063.stdout:5/672: dwrite d5/d11/d23/d2d/d62/d81/da4/da6/fc6 [0,4194304] 0 2024-04-24T22:45:35.168 INFO:tasks.workunit.client.1.smithi192.stdout:6/504: chown d0/d8/d3d/d9d 919062347 1 2024-04-24T22:45:35.169 INFO:tasks.workunit.client.1.smithi192.stdout:6/505: fdatasync d0/d8/d3d/d3e/d7e/fae 0 2024-04-24T22:45:35.169 INFO:tasks.workunit.client.0.smithi063.stdout:6/720: dread db/d1d/d20/fa1 [0,4194304] 0 2024-04-24T22:45:35.170 INFO:tasks.workunit.client.1.smithi192.stdout:9/663: dread d3/f71 [0,4194304] 0 2024-04-24T22:45:35.172 INFO:tasks.workunit.client.0.smithi063.stdout:9/711: dwrite d6/f7 [0,4194304] 0 2024-04-24T22:45:35.172 INFO:tasks.workunit.client.0.smithi063.stdout:6/721: symlink db/d1d/d20/d84/lf2 0 2024-04-24T22:45:35.177 INFO:tasks.workunit.client.0.smithi063.stdout:5/673: rename d5/d11/d23/d2d/d62/cca to d5/d24/db6/ce7 0 2024-04-24T22:45:35.178 INFO:tasks.workunit.client.0.smithi063.stdout:5/674: chown d5/d24/d36/f5d 592 1 2024-04-24T22:45:35.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/722: mknod db/d1c/cf3 0 2024-04-24T22:45:35.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/723: read db/d1d/fc9 [2624601,16606] 0 2024-04-24T22:45:35.183 INFO:tasks.workunit.client.0.smithi063.stdout:3/699: write da/d48/f95 [1347783,3903] 0 2024-04-24T22:45:35.186 INFO:tasks.workunit.client.0.smithi063.stdout:8/720: dwrite d1/d21/f4f [0,4194304] 0 2024-04-24T22:45:35.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/642: rmdir d6/d3a/dba 39 2024-04-24T22:45:35.187 INFO:tasks.workunit.client.0.smithi063.stdout:8/721: readlink d1/d2/d17/l97 0 2024-04-24T22:45:35.188 INFO:tasks.workunit.client.0.smithi063.stdout:0/643: dread - d6/da/d2b/fa7 zero size 2024-04-24T22:45:35.189 INFO:tasks.workunit.client.0.smithi063.stdout:0/644: stat d6/d64/da0/d33/fb8 0 2024-04-24T22:45:35.189 INFO:tasks.workunit.client.0.smithi063.stdout:3/700: getdents da/d18/d47/d4d/d9b 0 2024-04-24T22:45:35.190 INFO:tasks.workunit.client.0.smithi063.stdout:8/722: creat d1/d11/d32/d38/fed x:0 0 0 2024-04-24T22:45:35.192 INFO:tasks.workunit.client.0.smithi063.stdout:3/701: fsync da/d18/d47/d7b/f91 0 2024-04-24T22:45:35.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/702: truncate da/fe2 878927 0 2024-04-24T22:45:35.194 INFO:tasks.workunit.client.0.smithi063.stdout:3/703: write da/d39/d94/db3/fdf [2559866,42752] 0 2024-04-24T22:45:35.195 INFO:tasks.workunit.client.0.smithi063.stdout:0/645: mknod d6/d64/d24/d42/dd2/ce1 0 2024-04-24T22:45:35.198 INFO:tasks.workunit.client.0.smithi063.stdout:3/704: rename da/f60 to da/d18/d9e/da3/dc1/ff5 0 2024-04-24T22:45:35.202 INFO:tasks.workunit.client.0.smithi063.stdout:3/705: creat da/d18/d9e/da3/dc1/ff6 x:0 0 0 2024-04-24T22:45:35.204 INFO:tasks.workunit.client.0.smithi063.stdout:3/706: fdatasync da/fa0 0 2024-04-24T22:45:35.208 INFO:tasks.workunit.client.0.smithi063.stdout:3/707: mkdir da/d39/d94/db3/dbe/df7 0 2024-04-24T22:45:35.214 INFO:tasks.workunit.client.0.smithi063.stdout:1/764: dwrite d3/de0/d79/fc1 [0,4194304] 0 2024-04-24T22:45:35.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/723: dread d1/d22/d33/da6/d54/fad [0,4194304] 0 2024-04-24T22:45:35.216 INFO:tasks.workunit.client.0.smithi063.stdout:4/669: dwrite d2/d8/d40/d51/f6c [0,4194304] 0 2024-04-24T22:45:35.218 INFO:tasks.workunit.client.0.smithi063.stdout:1/765: unlink d3/db5/d21/l25 0 2024-04-24T22:45:35.219 INFO:tasks.workunit.client.0.smithi063.stdout:8/724: creat d1/d22/d33/fee x:0 0 0 2024-04-24T22:45:35.220 INFO:tasks.workunit.client.0.smithi063.stdout:1/766: fdatasync d3/db5/d21/d49/fd2 0 2024-04-24T22:45:35.226 INFO:tasks.workunit.client.0.smithi063.stdout:1/767: write d3/da/da2/dba/d11/f35 [2853687,130367] 0 2024-04-24T22:45:35.227 INFO:tasks.workunit.client.0.smithi063.stdout:7/636: dwrite d0/d4/de/d17/d35/d69/f7b [0,4194304] 0 2024-04-24T22:45:35.229 INFO:tasks.workunit.client.1.smithi192.stdout:1/479: creat d9/d15/d5e/d67/fa2 x:0 0 0 2024-04-24T22:45:35.229 INFO:tasks.workunit.client.0.smithi063.stdout:2/782: dwrite d0/d2/d27/f34 [4194304,4194304] 0 2024-04-24T22:45:35.231 INFO:tasks.workunit.client.1.smithi192.stdout:9/664: write d3/dd/de/dba/f81 [3711237,78946] 0 2024-04-24T22:45:35.232 INFO:tasks.workunit.client.0.smithi063.stdout:1/768: mkdir d3/da/da2/dba/d39/d5a/dff 0 2024-04-24T22:45:35.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/444: write d0/d13/d19/d64/d4f/d52/f67 [1062696,130118] 0 2024-04-24T22:45:35.233 INFO:tasks.workunit.client.0.smithi063.stdout:7/637: creat d0/d4/de/d17/d35/fd5 x:0 0 0 2024-04-24T22:45:35.234 INFO:tasks.workunit.client.1.smithi192.stdout:4/468: write d2/d9/d25/d2e/f66 [103769,80280] 0 2024-04-24T22:45:35.240 INFO:tasks.workunit.client.1.smithi192.stdout:8/561: write db/dc/d22/f35 [432613,100191] 0 2024-04-24T22:45:35.241 INFO:tasks.workunit.client.1.smithi192.stdout:2/666: link d4/d9/d10/da7/fca d4/d9/d10/d6c/d95/dc0/fd8 0 2024-04-24T22:45:35.242 INFO:tasks.workunit.client.1.smithi192.stdout:6/506: dread d0/d8/d3d/d3e/d7e/f8e [0,4194304] 0 2024-04-24T22:45:35.247 INFO:tasks.workunit.client.1.smithi192.stdout:3/580: dwrite d2/d4/d12/d1a/d1d/f3f [0,4194304] 0 2024-04-24T22:45:35.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/600: dwrite d2/db/d21/d42/d5b/f75 [0,4194304] 0 2024-04-24T22:45:35.250 INFO:tasks.workunit.client.0.smithi063.stdout:9/712: write d6/d2f/dcd/db/d6b/f46 [1918492,50252] 0 2024-04-24T22:45:35.252 INFO:tasks.workunit.client.0.smithi063.stdout:9/713: write d6/d2f/dcd/d53/f99 [1153171,50623] 0 2024-04-24T22:45:35.263 INFO:tasks.workunit.client.1.smithi192.stdout:9/665: write d3/dd/de/da7/dae/fd6 [320493,68399] 0 2024-04-24T22:45:35.263 INFO:tasks.workunit.client.0.smithi063.stdout:5/675: dwrite d5/f14 [0,4194304] 0 2024-04-24T22:45:35.267 INFO:tasks.workunit.client.1.smithi192.stdout:4/469: dread d2/f3 [4194304,4194304] 0 2024-04-24T22:45:35.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/714: mkdir d6/d2f/d42/dea 0 2024-04-24T22:45:35.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/562: creat db/dc/d22/d31/d33/d8f/fac x:0 0 0 2024-04-24T22:45:35.273 INFO:tasks.workunit.client.0.smithi063.stdout:0/646: dwrite d6/d64/d24/f73 [0,4194304] 0 2024-04-24T22:45:35.274 INFO:tasks.workunit.client.1.smithi192.stdout:2/667: creat d4/d9/d10/d6c/d95/db7/dbf/fd9 x:0 0 0 2024-04-24T22:45:35.274 INFO:tasks.workunit.client.0.smithi063.stdout:5/676: dread d5/d11/d76/f9a [0,4194304] 0 2024-04-24T22:45:35.277 INFO:tasks.workunit.client.1.smithi192.stdout:2/668: truncate d4/d9/d10/da7/fcb 763460 0 2024-04-24T22:45:35.278 INFO:tasks.workunit.client.1.smithi192.stdout:6/507: dwrite d0/d8/f2b [4194304,4194304] 0 2024-04-24T22:45:35.278 INFO:tasks.workunit.client.1.smithi192.stdout:3/581: unlink d2/d4/d7/d4f/c57 0 2024-04-24T22:45:35.278 INFO:tasks.workunit.client.0.smithi063.stdout:3/708: write da/fb4 [321070,3023] 0 2024-04-24T22:45:35.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/582: stat d2/d27/d28/c2f 0 2024-04-24T22:45:35.281 INFO:tasks.workunit.client.1.smithi192.stdout:5/445: symlink d0/l98 0 2024-04-24T22:45:35.287 INFO:tasks.workunit.client.0.smithi063.stdout:6/724: sync 2024-04-24T22:45:35.288 INFO:tasks.workunit.client.0.smithi063.stdout:6/725: readlink db/d85/db5/dc4/lcd 0 2024-04-24T22:45:35.288 INFO:tasks.workunit.client.0.smithi063.stdout:4/670: write d2/f99 [3441711,88946] 0 2024-04-24T22:45:35.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/480: truncate d9/d15/d2d/d31/d2f/f73 773566 0 2024-04-24T22:45:35.290 INFO:tasks.workunit.client.0.smithi063.stdout:6/726: stat db/d1d/d20/d3f/d7f/d68/d70/fad 0 2024-04-24T22:45:35.291 INFO:tasks.workunit.client.1.smithi192.stdout:1/481: write d9/d15/f2e [1583747,86521] 0 2024-04-24T22:45:35.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/727: dread - db/d1d/d20/d3f/d7f/d68/fe9 zero size 2024-04-24T22:45:35.292 INFO:tasks.workunit.client.0.smithi063.stdout:6/728: stat db/d1c/dea/lbe 0 2024-04-24T22:45:35.295 INFO:tasks.workunit.client.1.smithi192.stdout:4/470: creat d2/db/d5d/d77/d82/fa3 x:0 0 0 2024-04-24T22:45:35.295 INFO:tasks.workunit.client.1.smithi192.stdout:4/471: fsync d2/d5c/f86 0 2024-04-24T22:45:35.296 INFO:tasks.workunit.client.1.smithi192.stdout:4/472: readlink d2/d9/d6b/l6d 0 2024-04-24T22:45:35.297 INFO:tasks.workunit.client.1.smithi192.stdout:8/563: mkdir db/d43/d76/d92/dad 0 2024-04-24T22:45:35.299 INFO:tasks.workunit.client.0.smithi063.stdout:2/783: dwrite d0/d2/d27/f7f [0,4194304] 0 2024-04-24T22:45:35.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/583: mkdir d2/d4/d12/d39/d86/db0 0 2024-04-24T22:45:35.301 INFO:tasks.workunit.client.1.smithi192.stdout:3/584: read d2/d4/f69 [175482,16258] 0 2024-04-24T22:45:35.302 INFO:tasks.workunit.client.1.smithi192.stdout:5/446: mknod d0/d13/d53/c99 0 2024-04-24T22:45:35.302 INFO:tasks.workunit.client.1.smithi192.stdout:3/585: dread - d2/f9f zero size 2024-04-24T22:45:35.304 INFO:tasks.workunit.client.1.smithi192.stdout:7/601: mkdir d2/db/d21/d42/d67/dae/dbb 0 2024-04-24T22:45:35.306 INFO:tasks.workunit.client.1.smithi192.stdout:7/602: chown d2/db/d21/d42/d5b/l87 198 1 2024-04-24T22:45:35.309 INFO:tasks.workunit.client.0.smithi063.stdout:1/769: write d3/da/d34/d6d/fa0 [680948,120179] 0 2024-04-24T22:45:35.312 INFO:tasks.workunit.client.1.smithi192.stdout:4/473: rmdir d2/d5c 39 2024-04-24T22:45:35.314 INFO:tasks.workunit.client.1.smithi192.stdout:3/586: dwrite d2/d4/d12/d1a/d1d/d55/f24 [4194304,4194304] 0 2024-04-24T22:45:35.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/564: rmdir db/d75 39 2024-04-24T22:45:35.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/447: dwrite d0/d13/d19/d64/d5f/d2c/d3c/f5a [0,4194304] 0 2024-04-24T22:45:35.319 INFO:tasks.workunit.client.1.smithi192.stdout:5/448: fdatasync d0/d13/d19/d64/d5f/d2c/d3c/f77 0 2024-04-24T22:45:35.319 INFO:tasks.workunit.client.1.smithi192.stdout:2/669: symlink d4/d9/d10/d6c/d6f/lda 0 2024-04-24T22:45:35.326 INFO:tasks.workunit.client.1.smithi192.stdout:1/482: mknod d9/dd/ca3 0 2024-04-24T22:45:35.338 INFO:tasks.workunit.client.1.smithi192.stdout:8/565: dread db/d5d/f84 [0,4194304] 0 2024-04-24T22:45:35.338 INFO:tasks.workunit.client.1.smithi192.stdout:8/566: stat db/dc/d29 0 2024-04-24T22:45:35.339 INFO:tasks.workunit.client.1.smithi192.stdout:3/587: mkdir d2/d4/d12/d1a/d1d/d55/d5f/db1 0 2024-04-24T22:45:35.344 INFO:tasks.workunit.client.1.smithi192.stdout:7/603: dread f0 [0,4194304] 0 2024-04-24T22:45:35.346 INFO:tasks.workunit.client.0.smithi063.stdout:8/725: sync 2024-04-24T22:45:35.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/483: dread d9/f6a [0,4194304] 0 2024-04-24T22:45:35.349 INFO:tasks.workunit.client.1.smithi192.stdout:9/666: write d3/dd/de/da7/fd3 [607378,52772] 0 2024-04-24T22:45:35.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/474: chown d2/db/d75/da1/d36/f62 215 1 2024-04-24T22:45:35.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/588: rename d2/d4/d12/d1a/d1d/d71 to d2/d62/db2 0 2024-04-24T22:45:35.356 INFO:tasks.workunit.client.0.smithi063.stdout:7/638: creat d0/d4/de/d6f/fd6 x:0 0 0 2024-04-24T22:45:35.358 INFO:tasks.workunit.client.0.smithi063.stdout:7/639: chown d0/d4/de/d6f/d5c/d8b/dc0/fd0 1003827 1 2024-04-24T22:45:35.358 INFO:tasks.workunit.client.1.smithi192.stdout:4/475: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/fa2 [0,4194304] 0 2024-04-24T22:45:35.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/715: creat d6/d2f/dcd/d1f/feb x:0 0 0 2024-04-24T22:45:35.361 INFO:tasks.workunit.client.1.smithi192.stdout:9/667: mkdir d3/dd/d53/deb 0 2024-04-24T22:45:35.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/677: unlink d5/f6d 0 2024-04-24T22:45:35.368 INFO:tasks.workunit.client.1.smithi192.stdout:3/589: creat d2/d4/d7/d4b/d54/fb3 x:0 0 0 2024-04-24T22:45:35.370 INFO:tasks.workunit.client.1.smithi192.stdout:7/604: mknod d2/db/cbc 0 2024-04-24T22:45:35.372 INFO:tasks.workunit.client.1.smithi192.stdout:7/605: truncate d2/db/d21/d42/d5b/f9a 496590 0 2024-04-24T22:45:35.378 INFO:tasks.workunit.client.1.smithi192.stdout:4/476: creat d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 x:0 0 0 2024-04-24T22:45:35.381 INFO:tasks.workunit.client.0.smithi063.stdout:2/784: mknod d0/d2/d35/dc6/cf1 0 2024-04-24T22:45:35.383 INFO:tasks.workunit.client.1.smithi192.stdout:8/567: creat db/fae x:0 0 0 2024-04-24T22:45:35.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/785: dread d0/dd/d72/d77/d87/f98 [0,4194304] 0 2024-04-24T22:45:35.389 INFO:tasks.workunit.client.1.smithi192.stdout:9/668: creat d3/dd/dce/fec x:0 0 0 2024-04-24T22:45:35.391 INFO:tasks.workunit.client.1.smithi192.stdout:9/669: write d3/d5d/fa9 [5027867,79405] 0 2024-04-24T22:45:35.392 INFO:tasks.workunit.client.1.smithi192.stdout:8/568: creat db/dc/d22/d31/d33/d81/faf x:0 0 0 2024-04-24T22:45:35.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/670: truncate d4/f78 3714093 0 2024-04-24T22:45:35.393 INFO:tasks.workunit.client.1.smithi192.stdout:7/606: dread d2/db/f45 [0,4194304] 0 2024-04-24T22:45:35.393 INFO:tasks.workunit.client.1.smithi192.stdout:9/670: write d3/dd/de/dba/f96 [2063249,29867] 0 2024-04-24T22:45:35.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/477: mkdir d2/da5 0 2024-04-24T22:45:35.397 INFO:tasks.workunit.client.1.smithi192.stdout:4/478: fdatasync d2/db/d75/da1/d7d/f9b 0 2024-04-24T22:45:35.403 INFO:tasks.workunit.client.0.smithi063.stdout:3/709: dwrite da/d18/d3a/f96 [0,4194304] 0 2024-04-24T22:45:35.405 INFO:tasks.workunit.client.1.smithi192.stdout:3/590: write d2/d4/d7/f60 [1259725,128987] 0 2024-04-24T22:45:35.408 INFO:tasks.workunit.client.0.smithi063.stdout:6/729: dwrite db/d1d/d20/d84/fae [0,4194304] 0 2024-04-24T22:45:35.408 INFO:tasks.workunit.client.1.smithi192.stdout:7/607: creat d2/db/d2a/d57/da9/fbd x:0 0 0 2024-04-24T22:45:35.411 INFO:tasks.workunit.client.0.smithi063.stdout:7/640: rename d0/d4/de/d6f/d5c/d63 to d0/d4/de/d17/dd7 0 2024-04-24T22:45:35.413 INFO:tasks.workunit.client.0.smithi063.stdout:0/647: dwrite d6/d64/d21/d80/f9e [0,4194304] 0 2024-04-24T22:45:35.413 INFO:tasks.workunit.client.1.smithi192.stdout:4/479: fsync d2/d9/d25/d2e/f66 0 2024-04-24T22:45:35.414 INFO:tasks.workunit.client.1.smithi192.stdout:4/480: chown d2/d9/l2a 1040 1 2024-04-24T22:45:35.417 INFO:tasks.workunit.client.1.smithi192.stdout:4/481: truncate d2/db/d5d/d77/d82/fa3 927036 0 2024-04-24T22:45:35.418 INFO:tasks.workunit.client.1.smithi192.stdout:8/569: write db/dc/f13 [3930321,117540] 0 2024-04-24T22:45:35.420 INFO:tasks.workunit.client.0.smithi063.stdout:2/786: symlink d0/d2/lf2 0 2024-04-24T22:45:35.421 INFO:tasks.workunit.client.0.smithi063.stdout:8/726: sync 2024-04-24T22:45:35.422 INFO:tasks.workunit.client.0.smithi063.stdout:8/727: chown d1/d2/d3/f56 35 1 2024-04-24T22:45:35.422 INFO:tasks.workunit.client.1.smithi192.stdout:3/591: chown d2/d4/d7/f17 12473 1 2024-04-24T22:45:35.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/728: truncate d1/d11/d32/dd3/fe8 79042 0 2024-04-24T22:45:35.423 INFO:tasks.workunit.client.0.smithi063.stdout:5/678: symlink d5/d1d/da3/le8 0 2024-04-24T22:45:35.427 INFO:tasks.workunit.client.0.smithi063.stdout:4/671: getdents d2/d8/d40/da0 0 2024-04-24T22:45:35.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/770: link d3/da/da2/dba/d11/d5e/fa6 d3/da/da2/dba/f100 0 2024-04-24T22:45:35.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/729: dread d1/d11/d3c/f9e [0,4194304] 0 2024-04-24T22:45:35.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/671: rename d4/d9/d37/f8a to d4/d9/d10/d6c/fdb 0 2024-04-24T22:45:35.435 INFO:tasks.workunit.client.1.smithi192.stdout:2/672: chown d4/d9/d37/f3b 0 1 2024-04-24T22:45:35.437 INFO:tasks.workunit.client.0.smithi063.stdout:9/716: rename d6/d2f/dcd/db/fc1 to d6/dbd/dcf/fec 0 2024-04-24T22:45:35.437 INFO:tasks.workunit.client.1.smithi192.stdout:4/482: dwrite d2/db/f7a [0,4194304] 0 2024-04-24T22:45:35.437 INFO:tasks.workunit.client.1.smithi192.stdout:2/673: fdatasync d4/d9/d10/d6c/d95/db7/dbf/fd9 0 2024-04-24T22:45:35.506 INFO:tasks.workunit.client.0.smithi063.stdout:4/672: sync 2024-04-24T22:45:35.551 INFO:tasks.workunit.client.0.smithi063.stdout:0/648: unlink d6/d74/la2 0 2024-04-24T22:45:35.553 INFO:tasks.workunit.client.1.smithi192.stdout:7/608: rename d2/db to d2/db/dbe 22 2024-04-24T22:45:35.554 INFO:tasks.workunit.client.1.smithi192.stdout:7/609: chown d2/db/f10 21 1 2024-04-24T22:45:35.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/679: fsync d5/d24/d36/f5a 0 2024-04-24T22:45:35.557 INFO:tasks.workunit.client.1.smithi192.stdout:4/483: dread - d2/d5c/f99 zero size 2024-04-24T22:45:35.558 INFO:tasks.workunit.client.1.smithi192.stdout:2/674: rmdir d4/d9/d37 39 2024-04-24T22:45:35.559 INFO:tasks.workunit.client.0.smithi063.stdout:7/641: dwrite d0/d4/de/d17/d28/f64 [4194304,4194304] 0 2024-04-24T22:45:35.559 INFO:tasks.workunit.client.0.smithi063.stdout:0/649: dwrite d6/d64/d21/d80/f9e [0,4194304] 0 2024-04-24T22:45:35.560 INFO:tasks.workunit.client.1.smithi192.stdout:9/671: getdents d3/d5d/da1/d5e/d98 0 2024-04-24T22:45:35.563 INFO:tasks.workunit.client.1.smithi192.stdout:3/592: link d2/d4/d12/f42 d2/d4/d12/fb4 0 2024-04-24T22:45:35.565 INFO:tasks.workunit.client.0.smithi063.stdout:3/710: rename da/d18/d47/d4d/dc2/cd8 to da/d39/d94/cf8 0 2024-04-24T22:45:35.565 INFO:tasks.workunit.client.0.smithi063.stdout:1/771: rename d3/da/da2/dba to d3/da/da2/dba/d39/d101 22 2024-04-24T22:45:35.566 INFO:tasks.workunit.client.0.smithi063.stdout:0/650: dwrite d6/d3a/f7a [0,4194304] 0 2024-04-24T22:45:35.567 INFO:tasks.workunit.client.0.smithi063.stdout:6/730: write db/d1d/d2d/d48/f6e [1301469,24172] 0 2024-04-24T22:45:35.567 INFO:tasks.workunit.client.1.smithi192.stdout:7/610: creat d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fbf x:0 0 0 2024-04-24T22:45:35.569 INFO:tasks.workunit.client.0.smithi063.stdout:9/717: symlink d6/d2f/dcd/d53/dbe/led 0 2024-04-24T22:45:35.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/673: unlink d2/d8/d36/fba 0 2024-04-24T22:45:35.576 INFO:tasks.workunit.client.1.smithi192.stdout:9/672: mkdir d3/dd/de/d17/d89/d8b/ded 0 2024-04-24T22:45:35.578 INFO:tasks.workunit.client.1.smithi192.stdout:3/593: symlink d2/d4/d12/d1a/lb5 0 2024-04-24T22:45:35.579 INFO:tasks.workunit.client.1.smithi192.stdout:4/484: mknod d2/db/d75/da1/d7d/d9c/ca6 0 2024-04-24T22:45:35.580 INFO:tasks.workunit.client.0.smithi063.stdout:8/730: getdents d1/d2/ddf 0 2024-04-24T22:45:35.581 INFO:tasks.workunit.client.1.smithi192.stdout:7/611: dwrite d2/db/d2a/d57/da9/fbd [0,4194304] 0 2024-04-24T22:45:35.581 INFO:tasks.workunit.client.1.smithi192.stdout:2/675: fsync d4/d9/f2c 0 2024-04-24T22:45:35.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/731: readlink d1/d2/d17/l97 0 2024-04-24T22:45:35.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/732: chown d1/d11/f63 2434 1 2024-04-24T22:45:35.584 INFO:tasks.workunit.client.0.smithi063.stdout:5/680: mknod d5/d11/d23/d2d/d62/d81/da4/da6/ce9 0 2024-04-24T22:45:35.589 INFO:tasks.workunit.client.1.smithi192.stdout:2/676: creat d4/d29/fdc x:0 0 0 2024-04-24T22:45:35.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/612: truncate d2/db/d25/f96 84444 0 2024-04-24T22:45:35.592 INFO:tasks.workunit.client.0.smithi063.stdout:2/787: dread d0/d2/d2f/f6d [0,4194304] 0 2024-04-24T22:45:35.594 INFO:tasks.workunit.client.1.smithi192.stdout:9/673: symlink d3/dd/de/dba/dbc/ddb/lee 0 2024-04-24T22:45:35.595 INFO:tasks.workunit.client.1.smithi192.stdout:7/613: truncate d2/db/d21/d42/d67/d9b/d6e/db5/db6/f8d 708231 0 2024-04-24T22:45:35.595 INFO:tasks.workunit.client.0.smithi063.stdout:6/731: read db/f1a [1718478,119976] 0 2024-04-24T22:45:35.596 INFO:tasks.workunit.client.0.smithi063.stdout:0/651: mknod d6/d64/d21/d25/d62/ce2 0 2024-04-24T22:45:35.599 INFO:tasks.workunit.client.1.smithi192.stdout:2/677: symlink d4/d29/d60/d9a/ldd 0 2024-04-24T22:45:35.599 INFO:tasks.workunit.client.1.smithi192.stdout:2/678: read d4/d29/d60/f89 [3743896,62456] 0 2024-04-24T22:45:35.600 INFO:tasks.workunit.client.0.smithi063.stdout:4/674: symlink d2/d8/dd4/dda/dc6/ldb 0 2024-04-24T22:45:35.601 INFO:tasks.workunit.client.0.smithi063.stdout:6/732: dwrite db/d1d/d2d/d69/d82/d96/fb4 [0,4194304] 0 2024-04-24T22:45:35.607 INFO:tasks.workunit.client.0.smithi063.stdout:8/733: rename d1/d21/la7 to d1/d2/d17/lef 0 2024-04-24T22:45:35.614 INFO:tasks.workunit.client.0.smithi063.stdout:7/642: creat d0/d4/de/d6f/d5c/d9c/fd8 x:0 0 0 2024-04-24T22:45:35.615 INFO:tasks.workunit.client.1.smithi192.stdout:2/679: chown d4/d9/cd 146902238 1 2024-04-24T22:45:35.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/680: readlink d4/d29/l4d 0 2024-04-24T22:45:35.616 INFO:tasks.workunit.client.0.smithi063.stdout:1/772: stat d3/db5/d21/c9b 0 2024-04-24T22:45:35.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/681: stat d4/c1b 0 2024-04-24T22:45:35.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/711: mkdir da/d18/d47/d4d/d9b/dde/df9 0 2024-04-24T22:45:35.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/712: chown da/d39/d94/db3/dbe/lcd 332213 1 2024-04-24T22:45:35.620 INFO:tasks.workunit.client.1.smithi192.stdout:3/594: getdents d2/d27/d28/da5/d93 0 2024-04-24T22:45:35.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/595: write d2/d4/d7/f60 [2408326,93557] 0 2024-04-24T22:45:35.624 INFO:tasks.workunit.client.1.smithi192.stdout:4/485: dwrite d2/db/f3d [0,4194304] 0 2024-04-24T22:45:35.627 INFO:tasks.workunit.client.0.smithi063.stdout:9/718: write d6/d2f/dcd/fe8 [1546397,51068] 0 2024-04-24T22:45:35.629 INFO:tasks.workunit.client.0.smithi063.stdout:9/719: chown d6/d2f/dcd/db/d1c/l67 470 1 2024-04-24T22:45:35.631 INFO:tasks.workunit.client.0.smithi063.stdout:6/733: creat db/d1d/d20/d29/dab/dd0/ff4 x:0 0 0 2024-04-24T22:45:35.632 INFO:tasks.workunit.client.1.smithi192.stdout:7/614: write d2/db/d21/d42/d5b/f5d [1382405,120384] 0 2024-04-24T22:45:35.633 INFO:tasks.workunit.client.0.smithi063.stdout:6/734: write db/d1d/d20/d3f/d7f/d68/d70/fad [87788,52213] 0 2024-04-24T22:45:35.638 INFO:tasks.workunit.client.1.smithi192.stdout:2/682: unlink d4/d9/d37/l65 0 2024-04-24T22:45:35.640 INFO:tasks.workunit.client.0.smithi063.stdout:5/681: dwrite d5/d1d/d7d/fe2 [0,4194304] 0 2024-04-24T22:45:35.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/674: link d3/dd/de/f16 d3/d5d/da1/d5e/fef 0 2024-04-24T22:45:35.643 INFO:tasks.workunit.client.1.smithi192.stdout:3/596: creat d2/d4/d12/d1a/d1d/d7b/fb6 x:0 0 0 2024-04-24T22:45:35.644 INFO:tasks.workunit.client.1.smithi192.stdout:3/597: chown d2/d4/d7/d4b/c56 0 1 2024-04-24T22:45:35.645 INFO:tasks.workunit.client.1.smithi192.stdout:3/598: chown d2/d4/d4e/fa3 128 1 2024-04-24T22:45:35.646 INFO:tasks.workunit.client.0.smithi063.stdout:4/675: unlink d2/c7 0 2024-04-24T22:45:35.647 INFO:tasks.workunit.client.0.smithi063.stdout:6/735: dwrite db/d1d/d20/d3f/d7f/d68/fe9 [0,4194304] 0 2024-04-24T22:45:35.647 INFO:tasks.workunit.client.0.smithi063.stdout:3/713: dread da/d18/d47/d4d/f5a [0,4194304] 0 2024-04-24T22:45:35.648 INFO:tasks.workunit.client.1.smithi192.stdout:2/683: rename d4/d29/d60/lb9 to d4/lde 0 2024-04-24T22:45:35.651 INFO:tasks.workunit.client.1.smithi192.stdout:4/486: mknod d2/db/d75/da1/d36/d5b/d28/d9f/ca7 0 2024-04-24T22:45:35.651 INFO:tasks.workunit.client.0.smithi063.stdout:2/788: dwrite d0/d2/d2d/d31/d89/f25 [0,4194304] 0 2024-04-24T22:45:35.653 INFO:tasks.workunit.client.0.smithi063.stdout:9/720: dread d6/d2f/dcd/f17 [0,4194304] 0 2024-04-24T22:45:35.655 INFO:tasks.workunit.client.1.smithi192.stdout:9/675: creat d3/dd/de/da7/dae/ff0 x:0 0 0 2024-04-24T22:45:35.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/643: fdatasync d0/d4/f3a 0 2024-04-24T22:45:35.661 INFO:tasks.workunit.client.0.smithi063.stdout:6/736: dwrite db/d1d/d20/d3f/d7f/d68/f7e [4194304,4194304] 0 2024-04-24T22:45:35.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/644: dread - d0/d4/de/d6f/fd6 zero size 2024-04-24T22:45:35.661 INFO:tasks.workunit.client.1.smithi192.stdout:9/676: write d3/f31 [867173,10738] 0 2024-04-24T22:45:35.662 INFO:tasks.workunit.client.0.smithi063.stdout:7/645: chown d0/d4/de/d17/dd7/f84 1 1 2024-04-24T22:45:35.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/684: mknod d4/d9/d10/d6c/d6f/cdf 0 2024-04-24T22:45:35.664 INFO:tasks.workunit.client.1.smithi192.stdout:2/685: stat d4/d9/d10/d6c/d8c/fab 0 2024-04-24T22:45:35.665 INFO:tasks.workunit.client.1.smithi192.stdout:2/686: chown d4/d9/d10/d6c/fb5 8685569 1 2024-04-24T22:45:35.669 INFO:tasks.workunit.client.0.smithi063.stdout:3/714: mkdir da/d39/d94/db3/dfa 0 2024-04-24T22:45:35.671 INFO:tasks.workunit.client.0.smithi063.stdout:1/773: dwrite d3/de0/d79/f52 [0,4194304] 0 2024-04-24T22:45:35.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/774: write d3/da/d34/d99/ff9 [275788,12096] 0 2024-04-24T22:45:35.675 INFO:tasks.workunit.client.1.smithi192.stdout:4/487: getdents d2/db/d75/da1/d36/d5b/d64 0 2024-04-24T22:45:35.678 INFO:tasks.workunit.client.0.smithi063.stdout:8/734: truncate d1/d22/d33/da6/d54/f8e 1411428 0 2024-04-24T22:45:35.679 INFO:tasks.workunit.client.0.smithi063.stdout:9/721: rmdir d6/d2f/dcd/d53/dbe 39 2024-04-24T22:45:35.681 INFO:tasks.workunit.client.1.smithi192.stdout:4/488: symlink d2/d9/d25/d2e/la8 0 2024-04-24T22:45:35.683 INFO:tasks.workunit.client.0.smithi063.stdout:6/737: mkdir db/d1d/d20/d29/d5d/df5 0 2024-04-24T22:45:35.684 INFO:tasks.workunit.client.0.smithi063.stdout:6/738: write db/ff [1596419,68619] 0 2024-04-24T22:45:35.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/652: getdents d6/da 0 2024-04-24T22:45:35.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/599: dwrite d2/d27/f4a [0,4194304] 0 2024-04-24T22:45:35.692 INFO:tasks.workunit.client.0.smithi063.stdout:2/789: creat d0/d2/d2f/ddc/ff3 x:0 0 0 2024-04-24T22:45:35.694 INFO:tasks.workunit.client.1.smithi192.stdout:2/687: dwrite d4/d9/d10/d6c/d95/db7/dbf/fc7 [0,4194304] 0 2024-04-24T22:45:35.695 INFO:tasks.workunit.client.0.smithi063.stdout:9/722: chown d6/d2f/dcd/db/d6b/d4f/dad/le2 2 1 2024-04-24T22:45:35.696 INFO:tasks.workunit.client.1.smithi192.stdout:3/600: creat d2/d27/d28/fb7 x:0 0 0 2024-04-24T22:45:35.701 INFO:tasks.workunit.client.0.smithi063.stdout:7/646: link d0/d4/de/d17/d35/d6c/d7a/fa5 d0/d4/de/d17/d35/d6c/d8e/fd9 0 2024-04-24T22:45:35.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/489: dread d2/db/d75/da1/f4c [0,4194304] 0 2024-04-24T22:45:35.703 INFO:tasks.workunit.client.1.smithi192.stdout:2/688: rmdir d4/d29/d60 39 2024-04-24T22:45:35.704 INFO:tasks.workunit.client.1.smithi192.stdout:2/689: fdatasync d4/d9/d37/f9c 0 2024-04-24T22:45:35.704 INFO:tasks.workunit.client.1.smithi192.stdout:4/490: chown d2/d9/d25/d2e/c38 444816092 1 2024-04-24T22:45:35.705 INFO:tasks.workunit.client.1.smithi192.stdout:2/690: chown d4/d9/d10/c42 106163 1 2024-04-24T22:45:35.706 INFO:tasks.workunit.client.1.smithi192.stdout:2/691: chown d4 3281807 1 2024-04-24T22:45:35.706 INFO:tasks.workunit.client.0.smithi063.stdout:6/739: mknod db/d85/db5/cf6 0 2024-04-24T22:45:35.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/740: truncate db/d1d/d20/d29/dab/dd0/ff4 530242 0 2024-04-24T22:45:35.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/653: creat d6/da/d2b/fe3 x:0 0 0 2024-04-24T22:45:35.709 INFO:tasks.workunit.client.0.smithi063.stdout:5/682: mkdir d5/d11/d76/d92/dea 0 2024-04-24T22:45:35.711 INFO:tasks.workunit.client.0.smithi063.stdout:5/683: truncate d5/d24/f51 765042 0 2024-04-24T22:45:35.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/715: mkdir da/d18/dfb 0 2024-04-24T22:45:35.713 INFO:tasks.workunit.client.0.smithi063.stdout:8/735: dwrite d1/d11/f29 [0,4194304] 0 2024-04-24T22:45:35.713 INFO:tasks.workunit.client.1.smithi192.stdout:3/601: truncate d2/d4/d12/fb4 2147910 0 2024-04-24T22:45:35.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/775: dread d3/db5/d21/d9f/daa/db7/f58 [0,4194304] 0 2024-04-24T22:45:35.715 INFO:tasks.workunit.client.1.smithi192.stdout:4/491: rename d2/l46 to d2/db/d75/da1/d36/d5b/d28/la9 0 2024-04-24T22:45:35.716 INFO:tasks.workunit.client.0.smithi063.stdout:2/790: creat d0/d2/d2d/d31/d89/d1c/ff4 x:0 0 0 2024-04-24T22:45:35.716 INFO:tasks.workunit.client.0.smithi063.stdout:2/791: chown d0/l4a 3235 1 2024-04-24T22:45:35.717 INFO:tasks.workunit.client.1.smithi192.stdout:2/692: mknod d4/d29/d60/d9a/db3/ce0 0 2024-04-24T22:45:35.718 INFO:tasks.workunit.client.0.smithi063.stdout:2/792: write d0/d2/d35/dc6/de2/de4/fe9 [846950,69854] 0 2024-04-24T22:45:35.718 INFO:tasks.workunit.client.1.smithi192.stdout:2/693: readlink d4/d29/d60/d9a/ldd 0 2024-04-24T22:45:35.721 INFO:tasks.workunit.client.1.smithi192.stdout:4/492: dread d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:35.722 INFO:tasks.workunit.client.1.smithi192.stdout:2/694: truncate d4/d9/d10/d6c/d6f/da8/fb0 368261 0 2024-04-24T22:45:35.722 INFO:tasks.workunit.client.1.smithi192.stdout:3/602: chown d2/d4/d7/d4b/f50 197744 1 2024-04-24T22:45:35.722 INFO:tasks.workunit.client.0.smithi063.stdout:7/647: mkdir d0/d4/de/d17/d35/d69/da0/dda 0 2024-04-24T22:45:35.722 INFO:tasks.workunit.client.0.smithi063.stdout:9/723: dwrite d6/d2f/dcd/db/d6b/d4f/f5a [0,4194304] 0 2024-04-24T22:45:35.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/493: write d2/d9/d25/f68 [183291,8849] 0 2024-04-24T22:45:35.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/724: chown d6/d2f/d42/l8b 1426 1 2024-04-24T22:45:35.724 INFO:tasks.workunit.client.1.smithi192.stdout:3/603: write d2/d4/d7/d4f/f8c [1070887,123388] 0 2024-04-24T22:45:35.727 INFO:tasks.workunit.client.1.smithi192.stdout:3/604: truncate d2/d4/d12/d1a/d1d/d7b/fb6 863876 0 2024-04-24T22:45:35.730 INFO:tasks.workunit.client.0.smithi063.stdout:4/676: getdents d2/d8/d40 0 2024-04-24T22:45:35.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/716: creat da/d18/d3a/d63/ffc x:0 0 0 2024-04-24T22:45:35.736 INFO:tasks.workunit.client.0.smithi063.stdout:8/736: rmdir d1/d22/d33/da6 39 2024-04-24T22:45:35.740 INFO:tasks.workunit.client.0.smithi063.stdout:1/776: chown d3/db5/d21/l81 1402 1 2024-04-24T22:45:35.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/793: mknod d0/d2/d2d/d31/d89/d7/d86/da4/cf5 0 2024-04-24T22:45:35.742 INFO:tasks.workunit.client.0.smithi063.stdout:6/741: write db/d1d/d20/d3f/d7f/d68/fb0 [192560,3713] 0 2024-04-24T22:45:35.744 INFO:tasks.workunit.client.0.smithi063.stdout:8/737: dread d1/d11/d8b/f8d [0,4194304] 0 2024-04-24T22:45:35.747 INFO:tasks.workunit.client.0.smithi063.stdout:0/654: dwrite d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:35.747 INFO:tasks.workunit.client.0.smithi063.stdout:3/717: symlink da/d18/d9e/da3/lfd 0 2024-04-24T22:45:35.748 INFO:tasks.workunit.client.0.smithi063.stdout:0/655: chown d6/da/dc/f71 452 1 2024-04-24T22:45:35.748 INFO:tasks.workunit.client.0.smithi063.stdout:2/794: mknod d0/dd/d72/dc9/cf6 0 2024-04-24T22:45:35.751 INFO:tasks.workunit.client.0.smithi063.stdout:8/738: link d1/d2/d17/f4b d1/d11/d8b/dd6/ff0 0 2024-04-24T22:45:35.752 INFO:tasks.workunit.client.0.smithi063.stdout:8/739: chown d1/d2/fba 130280224 1 2024-04-24T22:45:35.753 INFO:tasks.workunit.client.0.smithi063.stdout:2/795: fsync d0/d2/d35/dc6/de2/f6b 0 2024-04-24T22:45:35.754 INFO:tasks.workunit.client.0.smithi063.stdout:2/796: truncate d0/fbc 307587 0 2024-04-24T22:45:35.755 INFO:tasks.workunit.client.0.smithi063.stdout:3/718: getdents da/d18/d9e/da3/dc1 0 2024-04-24T22:45:35.755 INFO:tasks.workunit.client.0.smithi063.stdout:6/742: link db/d1d/l65 db/d1c/dc3/lf7 0 2024-04-24T22:45:35.756 INFO:tasks.workunit.client.0.smithi063.stdout:8/740: truncate d1/d22/fab 1965784 0 2024-04-24T22:45:35.758 INFO:tasks.workunit.client.0.smithi063.stdout:3/719: rename da/d39/d94/dbc/ccf to da/d39/d94/db3/dbe/cfe 0 2024-04-24T22:45:35.759 INFO:tasks.workunit.client.0.smithi063.stdout:2/797: dwrite d0/d2/d27/f85 [0,4194304] 0 2024-04-24T22:45:35.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/720: mkdir da/d39/dff 0 2024-04-24T22:45:35.762 INFO:tasks.workunit.client.0.smithi063.stdout:3/721: mknod da/d18/c100 0 2024-04-24T22:45:35.829 INFO:tasks.workunit.client.0.smithi063.stdout:4/677: read d2/d8/d3c/f75 [228106,67032] 0 2024-04-24T22:45:35.829 INFO:tasks.workunit.client.0.smithi063.stdout:4/678: fdatasync d2/d19/d33/dc1/f7f 0 2024-04-24T22:45:35.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/656: read d6/da/f14 [2354118,50365] 0 2024-04-24T22:45:35.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/494: dwrite d2/db/d75/da1/d36/d5b/f6e [0,4194304] 0 2024-04-24T22:45:35.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/695: rmdir d4/d9/d10/d6c/d6f/dc9 0 2024-04-24T22:45:35.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/679: dwrite d2/d80/f8f [0,4194304] 0 2024-04-24T22:45:35.838 INFO:tasks.workunit.client.1.smithi192.stdout:2/696: creat d4/fe1 x:0 0 0 2024-04-24T22:45:35.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/697: fdatasync d4/d9/d10/f58 0 2024-04-24T22:45:35.850 INFO:tasks.workunit.client.0.smithi063.stdout:4/680: symlink d2/d8/d40/d51/dd5/ldc 0 2024-04-24T22:45:35.850 INFO:tasks.workunit.client.0.smithi063.stdout:9/725: write d6/d2f/dcd/d1f/d20/d2a/dcb/f33 [1899608,118771] 0 2024-04-24T22:45:35.851 INFO:tasks.workunit.client.1.smithi192.stdout:3/605: write d2/d27/d28/f30 [3502033,58918] 0 2024-04-24T22:45:35.853 INFO:tasks.workunit.client.0.smithi063.stdout:5/684: write d5/d11/d76/f7b [4217074,6416] 0 2024-04-24T22:45:35.855 INFO:tasks.workunit.client.0.smithi063.stdout:9/726: rmdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e 39 2024-04-24T22:45:35.857 INFO:tasks.workunit.client.1.smithi192.stdout:3/606: mknod d2/d4/d12/d1a/d1d/cb8 0 2024-04-24T22:45:35.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/607: fdatasync d2/f33 0 2024-04-24T22:45:35.858 INFO:tasks.workunit.client.0.smithi063.stdout:6/743: sync 2024-04-24T22:45:35.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/648: truncate d0/d4/de/d6f/d5c/d8b/fae 3359512 0 2024-04-24T22:45:35.860 INFO:tasks.workunit.client.0.smithi063.stdout:4/681: creat d2/d19/d33/dc1/d50/dc3/fdd x:0 0 0 2024-04-24T22:45:35.860 INFO:tasks.workunit.client.0.smithi063.stdout:5/685: creat d5/d24/d49/d3b/dc5/feb x:0 0 0 2024-04-24T22:45:35.861 INFO:tasks.workunit.client.0.smithi063.stdout:1/777: dwrite d3/da/da2/dba/d11/f89 [0,4194304] 0 2024-04-24T22:45:35.861 INFO:tasks.workunit.client.0.smithi063.stdout:7/649: chown d0/d4/de/d17/dd7/f84 891458635 1 2024-04-24T22:45:35.862 INFO:tasks.workunit.client.0.smithi063.stdout:4/682: write d2/d8/f71 [3612859,34670] 0 2024-04-24T22:45:35.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/608: mknod d2/d4/d12/d39/cb9 0 2024-04-24T22:45:35.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/686: dwrite d5/d24/db6/fcc [0,4194304] 0 2024-04-24T22:45:35.870 INFO:tasks.workunit.client.0.smithi063.stdout:6/744: symlink db/d1d/d2d/lf8 0 2024-04-24T22:45:35.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/745: chown db/d1d/d2d/d48 201 1 2024-04-24T22:45:35.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/609: mknod d2/d4/d12/d39/d86/cba 0 2024-04-24T22:45:35.872 INFO:tasks.workunit.client.0.smithi063.stdout:5/687: write d5/f1c [979347,118677] 0 2024-04-24T22:45:35.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/746: readlink db/d1d/d2d/d69/dd4/de4/lba 0 2024-04-24T22:45:35.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/798: write d0/d2/d2d/d31/d89/d7/f13 [915487,47095] 0 2024-04-24T22:45:35.878 INFO:tasks.workunit.client.1.smithi192.stdout:3/610: mkdir d2/d4/d7/d4f/dbb 0 2024-04-24T22:45:35.879 INFO:tasks.workunit.client.0.smithi063.stdout:8/741: dread d1/d21/f5d [0,4194304] 0 2024-04-24T22:45:35.879 INFO:tasks.workunit.client.0.smithi063.stdout:2/799: read d0/d2/d2d/d31/d89/d48/f5d [978354,6198] 0 2024-04-24T22:45:35.879 INFO:tasks.workunit.client.1.smithi192.stdout:3/611: readlink d2/d4/d12/d1a/l99 0 2024-04-24T22:45:35.883 INFO:tasks.workunit.client.0.smithi063.stdout:2/800: write d0/dd/fc7 [300158,105937] 0 2024-04-24T22:45:35.883 INFO:tasks.workunit.client.0.smithi063.stdout:0/657: dread d6/da/d2b/d60/f76 [0,4194304] 0 2024-04-24T22:45:35.884 INFO:tasks.workunit.client.0.smithi063.stdout:6/747: stat db/d1d/d20/d3f/d7f/d68/d70/l79 0 2024-04-24T22:45:35.884 INFO:tasks.workunit.client.1.smithi192.stdout:2/698: write d4/f69 [3640098,110098] 0 2024-04-24T22:45:35.885 INFO:tasks.workunit.client.0.smithi063.stdout:0/658: stat d6/da/f3d 0 2024-04-24T22:45:35.885 INFO:tasks.workunit.client.0.smithi063.stdout:2/801: readlink d0/d2/d2d/d31/d89/l38 0 2024-04-24T22:45:35.886 INFO:tasks.workunit.client.0.smithi063.stdout:3/722: dwrite da/d18/f30 [0,4194304] 0 2024-04-24T22:45:35.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/699: dread - d4/d9/d10/fcf zero size 2024-04-24T22:45:35.891 INFO:tasks.workunit.client.1.smithi192.stdout:4/495: truncate d2/db/d75/da1/d36/d5b/d28/f2b 837598 0 2024-04-24T22:45:35.897 INFO:tasks.workunit.client.0.smithi063.stdout:9/727: link d6/l80 d6/d2f/dcd/db/d6b/d4f/dad/d48/lee 0 2024-04-24T22:45:35.898 INFO:tasks.workunit.client.0.smithi063.stdout:4/683: rmdir d2/d19/d33 39 2024-04-24T22:45:35.905 INFO:tasks.workunit.client.0.smithi063.stdout:7/650: dwrite d0/d4/de/f61 [0,4194304] 0 2024-04-24T22:45:35.911 INFO:tasks.workunit.client.0.smithi063.stdout:9/728: readlink d6/d2f/dcd/l26 0 2024-04-24T22:45:35.914 INFO:tasks.workunit.client.0.smithi063.stdout:8/742: dwrite d1/d11/d32/d38/f67 [0,4194304] 0 2024-04-24T22:45:35.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/659: dwrite d6/da/d2b/d60/fa1 [0,4194304] 0 2024-04-24T22:45:35.923 INFO:tasks.workunit.client.0.smithi063.stdout:7/651: creat d0/d4/de/d17/d35/d69/db1/fdb x:0 0 0 2024-04-24T22:45:35.926 INFO:tasks.workunit.client.0.smithi063.stdout:6/748: truncate db/d1c/f89 1661037 0 2024-04-24T22:45:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:35 smithi192 ceph-mon[33991]: pgmap v15: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 71 MiB/s rd, 209 MiB/s wr, 402 op/s 2024-04-24T22:45:35.929 INFO:tasks.workunit.client.0.smithi063.stdout:9/729: mkdir d6/d2f/d42/d81/def 0 2024-04-24T22:45:35.931 INFO:tasks.workunit.client.0.smithi063.stdout:4/684: unlink d2/d8/d36/f7d 0 2024-04-24T22:45:35.931 INFO:tasks.workunit.client.0.smithi063.stdout:3/723: write da/d40/f81 [173153,115486] 0 2024-04-24T22:45:35.936 INFO:tasks.workunit.client.0.smithi063.stdout:2/802: creat d0/d2/d2d/ff7 x:0 0 0 2024-04-24T22:45:35.936 INFO:tasks.workunit.client.0.smithi063.stdout:3/724: dread da/d48/f95 [0,4194304] 0 2024-04-24T22:45:35.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/749: write db/d1d/d2d/d48/f63 [777895,90987] 0 2024-04-24T22:45:35.941 INFO:tasks.workunit.client.0.smithi063.stdout:9/730: symlink d6/d2f/dcd/db/d6b/d4f/dad/d36/lf0 0 2024-04-24T22:45:35.942 INFO:tasks.workunit.client.0.smithi063.stdout:4/685: symlink d2/d19/d82/lde 0 2024-04-24T22:45:35.946 INFO:tasks.workunit.client.0.smithi063.stdout:9/731: dread d6/d2f/dcd/d1f/f28 [0,4194304] 0 2024-04-24T22:45:35.946 INFO:tasks.workunit.client.0.smithi063.stdout:9/732: dread - d6/dbd/dcf/fec zero size 2024-04-24T22:45:35.950 INFO:tasks.workunit.client.0.smithi063.stdout:6/750: unlink db/d1d/fc9 0 2024-04-24T22:45:35.954 INFO:tasks.workunit.client.1.smithi192.stdout:3/612: rename d2/d4/d12/d39/d86 to d2/d4/d12/dbc 0 2024-04-24T22:45:35.958 INFO:tasks.workunit.client.0.smithi063.stdout:7/652: dread d0/d4/f11 [0,4194304] 0 2024-04-24T22:45:35.959 INFO:tasks.workunit.client.0.smithi063.stdout:7/653: stat d0/d4/de/c50 0 2024-04-24T22:45:35.961 INFO:tasks.workunit.client.0.smithi063.stdout:8/743: mkdir d1/d22/d33/da6/d54/df1 0 2024-04-24T22:45:35.961 INFO:tasks.workunit.client.0.smithi063.stdout:2/803: dread d0/d2/d2d/d31/d89/d1c/d3f/f42 [0,4194304] 0 2024-04-24T22:45:35.965 INFO:tasks.workunit.client.1.smithi192.stdout:3/613: unlink d2/d4/d7/f7c 0 2024-04-24T22:45:35.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/686: dread d2/f2e [0,4194304] 0 2024-04-24T22:45:35.968 INFO:tasks.workunit.client.0.smithi063.stdout:9/733: symlink d6/d2f/dcd/db/d1c/d78/lf1 0 2024-04-24T22:45:35.972 INFO:tasks.workunit.client.1.smithi192.stdout:3/614: rmdir d2/d4/d12/d1a/d1d 39 2024-04-24T22:45:35.978 INFO:tasks.workunit.client.0.smithi063.stdout:4/687: dread d2/d8/f1a [0,4194304] 0 2024-04-24T22:45:35.979 INFO:tasks.workunit.client.0.smithi063.stdout:7/654: dread d0/d4/de/d17/f8a [0,4194304] 0 2024-04-24T22:45:35.979 INFO:tasks.workunit.client.0.smithi063.stdout:8/744: creat d1/d2/d3/d86/ff2 x:0 0 0 2024-04-24T22:45:35.979 INFO:tasks.workunit.client.0.smithi063.stdout:7/655: dread - d0/d4/de/d6f/fd6 zero size 2024-04-24T22:45:35.980 INFO:tasks.workunit.client.0.smithi063.stdout:0/660: link d6/d64/d24/l46 d6/da/dc/le4 0 2024-04-24T22:45:35.980 INFO:tasks.workunit.client.1.smithi192.stdout:3/615: dwrite d2/d4/d12/f1b [0,4194304] 0 2024-04-24T22:45:35.982 INFO:tasks.workunit.client.1.smithi192.stdout:3/616: write d2/d27/d28/da5/d93/faa [103639,121381] 0 2024-04-24T22:45:35.983 INFO:tasks.workunit.client.0.smithi063.stdout:9/734: mkdir d6/d2f/dcd/d1f/d20/d2a/df2 0 2024-04-24T22:45:35.989 INFO:tasks.workunit.client.0.smithi063.stdout:1/778: sync 2024-04-24T22:45:35.991 INFO:tasks.workunit.client.1.smithi192.stdout:3/617: mkdir d2/d4/d12/d1a/d1d/d55/d5f/dbd 0 2024-04-24T22:45:35.992 INFO:tasks.workunit.client.0.smithi063.stdout:7/656: mkdir d0/d4/de/d6f/d2e/d55/ddc 0 2024-04-24T22:45:35.993 INFO:tasks.workunit.client.0.smithi063.stdout:0/661: mknod d6/d64/d24/d42/dd2/ce5 0 2024-04-24T22:45:35.995 INFO:tasks.workunit.client.1.smithi192.stdout:3/618: rmdir d2/d4/d7/d4b/d54 39 2024-04-24T22:45:35.997 INFO:tasks.workunit.client.0.smithi063.stdout:0/662: dread d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:35.999 INFO:tasks.workunit.client.1.smithi192.stdout:3/619: creat d2/d4/d7/fbe x:0 0 0 2024-04-24T22:45:36.001 INFO:tasks.workunit.client.1.smithi192.stdout:4/496: write d2/f1f [2788340,56624] 0 2024-04-24T22:45:36.001 INFO:tasks.workunit.client.1.smithi192.stdout:3/620: chown d2/d4/d7/da0 84 1 2024-04-24T22:45:36.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/725: dwrite da/d18/d47/d4d/d62/f80 [0,4194304] 0 2024-04-24T22:45:36.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/726: dread - da/fa0 zero size 2024-04-24T22:45:36.001 INFO:tasks.workunit.client.0.smithi063.stdout:2/804: creat d0/ff8 x:0 0 0 2024-04-24T22:45:36.003 INFO:tasks.workunit.client.1.smithi192.stdout:2/700: truncate d4/d9/d10/d6c/d95/db7/dbf/fc7 691056 0 2024-04-24T22:45:36.005 INFO:tasks.workunit.client.1.smithi192.stdout:2/701: dread d4/d9/d10/d6c/f85 [0,4194304] 0 2024-04-24T22:45:36.005 INFO:tasks.workunit.client.1.smithi192.stdout:4/497: rmdir d2/db/d5d/d77 39 2024-04-24T22:45:36.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/751: write db/f10 [77403,30654] 0 2024-04-24T22:45:36.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/752: fsync db/d1d/d20/d3f/d7f/d68/fb0 0 2024-04-24T22:45:36.010 INFO:tasks.workunit.client.1.smithi192.stdout:2/702: mkdir d4/d9/d10/d6c/d8c/da5/de2 0 2024-04-24T22:45:36.011 INFO:tasks.workunit.client.1.smithi192.stdout:2/703: chown d4/ca 86 1 2024-04-24T22:45:36.013 INFO:tasks.workunit.client.0.smithi063.stdout:6/753: dread db/fb7 [0,4194304] 0 2024-04-24T22:45:36.014 INFO:tasks.workunit.client.1.smithi192.stdout:3/621: mknod d2/d4/d12/cbf 0 2024-04-24T22:45:36.015 INFO:tasks.workunit.client.0.smithi063.stdout:7/657: fdatasync d0/d4/de/d6f/d5c/fcd 0 2024-04-24T22:45:36.017 INFO:tasks.workunit.client.0.smithi063.stdout:9/735: write d6/d2f/dcd/db/d6b/d4f/f79 [143150,129898] 0 2024-04-24T22:45:36.020 INFO:tasks.workunit.client.1.smithi192.stdout:4/498: rename d2/db/d75/da1/d36/d5b/d64/f83 to d2/faa 0 2024-04-24T22:45:36.021 INFO:tasks.workunit.client.0.smithi063.stdout:4/688: truncate d2/d8/f71 3185317 0 2024-04-24T22:45:36.025 INFO:tasks.workunit.client.1.smithi192.stdout:3/622: symlink d2/d4/d12/d1a/d1d/d55/d5f/dbd/lc0 0 2024-04-24T22:45:36.026 INFO:tasks.workunit.client.0.smithi063.stdout:3/727: dread da/d39/f7a [0,4194304] 0 2024-04-24T22:45:36.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/745: truncate d1/d11/f29 1421852 0 2024-04-24T22:45:36.027 INFO:tasks.workunit.client.1.smithi192.stdout:3/623: truncate d2/d4/d12/d1a/d1d/d55/d95/f92 705546 0 2024-04-24T22:45:36.027 INFO:tasks.workunit.client.0.smithi063.stdout:3/728: chown da/fa2 3874 1 2024-04-24T22:45:36.028 INFO:tasks.workunit.client.0.smithi063.stdout:2/805: creat d0/d2/d2d/d31/d89/d7/d86/da0/d94/ff9 x:0 0 0 2024-04-24T22:45:36.029 INFO:tasks.workunit.client.0.smithi063.stdout:2/806: readlink d0/d2/d39/l45 0 2024-04-24T22:45:36.030 INFO:tasks.workunit.client.1.smithi192.stdout:4/499: mknod d2/db/d75/da1/d7d/d9c/cab 0 2024-04-24T22:45:36.032 INFO:tasks.workunit.client.0.smithi063.stdout:5/688: sync 2024-04-24T22:45:36.040 INFO:tasks.workunit.client.0.smithi063.stdout:0/663: dwrite d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:36.042 INFO:tasks.workunit.client.1.smithi192.stdout:3/624: dread d2/f33 [0,4194304] 0 2024-04-24T22:45:36.044 INFO:tasks.workunit.client.0.smithi063.stdout:9/736: symlink d6/d2f/dcd/db/d6b/d4f/d59/lf3 0 2024-04-24T22:45:36.044 INFO:tasks.workunit.client.0.smithi063.stdout:4/689: mkdir d2/d80/d8d/ddf 0 2024-04-24T22:45:36.047 INFO:tasks.workunit.client.0.smithi063.stdout:8/746: mkdir d1/d21/df3 0 2024-04-24T22:45:36.048 INFO:tasks.workunit.client.1.smithi192.stdout:4/500: write d2/f17 [2066626,36294] 0 2024-04-24T22:45:36.049 INFO:tasks.workunit.client.0.smithi063.stdout:8/747: read d1/d21/f83 [293263,76521] 0 2024-04-24T22:45:36.052 INFO:tasks.workunit.client.0.smithi063.stdout:5/689: read - d5/d24/d49/fb4 zero size 2024-04-24T22:45:36.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:35 smithi063 ceph-mon[26912]: pgmap v15: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 71 MiB/s rd, 209 MiB/s wr, 402 op/s 2024-04-24T22:45:36.054 INFO:tasks.workunit.client.0.smithi063.stdout:2/807: dread d0/d2/d2d/d31/d89/d7/d86/da0/d94/fc0 [0,4194304] 0 2024-04-24T22:45:36.055 INFO:tasks.workunit.client.0.smithi063.stdout:2/808: readlink d0/d2/d2d/d31/d89/d7/d86/l8d 0 2024-04-24T22:45:36.057 INFO:tasks.workunit.client.1.smithi192.stdout:4/501: dwrite d2/db/d5d/d77/d82/fa3 [0,4194304] 0 2024-04-24T22:45:36.058 INFO:tasks.workunit.client.0.smithi063.stdout:3/729: getdents da/d18/d9e/da3/dc1 0 2024-04-24T22:45:36.060 INFO:tasks.workunit.client.0.smithi063.stdout:1/779: getdents d3/da/d34/d99 0 2024-04-24T22:45:36.060 INFO:tasks.workunit.client.1.smithi192.stdout:4/502: write d2/db/d75/da1/d36/d5b/d28/d2f/fa2 [3579550,34470] 0 2024-04-24T22:45:36.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/658: rename d0/d4/de/d17/d35/c56 to d0/d4/de/d6f/cdd 0 2024-04-24T22:45:36.070 INFO:tasks.workunit.client.0.smithi063.stdout:6/754: write db/d1d/d2d/d48/f51 [1200773,107093] 0 2024-04-24T22:45:36.070 INFO:tasks.workunit.client.0.smithi063.stdout:4/690: dread d2/fd [0,4194304] 0 2024-04-24T22:45:36.070 INFO:tasks.workunit.client.0.smithi063.stdout:6/755: dread - db/d85/fc7 zero size 2024-04-24T22:45:36.072 INFO:tasks.workunit.client.1.smithi192.stdout:3/625: dread d2/d62/f75 [0,4194304] 0 2024-04-24T22:45:36.073 INFO:tasks.workunit.client.0.smithi063.stdout:2/809: creat d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa x:0 0 0 2024-04-24T22:45:36.077 INFO:tasks.workunit.client.0.smithi063.stdout:6/756: dread db/d1d/d2d/f4d [0,4194304] 0 2024-04-24T22:45:36.077 INFO:tasks.workunit.client.0.smithi063.stdout:1/780: creat d3/db5/d21/d9f/dd5/f102 x:0 0 0 2024-04-24T22:45:36.082 INFO:tasks.workunit.client.1.smithi192.stdout:3/626: mkdir d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1 0 2024-04-24T22:45:36.083 INFO:tasks.workunit.client.0.smithi063.stdout:5/690: unlink d5/d24/d49/d3b/cae 0 2024-04-24T22:45:36.086 INFO:tasks.workunit.client.0.smithi063.stdout:8/748: rename d1/d21/f61 to d1/d22/d33/da6/d54/d79/ff4 0 2024-04-24T22:45:36.087 INFO:tasks.workunit.client.1.smithi192.stdout:4/503: write d2/db/d75/da1/d36/f62 [793869,113827] 0 2024-04-24T22:45:36.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/810: unlink d0/d2/fa1 0 2024-04-24T22:45:36.091 INFO:tasks.workunit.client.1.smithi192.stdout:3/627: truncate d2/d4/d12/fb4 2672384 0 2024-04-24T22:45:36.092 INFO:tasks.workunit.client.0.smithi063.stdout:2/811: stat d0/d2/d2d/d31/d89/d7/f13 0 2024-04-24T22:45:36.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/737: write d6/d2f/dcd/db/d6b/d4f/dad/f34 [2006811,36591] 0 2024-04-24T22:45:36.097 INFO:tasks.workunit.client.1.smithi192.stdout:4/504: link d2/d9/d25/d2e/f66 d2/d9/d6b/fac 0 2024-04-24T22:45:36.098 INFO:tasks.workunit.client.1.smithi192.stdout:3/628: dwrite d2/d4/d12/d1a/d1d/d55/d95/f92 [0,4194304] 0 2024-04-24T22:45:36.099 INFO:tasks.workunit.client.0.smithi063.stdout:0/664: getdents d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:36.099 INFO:tasks.workunit.client.1.smithi192.stdout:4/505: chown d2/db/d75/da1/d36/d5b/f6e 1 1 2024-04-24T22:45:36.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/506: fdatasync d2/db/d75/da1/d36/d5b/d64/f91 0 2024-04-24T22:45:36.102 INFO:tasks.workunit.client.0.smithi063.stdout:8/749: chown d1/d22/l30 105222 1 2024-04-24T22:45:36.102 INFO:tasks.workunit.client.1.smithi192.stdout:4/507: readlink d2/d9/d6b/l6d 0 2024-04-24T22:45:36.106 INFO:tasks.workunit.client.0.smithi063.stdout:2/812: creat d0/d2/d2d/d31/d89/d7/d86/d8e/ffb x:0 0 0 2024-04-24T22:45:36.107 INFO:tasks.workunit.client.1.smithi192.stdout:3/629: creat d2/d27/d28/dab/fc2 x:0 0 0 2024-04-24T22:45:36.109 INFO:tasks.workunit.client.0.smithi063.stdout:9/738: rmdir d6/d2f/dcd/db/d6b 39 2024-04-24T22:45:36.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/659: dwrite d0/d4/de/d17/d28/d31/f6a [0,4194304] 0 2024-04-24T22:45:36.114 INFO:tasks.workunit.client.0.smithi063.stdout:5/691: mkdir d5/d11/dec 0 2024-04-24T22:45:36.114 INFO:tasks.workunit.client.1.smithi192.stdout:3/630: symlink d2/d4/d7/d4b/lc3 0 2024-04-24T22:45:36.116 INFO:tasks.workunit.client.0.smithi063.stdout:0/665: chown d6/d64/d21/d25/d62/d63/d89/d95/da4/ld8 2 1 2024-04-24T22:45:36.116 INFO:tasks.workunit.client.0.smithi063.stdout:8/750: creat d1/d21/de6/ff5 x:0 0 0 2024-04-24T22:45:36.116 INFO:tasks.workunit.client.0.smithi063.stdout:3/730: dwrite da/f1d [0,4194304] 0 2024-04-24T22:45:36.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/757: dwrite db/d1d/d20/d3f/d7f/d68/f6d [0,4194304] 0 2024-04-24T22:45:36.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/666: dread - d6/d74/fab zero size 2024-04-24T22:45:36.122 INFO:tasks.workunit.client.1.smithi192.stdout:4/508: getdents d2/db/d75 0 2024-04-24T22:45:36.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/813: chown d0/d2/f6a 1984 1 2024-04-24T22:45:36.124 INFO:tasks.workunit.client.1.smithi192.stdout:3/631: symlink d2/d4/d7/d41/dac/lc4 0 2024-04-24T22:45:36.126 INFO:tasks.workunit.client.1.smithi192.stdout:4/509: chown d2/d9/d25/f8e 7 1 2024-04-24T22:45:36.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/758: dwrite db/d1d/d20/fd6 [0,4194304] 0 2024-04-24T22:45:36.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/660: fsync d0/d4/f75 0 2024-04-24T22:45:36.131 INFO:tasks.workunit.client.0.smithi063.stdout:5/692: rmdir d5/d1d/da3 39 2024-04-24T22:45:36.131 INFO:tasks.workunit.client.1.smithi192.stdout:3/632: mknod d2/d27/d28/da5/cc5 0 2024-04-24T22:45:36.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/510: dwrite d2/db/d75/da1/d36/d5b/f47 [0,4194304] 0 2024-04-24T22:45:36.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/511: truncate d2/db/d75/da1/d36/d5b/d28/d2f/f3c 109044 0 2024-04-24T22:45:36.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/512: stat d2/db/d75/da1/d36/d5b/d28/c7c 0 2024-04-24T22:45:36.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/513: rename d2/c5 to d2/da5/cad 0 2024-04-24T22:45:36.148 INFO:tasks.workunit.client.1.smithi192.stdout:4/514: unlink d2/d9/l4b 0 2024-04-24T22:45:36.149 INFO:tasks.workunit.client.1.smithi192.stdout:4/515: chown d2/d9/d25/f32 3171 1 2024-04-24T22:45:36.150 INFO:tasks.workunit.client.1.smithi192.stdout:4/516: write d2/d5c/f86 [520553,102542] 0 2024-04-24T22:45:36.169 INFO:tasks.workunit.client.0.smithi063.stdout:6/759: rename db/d1d/d20/d3f/cef to db/d1d/d2d/d69/dd4/cf9 0 2024-04-24T22:45:36.172 INFO:tasks.workunit.client.0.smithi063.stdout:8/751: rmdir d1/d22/d33/da6/d91/de5 0 2024-04-24T22:45:36.173 INFO:tasks.workunit.client.1.smithi192.stdout:4/517: dread d2/db/f4e [0,4194304] 0 2024-04-24T22:45:36.174 INFO:tasks.workunit.client.0.smithi063.stdout:7/661: rename d0/db5/cc8 to d0/d4/de/d95/cde 0 2024-04-24T22:45:36.175 INFO:tasks.workunit.client.0.smithi063.stdout:9/739: dread d6/d2f/dcd/d1f/d20/d22/f47 [0,4194304] 0 2024-04-24T22:45:36.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/731: dwrite da/f2e [0,4194304] 0 2024-04-24T22:45:36.176 INFO:tasks.workunit.client.0.smithi063.stdout:6/760: rmdir db/d1c/dea 39 2024-04-24T22:45:36.177 INFO:tasks.workunit.client.1.smithi192.stdout:4/518: symlink d2/db/d75/da1/d36/d5b/d28/lae 0 2024-04-24T22:45:36.184 INFO:tasks.workunit.client.1.smithi192.stdout:4/519: fsync d2/d9/d6b/fac 0 2024-04-24T22:45:36.185 INFO:tasks.workunit.client.0.smithi063.stdout:2/814: rename d0/d2/d2d/d31/d89/d7/c2e to d0/d2/d2d/d31/d89/d7/d86/da4/dd9/cfc 0 2024-04-24T22:45:36.187 INFO:tasks.workunit.client.0.smithi063.stdout:3/732: chown da/f1a 5572 1 2024-04-24T22:45:36.188 INFO:tasks.workunit.client.0.smithi063.stdout:5/693: dwrite d5/d24/d49/f6b [0,4194304] 0 2024-04-24T22:45:36.188 INFO:tasks.workunit.client.0.smithi063.stdout:7/662: mknod d0/d4/de/d17/d35/d69/db1/cdf 0 2024-04-24T22:45:36.189 INFO:tasks.workunit.client.1.smithi192.stdout:4/520: mknod d2/db/d75/da1/d7d/d9c/caf 0 2024-04-24T22:45:36.190 INFO:tasks.workunit.client.0.smithi063.stdout:6/761: creat db/d1d/d20/d29/dab/dd0/ffa x:0 0 0 2024-04-24T22:45:36.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/815: truncate d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa 797623 0 2024-04-24T22:45:36.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/733: creat da/db6/f101 x:0 0 0 2024-04-24T22:45:36.194 INFO:tasks.workunit.client.0.smithi063.stdout:3/734: chown da/d18/d47/d4d/ld1 3593282 1 2024-04-24T22:45:36.196 INFO:tasks.workunit.client.0.smithi063.stdout:6/762: unlink l9 0 2024-04-24T22:45:36.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/521: dwrite d2/d5c/f99 [0,4194304] 0 2024-04-24T22:45:36.199 INFO:tasks.workunit.client.0.smithi063.stdout:8/752: link d1/d21/f5d d1/d22/d33/ff6 0 2024-04-24T22:45:36.205 INFO:tasks.workunit.client.0.smithi063.stdout:5/694: creat d5/de5/fed x:0 0 0 2024-04-24T22:45:36.207 INFO:tasks.workunit.client.1.smithi192.stdout:4/522: symlink d2/lb0 0 2024-04-24T22:45:36.208 INFO:tasks.workunit.client.0.smithi063.stdout:5/695: dread d5/d1d/d7d/d67/f78 [0,4194304] 0 2024-04-24T22:45:36.209 INFO:tasks.workunit.client.0.smithi063.stdout:3/735: unlink da/d18/d47/d4d/l70 0 2024-04-24T22:45:36.209 INFO:tasks.workunit.client.0.smithi063.stdout:6/763: dread - db/d1d/d2d/f76 zero size 2024-04-24T22:45:36.211 INFO:tasks.workunit.client.0.smithi063.stdout:6/764: write db/d1d/d20/d29/f2e [4244029,48171] 0 2024-04-24T22:45:36.213 INFO:tasks.workunit.client.0.smithi063.stdout:6/765: fsync db/d1c/dc3/fc8 0 2024-04-24T22:45:36.213 INFO:tasks.workunit.client.1.smithi192.stdout:4/523: rename d2/d9/d25/f23 to d2/db/d75/da1/d36/d5b/d28/d2f/fb1 0 2024-04-24T22:45:36.214 INFO:tasks.workunit.client.1.smithi192.stdout:4/524: dread - d2/d9/d25/f71 zero size 2024-04-24T22:45:36.217 INFO:tasks.workunit.client.1.smithi192.stdout:4/525: truncate d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 808791 0 2024-04-24T22:45:36.218 INFO:tasks.workunit.client.0.smithi063.stdout:3/736: fdatasync da/f31 0 2024-04-24T22:45:36.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/526: symlink d2/db/d75/da1/lb2 0 2024-04-24T22:45:36.222 INFO:tasks.workunit.client.0.smithi063.stdout:3/737: dread - da/d39/d94/db3/dbe/ff4 zero size 2024-04-24T22:45:36.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/663: dread d0/d4/de/d17/f9b [0,4194304] 0 2024-04-24T22:45:36.224 INFO:tasks.workunit.client.0.smithi063.stdout:2/816: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:36.224 INFO:tasks.workunit.client.0.smithi063.stdout:5/696: dread d5/d24/f91 [0,4194304] 0 2024-04-24T22:45:36.225 INFO:tasks.workunit.client.0.smithi063.stdout:6/766: link db/d85/dd2/dd7/ld9 db/d1d/d20/d3e/lfb 0 2024-04-24T22:45:36.226 INFO:tasks.workunit.client.1.smithi192.stdout:4/527: mkdir d2/db/d5d/db3 0 2024-04-24T22:45:36.227 INFO:tasks.workunit.client.0.smithi063.stdout:2/817: chown d0/d2/f47 360 1 2024-04-24T22:45:36.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/767: mkdir db/d1d/d8d/dfc 0 2024-04-24T22:45:36.231 INFO:tasks.workunit.client.0.smithi063.stdout:8/753: truncate d1/d11/d32/fb2 17083 0 2024-04-24T22:45:36.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/768: stat db/d1d/d20/d29/dab/dd0/ffa 0 2024-04-24T22:45:36.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/528: dread d2/db/d75/da1/d36/d5b/f47 [0,4194304] 0 2024-04-24T22:45:36.232 INFO:tasks.workunit.client.0.smithi063.stdout:5/697: mkdir d5/d11/d23/d2d/d62/d81/da4/dee 0 2024-04-24T22:45:36.233 INFO:tasks.workunit.client.0.smithi063.stdout:5/698: chown d5/d24/d49/f6b 5388631 1 2024-04-24T22:45:36.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/664: rename d0/d4/de/l52 to d0/d4/de/d6f/d2e/d55/ddc/le0 0 2024-04-24T22:45:36.236 INFO:tasks.workunit.client.0.smithi063.stdout:6/769: dwrite db/d1d/d20/d29/d5d/fb6 [0,4194304] 0 2024-04-24T22:45:36.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/770: mkdir db/d1d/d2d/d48/dfd 0 2024-04-24T22:45:36.241 INFO:tasks.workunit.client.0.smithi063.stdout:3/738: link da/d18/d3a/da1/cc6 da/d18/dfb/c102 0 2024-04-24T22:45:36.242 INFO:tasks.workunit.client.1.smithi192.stdout:4/529: dread d2/db/d75/da1/d36/d5b/f24 [0,4194304] 0 2024-04-24T22:45:36.243 INFO:tasks.workunit.client.0.smithi063.stdout:3/739: readlink da/lb 0 2024-04-24T22:45:36.244 INFO:tasks.workunit.client.1.smithi192.stdout:4/530: symlink d2/d9/d6b/lb4 0 2024-04-24T22:45:36.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/754: getdents d1/d22/d33 0 2024-04-24T22:45:36.245 INFO:tasks.workunit.client.1.smithi192.stdout:4/531: readlink d2/d9/d25/d2e/l51 0 2024-04-24T22:45:36.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/771: mknod db/d1d/d20/d3f/d7f/cfe 0 2024-04-24T22:45:36.250 INFO:tasks.workunit.client.1.smithi192.stdout:4/532: symlink d2/db/d75/da1/d36/d5b/lb5 0 2024-04-24T22:45:36.252 INFO:tasks.workunit.client.0.smithi063.stdout:8/755: unlink d1/d11/d32/d38/c45 0 2024-04-24T22:45:36.252 INFO:tasks.workunit.client.0.smithi063.stdout:7/665: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:36.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/533: rmdir d2/d9/d6b 39 2024-04-24T22:45:36.256 INFO:tasks.workunit.client.0.smithi063.stdout:6/772: unlink db/d1d/d20/d3f/l5b 0 2024-04-24T22:45:36.266 INFO:tasks.workunit.client.0.smithi063.stdout:3/740: getdents da 0 2024-04-24T22:45:36.266 INFO:tasks.workunit.client.0.smithi063.stdout:7/666: write d0/d4/de/d17/d35/d6c/d7a/fa5 [1166658,29988] 0 2024-04-24T22:45:36.269 INFO:tasks.workunit.client.0.smithi063.stdout:5/699: dwrite d5/d24/d49/d3b/f58 [0,4194304] 0 2024-04-24T22:45:36.270 INFO:tasks.workunit.client.0.smithi063.stdout:6/773: dwrite db/d1d/d20/d29/fcc [0,4194304] 0 2024-04-24T22:45:36.271 INFO:tasks.workunit.client.1.smithi192.stdout:4/534: dwrite d2/db/d75/da1/f4a [0,4194304] 0 2024-04-24T22:45:36.271 INFO:tasks.workunit.client.0.smithi063.stdout:6/774: fsync db/d1d/d20/d3e/f43 0 2024-04-24T22:45:36.272 INFO:tasks.workunit.client.1.smithi192.stdout:4/535: write d2/f17 [348932,62561] 0 2024-04-24T22:45:36.273 INFO:tasks.workunit.client.0.smithi063.stdout:3/741: dread da/d18/d9e/fc4 [0,4194304] 0 2024-04-24T22:45:36.276 INFO:tasks.workunit.client.1.smithi192.stdout:4/536: creat d2/db/d10/fb6 x:0 0 0 2024-04-24T22:45:36.285 INFO:tasks.workunit.client.0.smithi063.stdout:3/742: dread da/d18/d3a/fb7 [0,4194304] 0 2024-04-24T22:45:36.286 INFO:tasks.workunit.client.1.smithi192.stdout:4/537: dwrite d2/db/d75/da1/d36/d5b/d64/f91 [0,4194304] 0 2024-04-24T22:45:36.289 INFO:tasks.workunit.client.0.smithi063.stdout:1/781: sync 2024-04-24T22:45:36.289 INFO:tasks.workunit.client.0.smithi063.stdout:4/691: sync 2024-04-24T22:45:36.290 INFO:tasks.workunit.client.1.smithi192.stdout:4/538: rmdir d2/db/d75/da1/d36/d5b/d64 39 2024-04-24T22:45:36.290 INFO:tasks.workunit.client.0.smithi063.stdout:4/692: fdatasync d2/f74 0 2024-04-24T22:45:36.296 INFO:tasks.workunit.client.1.smithi192.stdout:4/539: rename d2/db/d75/da1/lb2 to d2/db/d75/da1/d36/d5b/d28/d2f/d7e/lb7 0 2024-04-24T22:45:36.298 INFO:tasks.workunit.client.0.smithi063.stdout:6/775: mknod db/d1c/dc2/cff 0 2024-04-24T22:45:36.298 INFO:tasks.workunit.client.1.smithi192.stdout:0/570: sync 2024-04-24T22:45:36.298 INFO:tasks.workunit.client.1.smithi192.stdout:4/540: write d2/db/d75/da1/d36/d5b/f6e [2660202,32637] 0 2024-04-24T22:45:36.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/743: symlink da/l103 0 2024-04-24T22:45:36.301 INFO:tasks.workunit.client.0.smithi063.stdout:8/756: getdents d1/d22/d33/da6/d54/d7b 0 2024-04-24T22:45:36.304 INFO:tasks.workunit.client.1.smithi192.stdout:4/541: dread d2/db/d75/da1/d36/d5b/d28/d2f/fa2 [0,4194304] 0 2024-04-24T22:45:36.310 INFO:tasks.workunit.client.1.smithi192.stdout:4/542: rmdir d2/db/d75/da1/d36/d5b/d64 39 2024-04-24T22:45:36.314 INFO:tasks.workunit.client.0.smithi063.stdout:8/757: rename d1/d2/d17/f4b to d1/d11/d32/ff7 0 2024-04-24T22:45:36.314 INFO:tasks.workunit.client.0.smithi063.stdout:5/700: write d5/f80 [210996,31877] 0 2024-04-24T22:45:36.315 INFO:tasks.workunit.client.1.smithi192.stdout:4/543: unlink d2/db/f4e 0 2024-04-24T22:45:36.316 INFO:tasks.workunit.client.0.smithi063.stdout:1/782: creat d3/da/da2/dba/df5/f103 x:0 0 0 2024-04-24T22:45:36.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/783: chown d3/db5/d21/c9b 985002 1 2024-04-24T22:45:36.318 INFO:tasks.workunit.client.0.smithi063.stdout:1/784: read - d3/db5/d21/d9f/dd5/f102 zero size 2024-04-24T22:45:36.318 INFO:tasks.workunit.client.0.smithi063.stdout:4/693: truncate d2/d19/fa2 458803 0 2024-04-24T22:45:36.320 INFO:tasks.workunit.client.0.smithi063.stdout:6/776: dread f3 [0,4194304] 0 2024-04-24T22:45:36.323 INFO:tasks.workunit.client.1.smithi192.stdout:0/571: write d3/d40/f50 [4195573,39246] 0 2024-04-24T22:45:36.323 INFO:tasks.workunit.client.0.smithi063.stdout:8/758: symlink d1/d11/d32/dd3/lf8 0 2024-04-24T22:45:36.325 INFO:tasks.workunit.client.0.smithi063.stdout:1/785: dwrite d3/db5/d21/d9f/dd5/f102 [0,4194304] 0 2024-04-24T22:45:36.325 INFO:tasks.workunit.client.0.smithi063.stdout:4/694: symlink d2/d19/d82/le0 0 2024-04-24T22:45:36.326 INFO:tasks.workunit.client.0.smithi063.stdout:1/786: read - d3/ff2 zero size 2024-04-24T22:45:36.328 INFO:tasks.workunit.client.0.smithi063.stdout:6/777: write db/d1d/d20/fa1 [4500800,90338] 0 2024-04-24T22:45:36.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/572: link d3/d27/d28/d2e/d3f/lb2 d3/d27/d28/d2e/d3f/lc5 0 2024-04-24T22:45:36.338 INFO:tasks.workunit.client.0.smithi063.stdout:8/759: write d1/d21/f3a [4360875,31281] 0 2024-04-24T22:45:36.339 INFO:tasks.workunit.client.0.smithi063.stdout:6/778: mkdir db/d1d/d20/d84/d100 0 2024-04-24T22:45:36.339 INFO:tasks.workunit.client.0.smithi063.stdout:5/701: dread d5/d24/d44/f46 [0,4194304] 0 2024-04-24T22:45:36.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/544: dwrite d2/db/d75/da1/d36/d5b/d28/f2b [0,4194304] 0 2024-04-24T22:45:36.341 INFO:tasks.workunit.client.0.smithi063.stdout:5/702: chown d5/d11/d23/d2d/d62/d81/cb8 23 1 2024-04-24T22:45:36.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/545: truncate d2/d9/d25/f8e 812777 0 2024-04-24T22:45:36.347 INFO:tasks.workunit.client.1.smithi192.stdout:4/546: mknod d2/db/d75/cb8 0 2024-04-24T22:45:36.349 INFO:tasks.workunit.client.0.smithi063.stdout:9/740: sync 2024-04-24T22:45:36.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/667: sync 2024-04-24T22:45:36.349 INFO:tasks.workunit.client.0.smithi063.stdout:2/818: sync 2024-04-24T22:45:36.349 INFO:tasks.workunit.client.0.smithi063.stdout:3/744: write da/d18/f8d [5047318,83816] 0 2024-04-24T22:45:36.350 INFO:tasks.workunit.client.1.smithi192.stdout:4/547: write d2/d9/f3f [5081444,121546] 0 2024-04-24T22:45:36.351 INFO:tasks.workunit.client.0.smithi063.stdout:4/695: dread d2/f99 [0,4194304] 0 2024-04-24T22:45:36.352 INFO:tasks.workunit.client.0.smithi063.stdout:0/668: symlink d6/d64/d21/d80/dde/le6 0 2024-04-24T22:45:36.352 INFO:tasks.workunit.client.0.smithi063.stdout:5/703: write d5/d24/d44/f5c [1171351,32518] 0 2024-04-24T22:45:36.353 INFO:tasks.workunit.client.1.smithi192.stdout:4/548: creat d2/da5/fb9 x:0 0 0 2024-04-24T22:45:36.356 INFO:tasks.workunit.client.1.smithi192.stdout:4/549: getdents d2/db/d75/da1/d36 0 2024-04-24T22:45:36.358 INFO:tasks.workunit.client.1.smithi192.stdout:4/550: mknod d2/db/d5d/db3/cba 0 2024-04-24T22:45:36.362 INFO:tasks.workunit.client.1.smithi192.stdout:4/551: dread d2/d9/f3f [4194304,4194304] 0 2024-04-24T22:45:36.362 INFO:tasks.workunit.client.1.smithi192.stdout:4/552: fdatasync d2/db/d10/f96 0 2024-04-24T22:45:36.365 INFO:tasks.workunit.client.1.smithi192.stdout:4/553: mkdir d2/d5c/dbb 0 2024-04-24T22:45:36.368 INFO:tasks.workunit.client.1.smithi192.stdout:4/554: getdents d2/db/d75/da1/d36/d5b/d28/d2f 0 2024-04-24T22:45:36.374 INFO:tasks.workunit.client.0.smithi063.stdout:1/787: dwrite d3/da/d34/f83 [0,4194304] 0 2024-04-24T22:45:36.375 INFO:tasks.workunit.client.0.smithi063.stdout:5/704: mknod d5/d24/d49/cef 0 2024-04-24T22:45:36.379 INFO:tasks.workunit.client.0.smithi063.stdout:0/669: creat d6/d64/da0/d98/dd6/fe7 x:0 0 0 2024-04-24T22:45:36.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/555: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/fa2 [0,4194304] 0 2024-04-24T22:45:36.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/745: getdents da/d18/d3a 0 2024-04-24T22:45:36.380 INFO:tasks.workunit.client.0.smithi063.stdout:8/760: dread d1/d11/d32/d38/fa0 [0,4194304] 0 2024-04-24T22:45:36.383 INFO:tasks.workunit.client.0.smithi063.stdout:5/705: chown d5/d1d/dc7/fd3 1888916 1 2024-04-24T22:45:36.384 INFO:tasks.workunit.client.0.smithi063.stdout:8/761: read d1/d22/d33/fc4 [93365,60447] 0 2024-04-24T22:45:36.385 INFO:tasks.workunit.client.0.smithi063.stdout:3/746: chown da/f2e 34 1 2024-04-24T22:45:36.389 INFO:tasks.workunit.client.0.smithi063.stdout:6/779: dread db/d1d/d20/d3e/f74 [0,4194304] 0 2024-04-24T22:45:36.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/819: truncate d0/d2/d2d/d31/d89/d7/d86/d8e/fbe 510139 0 2024-04-24T22:45:36.391 INFO:tasks.workunit.client.0.smithi063.stdout:5/706: rename d5/d11/d23/f27 to d5/d11/d76/d92/dea/ff0 0 2024-04-24T22:45:36.391 INFO:tasks.workunit.client.1.smithi192.stdout:4/556: link d2/db/d75/da1/d36/d5b/lb5 d2/db/d75/da1/d53/lbc 0 2024-04-24T22:45:36.393 INFO:tasks.workunit.client.0.smithi063.stdout:2/820: creat d0/d2/d39/ffd x:0 0 0 2024-04-24T22:45:36.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/557: mkdir d2/db/d75/da1/d7d/dbd 0 2024-04-24T22:45:36.396 INFO:tasks.workunit.client.0.smithi063.stdout:6/780: mknod db/d1d/d20/d3f/d5f/d93/c101 0 2024-04-24T22:45:36.397 INFO:tasks.workunit.client.0.smithi063.stdout:4/696: dread d2/d8/d5c/f67 [0,4194304] 0 2024-04-24T22:45:36.398 INFO:tasks.workunit.client.0.smithi063.stdout:6/781: write db/d1d/d20/d84/fae [2549589,80386] 0 2024-04-24T22:45:36.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/558: fsync d2/d9/d25/f8e 0 2024-04-24T22:45:36.400 INFO:tasks.workunit.client.0.smithi063.stdout:5/707: dread d5/d11/d76/f9a [0,4194304] 0 2024-04-24T22:45:36.401 INFO:tasks.workunit.client.0.smithi063.stdout:2/821: mkdir d0/dfe 0 2024-04-24T22:45:36.402 INFO:tasks.workunit.client.0.smithi063.stdout:4/697: creat d2/d80/d81/d90/fe1 x:0 0 0 2024-04-24T22:45:36.402 INFO:tasks.workunit.client.0.smithi063.stdout:2/822: write d0/d2/d27/f85 [1509327,118914] 0 2024-04-24T22:45:36.403 INFO:tasks.workunit.client.0.smithi063.stdout:1/788: dread d3/da/d34/d68/fa8 [0,4194304] 0 2024-04-24T22:45:36.405 INFO:tasks.workunit.client.1.smithi192.stdout:4/559: write d2/db/d10/f11 [4786992,45790] 0 2024-04-24T22:45:36.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/747: write da/d18/d47/f85 [2874396,10457] 0 2024-04-24T22:45:36.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/698: rename d2/d8/d5c/f67 to d2/d8/d5c/fe2 0 2024-04-24T22:45:36.408 INFO:tasks.workunit.client.0.smithi063.stdout:0/670: dread f5 [0,4194304] 0 2024-04-24T22:45:36.410 INFO:tasks.workunit.client.0.smithi063.stdout:6/782: symlink db/d1c/dea/l102 0 2024-04-24T22:45:36.412 INFO:tasks.workunit.client.0.smithi063.stdout:5/708: symlink d5/lf1 0 2024-04-24T22:45:36.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/762: truncate d1/d22/d33/da6/d54/d79/ff4 1331236 0 2024-04-24T22:45:36.413 INFO:tasks.workunit.client.1.smithi192.stdout:4/560: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 [0,4194304] 0 2024-04-24T22:45:36.414 INFO:tasks.workunit.client.0.smithi063.stdout:6/783: dread db/d1d/d20/d3f/d7f/d68/f6d [0,4194304] 0 2024-04-24T22:45:36.415 INFO:tasks.workunit.client.0.smithi063.stdout:7/667: sync 2024-04-24T22:45:36.416 INFO:tasks.workunit.client.0.smithi063.stdout:2/823: creat d0/d2/d2d/d31/d89/d1c/d3f/fff x:0 0 0 2024-04-24T22:45:36.418 INFO:tasks.workunit.client.1.smithi192.stdout:4/561: unlink d2/db/d75/da1/d36/d5b/d28/d2f/f3c 0 2024-04-24T22:45:36.420 INFO:tasks.workunit.client.0.smithi063.stdout:4/699: mknod d2/d8/d40/d51/dd5/ce3 0 2024-04-24T22:45:36.420 INFO:tasks.workunit.client.0.smithi063.stdout:1/789: dread d3/da/f6a [0,4194304] 0 2024-04-24T22:45:36.421 INFO:tasks.workunit.client.1.smithi192.stdout:4/562: mknod d2/db/d75/da1/d36/d5b/d28/d9f/cbe 0 2024-04-24T22:45:36.426 INFO:tasks.workunit.client.0.smithi063.stdout:7/668: readlink d0/d4/de/d6f/d2e/l66 0 2024-04-24T22:45:36.427 INFO:tasks.workunit.client.0.smithi063.stdout:7/669: stat d0/d4/de/d17/d28/l4d 0 2024-04-24T22:45:36.431 INFO:tasks.workunit.client.0.smithi063.stdout:6/784: truncate db/d1d/d20/d3e/f45 761499 0 2024-04-24T22:45:36.431 INFO:tasks.workunit.client.0.smithi063.stdout:0/671: getdents d6/d64/d24/dd1/dd5 0 2024-04-24T22:45:36.432 INFO:tasks.workunit.client.0.smithi063.stdout:0/672: chown d6/d64/d21/d25/f55 219514 1 2024-04-24T22:45:36.433 INFO:tasks.workunit.client.0.smithi063.stdout:4/700: rename d2/l42 to d2/d8/d40/d51/dd5/le4 0 2024-04-24T22:45:36.438 INFO:tasks.workunit.client.0.smithi063.stdout:5/709: write d5/d24/d49/f84 [4973406,99676] 0 2024-04-24T22:45:36.438 INFO:tasks.workunit.client.0.smithi063.stdout:7/670: rmdir d0/d4/de/d17/dd7 39 2024-04-24T22:45:36.439 INFO:tasks.workunit.client.1.smithi192.stdout:4/563: write d2/db/d75/da1/d36/d5b/f24 [467135,105181] 0 2024-04-24T22:45:36.440 INFO:tasks.workunit.client.0.smithi063.stdout:2/824: mkdir d0/d2/d27/d100 0 2024-04-24T22:45:36.441 INFO:tasks.workunit.client.0.smithi063.stdout:6/785: fsync db/d1d/d20/f71 0 2024-04-24T22:45:36.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/790: symlink d3/da/d34/l104 0 2024-04-24T22:45:36.442 INFO:tasks.workunit.client.1.smithi192.stdout:4/564: readlink d2/db/d75/da1/d53/lbc 0 2024-04-24T22:45:36.442 INFO:tasks.workunit.client.1.smithi192.stdout:4/565: stat d2/d9/d25/d2e 0 2024-04-24T22:45:36.443 INFO:tasks.workunit.client.0.smithi063.stdout:4/701: creat d2/d8/dd4/dda/dc6/fe5 x:0 0 0 2024-04-24T22:45:36.443 INFO:tasks.workunit.client.0.smithi063.stdout:8/763: link d1/d11/d32/l7a d1/d22/d33/da6/d54/d79/lf9 0 2024-04-24T22:45:36.444 INFO:tasks.workunit.client.0.smithi063.stdout:3/748: getdents da/d18/dfb 0 2024-04-24T22:45:36.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/566: link d2/f1f d2/d5c/fbf 0 2024-04-24T22:45:36.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/710: rename d5/d11/d76/f7b to d5/d24/d49/d3b/dc5/dda/ff2 0 2024-04-24T22:45:36.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/711: write d5/d24/f51 [394477,73337] 0 2024-04-24T22:45:36.448 INFO:tasks.workunit.client.0.smithi063.stdout:2/825: rmdir d0/d2/d2d/d31/d89/d48 39 2024-04-24T22:45:36.452 INFO:tasks.workunit.client.0.smithi063.stdout:4/702: fsync d2/d19/d33/dc1/f7f 0 2024-04-24T22:45:36.454 INFO:tasks.workunit.client.1.smithi192.stdout:4/567: rename d2/d9/d25/d2e/l8f to d2/d9/d6b/lc0 0 2024-04-24T22:45:36.455 INFO:tasks.workunit.client.1.smithi192.stdout:4/568: chown d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 0 1 2024-04-24T22:45:36.456 INFO:tasks.workunit.client.0.smithi063.stdout:2/826: rmdir d0/d2/d39 39 2024-04-24T22:45:36.458 INFO:tasks.workunit.client.0.smithi063.stdout:9/741: sync 2024-04-24T22:45:36.459 INFO:tasks.workunit.client.0.smithi063.stdout:0/673: creat d6/d64/d21/d25/d62/d63/d89/d95/fe8 x:0 0 0 2024-04-24T22:45:36.460 INFO:tasks.workunit.client.0.smithi063.stdout:0/674: write d6/d64/d21/f36 [4045617,3486] 0 2024-04-24T22:45:36.461 INFO:tasks.workunit.client.0.smithi063.stdout:0/675: chown d6/l52 863 1 2024-04-24T22:45:36.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/569: rename d2/db/d75/da1/c41 to d2/d5c/cc1 0 2024-04-24T22:45:36.462 INFO:tasks.workunit.client.1.smithi192.stdout:4/570: stat d2/db/d10/l3e 0 2024-04-24T22:45:36.464 INFO:tasks.workunit.client.0.smithi063.stdout:7/671: dread d0/d4/de/f25 [0,4194304] 0 2024-04-24T22:45:36.465 INFO:tasks.workunit.client.0.smithi063.stdout:9/742: dread d6/d2f/dcd/d1f/d20/d2a/dcb/f33 [4194304,4194304] 0 2024-04-24T22:45:36.473 INFO:tasks.workunit.client.0.smithi063.stdout:0/676: fdatasync d6/da/f14 0 2024-04-24T22:45:36.476 INFO:tasks.workunit.client.0.smithi063.stdout:7/672: write d0/d4/de/d6f/d5d/f80 [1062780,103048] 0 2024-04-24T22:45:36.476 INFO:tasks.workunit.client.0.smithi063.stdout:8/764: rename d1/d21/f4f to d1/d22/d33/da6/d54/ffa 0 2024-04-24T22:45:36.479 INFO:tasks.workunit.client.0.smithi063.stdout:3/749: write da/f54 [25515,6590] 0 2024-04-24T22:45:36.483 INFO:tasks.workunit.client.0.smithi063.stdout:6/786: getdents db/d1d/d2d/d69/dd4/de4 0 2024-04-24T22:45:36.483 INFO:tasks.workunit.client.0.smithi063.stdout:9/743: dread d6/d2f/dcd/db/d93/fd8 [0,4194304] 0 2024-04-24T22:45:36.484 INFO:tasks.workunit.client.1.smithi192.stdout:4/571: dread d2/d9/d25/f68 [0,4194304] 0 2024-04-24T22:45:36.485 INFO:tasks.workunit.client.1.smithi192.stdout:4/572: chown d2/db/l4d 98 1 2024-04-24T22:45:36.487 INFO:tasks.workunit.client.0.smithi063.stdout:7/673: readlink d0/d4/de/d17/d35/d6c/d7a/lbf 0 2024-04-24T22:45:36.489 INFO:tasks.workunit.client.1.smithi192.stdout:4/573: rename d2/db/d75/da1/d36/d5b/d28/f9e to d2/db/d5d/db3/fc2 0 2024-04-24T22:45:36.489 INFO:tasks.workunit.client.0.smithi063.stdout:4/703: dwrite d2/f5 [0,4194304] 0 2024-04-24T22:45:36.491 INFO:tasks.workunit.client.0.smithi063.stdout:5/712: getdents d5/d24/d49/d3b/dc5 0 2024-04-24T22:45:36.491 INFO:tasks.workunit.client.0.smithi063.stdout:6/787: dwrite db/ff [0,4194304] 0 2024-04-24T22:45:36.493 INFO:tasks.workunit.client.0.smithi063.stdout:2/827: creat d0/f101 x:0 0 0 2024-04-24T22:45:36.494 INFO:tasks.workunit.client.0.smithi063.stdout:6/788: readlink db/d1d/d20/d3f/l72 0 2024-04-24T22:45:36.494 INFO:tasks.workunit.client.0.smithi063.stdout:3/750: creat da/d18/d3a/f104 x:0 0 0 2024-04-24T22:45:36.495 INFO:tasks.workunit.client.1.smithi192.stdout:4/574: dwrite d2/d9/f8a [0,4194304] 0 2024-04-24T22:45:36.495 INFO:tasks.workunit.client.0.smithi063.stdout:1/791: link d3/da/da2/de6/lfa d3/db5/d21/l105 0 2024-04-24T22:45:36.497 INFO:tasks.workunit.client.0.smithi063.stdout:3/751: dread - da/d18/d9e/da3/dc1/ff6 zero size 2024-04-24T22:45:36.497 INFO:tasks.workunit.client.0.smithi063.stdout:9/744: unlink d6/d2f/dcd/db/d93/da0/fae 0 2024-04-24T22:45:36.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/752: write da/f2e [1666954,25752] 0 2024-04-24T22:45:36.504 INFO:tasks.workunit.client.0.smithi063.stdout:4/704: unlink d2/d19/d33/dc1/lc9 0 2024-04-24T22:45:36.505 INFO:tasks.workunit.client.1.smithi192.stdout:6/508: sync 2024-04-24T22:45:36.505 INFO:tasks.workunit.client.0.smithi063.stdout:9/745: dwrite d6/d2f/f35 [4194304,4194304] 0 2024-04-24T22:45:36.506 INFO:tasks.workunit.client.1.smithi192.stdout:5/449: sync 2024-04-24T22:45:36.506 INFO:tasks.workunit.client.1.smithi192.stdout:1/484: sync 2024-04-24T22:45:36.506 INFO:tasks.workunit.client.1.smithi192.stdout:6/509: chown d0/d8/f6f 92 1 2024-04-24T22:45:36.507 INFO:tasks.workunit.client.1.smithi192.stdout:5/450: write d0/f8b [431873,41309] 0 2024-04-24T22:45:36.508 INFO:tasks.workunit.client.1.smithi192.stdout:4/575: rename d2/db/d75/da1/d36/d5b/d64/f6c to d2/db/d75/da1/d7d/dbd/fc3 0 2024-04-24T22:45:36.511 INFO:tasks.workunit.client.1.smithi192.stdout:1/485: mkdir d9/d15/d2d/d48/d93/da4 0 2024-04-24T22:45:36.513 INFO:tasks.workunit.client.1.smithi192.stdout:5/451: fdatasync d0/f4a 0 2024-04-24T22:45:36.514 INFO:tasks.workunit.client.0.smithi063.stdout:2/828: symlink d0/dd/d72/d77/dda/d7c/l102 0 2024-04-24T22:45:36.514 INFO:tasks.workunit.client.1.smithi192.stdout:5/452: truncate d0/d13/d53/f94 5123 0 2024-04-24T22:45:36.515 INFO:tasks.workunit.client.1.smithi192.stdout:4/576: creat d2/db/d75/da1/d7d/d9c/fc4 x:0 0 0 2024-04-24T22:45:36.520 INFO:tasks.workunit.client.1.smithi192.stdout:4/577: dread d2/db/d75/da1/f4c [0,4194304] 0 2024-04-24T22:45:36.523 INFO:tasks.workunit.client.1.smithi192.stdout:5/453: mkdir d0/d13/d19/d71/d7f/d9a 0 2024-04-24T22:45:36.528 INFO:tasks.workunit.client.1.smithi192.stdout:6/510: link d0/d8/d29/d4c/d4d/f56 d0/d8/d29/d4c/db0/fb3 0 2024-04-24T22:45:36.528 INFO:tasks.workunit.client.1.smithi192.stdout:4/578: dwrite d2/db/d75/da1/d36/d5b/d28/d9f/fa0 [0,4194304] 0 2024-04-24T22:45:36.529 INFO:tasks.workunit.client.0.smithi063.stdout:4/705: creat d2/d8/d40/fe6 x:0 0 0 2024-04-24T22:45:36.530 INFO:tasks.workunit.client.0.smithi063.stdout:4/706: fdatasync d2/d80/f8f 0 2024-04-24T22:45:36.534 INFO:tasks.workunit.client.1.smithi192.stdout:4/579: dread d2/db/d75/da1/d36/d5b/d28/d2f/f34 [0,4194304] 0 2024-04-24T22:45:36.534 INFO:tasks.workunit.client.0.smithi063.stdout:8/765: dwrite d1/d22/d33/da6/f7c [0,4194304] 0 2024-04-24T22:45:36.535 INFO:tasks.workunit.client.0.smithi063.stdout:0/677: getdents d6/d64/d24/d42/dd2 0 2024-04-24T22:45:36.535 INFO:tasks.workunit.client.1.smithi192.stdout:5/454: dread - d0/d13/d19/d64/d4f/d52/f6a zero size 2024-04-24T22:45:36.536 INFO:tasks.workunit.client.1.smithi192.stdout:4/580: truncate d2/db/d75/da1/d7d/f9b 375893 0 2024-04-24T22:45:36.538 INFO:tasks.workunit.client.0.smithi063.stdout:1/792: fsync d3/db5/d21/d49/fe2 0 2024-04-24T22:45:36.540 INFO:tasks.workunit.client.0.smithi063.stdout:4/707: dwrite d2/d19/d33/dc1/d50/fcd [0,4194304] 0 2024-04-24T22:45:36.544 INFO:tasks.workunit.client.0.smithi063.stdout:4/708: truncate d2/d8/d3c/f62 1199405 0 2024-04-24T22:45:36.544 INFO:tasks.workunit.client.1.smithi192.stdout:4/581: dwrite d2/db/f7a [0,4194304] 0 2024-04-24T22:45:36.546 INFO:tasks.workunit.client.1.smithi192.stdout:0/573: sync 2024-04-24T22:45:36.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/677: sync 2024-04-24T22:45:36.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/633: sync 2024-04-24T22:45:36.546 INFO:tasks.workunit.client.1.smithi192.stdout:8/570: sync 2024-04-24T22:45:36.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/615: sync 2024-04-24T22:45:36.547 INFO:tasks.workunit.client.1.smithi192.stdout:2/704: sync 2024-04-24T22:45:36.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/709: write d2/d19/d33/d9d/fbf [784502,105328] 0 2024-04-24T22:45:36.548 INFO:tasks.workunit.client.0.smithi063.stdout:3/753: mknod da/d39/d94/db3/dbe/c105 0 2024-04-24T22:45:36.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/754: write da/f54 [213905,104971] 0 2024-04-24T22:45:36.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/755: fdatasync da/d18/d47/d4d/d62/f80 0 2024-04-24T22:45:36.552 INFO:tasks.workunit.client.1.smithi192.stdout:8/571: write db/dc/d29/f2b [7151407,2527] 0 2024-04-24T22:45:36.553 INFO:tasks.workunit.client.0.smithi063.stdout:5/713: creat d5/d24/d49/ff3 x:0 0 0 2024-04-24T22:45:36.557 INFO:tasks.workunit.client.1.smithi192.stdout:8/572: read f9 [1336757,101959] 0 2024-04-24T22:45:36.558 INFO:tasks.workunit.client.1.smithi192.stdout:1/486: dwrite d9/d15/d2d/d31/d2f/f3a [4194304,4194304] 0 2024-04-24T22:45:36.560 INFO:tasks.workunit.client.0.smithi063.stdout:5/714: dwrite d5/d11/d23/d2d/fd1 [4194304,4194304] 0 2024-04-24T22:45:36.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/487: chown d9/dd/ca3 2495390 1 2024-04-24T22:45:36.562 INFO:tasks.workunit.client.1.smithi192.stdout:3/634: rename d2/d4/d7 to d2/d27/dc6 0 2024-04-24T22:45:36.567 INFO:tasks.workunit.client.1.smithi192.stdout:6/511: write d0/d8/f76 [660185,73180] 0 2024-04-24T22:45:36.568 INFO:tasks.workunit.client.1.smithi192.stdout:0/574: mknod d3/d27/d28/d2e/d3f/cc6 0 2024-04-24T22:45:36.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/575: chown d3/d27/d28/f8e 828 1 2024-04-24T22:45:36.570 INFO:tasks.workunit.client.0.smithi063.stdout:0/678: symlink d6/d64/d21/d80/le9 0 2024-04-24T22:45:36.570 INFO:tasks.workunit.client.1.smithi192.stdout:0/576: truncate d3/f48 1552253 0 2024-04-24T22:45:36.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/678: mknod d3/dd/de/d17/d89/db4/db9/cf1 0 2024-04-24T22:45:36.574 INFO:tasks.workunit.client.1.smithi192.stdout:5/455: dwrite d0/f4a [0,4194304] 0 2024-04-24T22:45:36.574 INFO:tasks.workunit.client.0.smithi063.stdout:7/674: sync 2024-04-24T22:45:36.575 INFO:tasks.workunit.client.0.smithi063.stdout:6/789: link db/l1f db/d1d/d2d/d48/dfd/l103 0 2024-04-24T22:45:36.576 INFO:tasks.workunit.client.0.smithi063.stdout:2/829: dwrite d0/d2/d2f/f6d [0,4194304] 0 2024-04-24T22:45:36.577 INFO:tasks.workunit.client.0.smithi063.stdout:7/675: fdatasync d0/d4/de/d6f/d5d/f80 0 2024-04-24T22:45:36.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/616: symlink d2/lc0 0 2024-04-24T22:45:36.581 INFO:tasks.workunit.client.1.smithi192.stdout:6/512: fdatasync d0/d8/f6f 0 2024-04-24T22:45:36.582 INFO:tasks.workunit.client.1.smithi192.stdout:9/679: dwrite d3/f31 [0,4194304] 0 2024-04-24T22:45:36.584 INFO:tasks.workunit.client.0.smithi063.stdout:2/830: dwrite d0/dd/d72/d77/dda/d7c/f61 [0,4194304] 0 2024-04-24T22:45:36.585 INFO:tasks.workunit.client.1.smithi192.stdout:3/635: rename d2/d4/d12/d1a/d1d/d55/f32 to d2/d4/d12/d1a/d1d/d55/fc7 0 2024-04-24T22:45:36.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/766: unlink d1/d22/d33/da6/d54/d79/ff4 0 2024-04-24T22:45:36.595 INFO:tasks.workunit.client.0.smithi063.stdout:5/715: symlink d5/d1d/dc7/lf4 0 2024-04-24T22:45:36.595 INFO:tasks.workunit.client.0.smithi063.stdout:3/756: dwrite da/d48/fd7 [0,4194304] 0 2024-04-24T22:45:36.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/456: truncate d0/d13/d19/d64/d5f/d2c/f6f 83328 0 2024-04-24T22:45:36.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/767: write d1/d22/d33/fee [690187,55234] 0 2024-04-24T22:45:36.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/716: chown d5/d24/d49 188930668 1 2024-04-24T22:45:36.600 INFO:tasks.workunit.client.0.smithi063.stdout:0/679: unlink d6/da/dc/c1d 0 2024-04-24T22:45:36.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/705: rename d4/d9/d10/c17 to d4/d9/d10/d6c/d95/db7/ce3 0 2024-04-24T22:45:36.602 INFO:tasks.workunit.client.0.smithi063.stdout:1/793: truncate d3/de0/d79/ff0 3711733 0 2024-04-24T22:45:36.605 INFO:tasks.workunit.client.1.smithi192.stdout:7/617: mkdir d2/db/d21/d42/d67/d9b/dc1 0 2024-04-24T22:45:36.608 INFO:tasks.workunit.client.0.smithi063.stdout:6/790: creat db/d85/db5/dc4/f104 x:0 0 0 2024-04-24T22:45:36.608 INFO:tasks.workunit.client.0.smithi063.stdout:6/791: fdatasync db/fc6 0 2024-04-24T22:45:36.609 INFO:tasks.workunit.client.0.smithi063.stdout:7/676: dread d0/d4/de/f12 [0,4194304] 0 2024-04-24T22:45:36.611 INFO:tasks.workunit.client.1.smithi192.stdout:6/513: creat d0/d8/d22/fb4 x:0 0 0 2024-04-24T22:45:36.613 INFO:tasks.workunit.client.1.smithi192.stdout:0/577: mkdir d3/d27/d28/d2e/dc7 0 2024-04-24T22:45:36.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/746: link d6/d2f/dcd/db/d6b/d4f/d59/lf3 d6/d2f/dcd/db/d1c/lf4 0 2024-04-24T22:45:36.616 INFO:tasks.workunit.client.1.smithi192.stdout:3/636: chown d2/d4/d12/d1a/d1d/d55/cb 36418 1 2024-04-24T22:45:36.617 INFO:tasks.workunit.client.1.smithi192.stdout:8/573: creat db/dc/fb0 x:0 0 0 2024-04-24T22:45:36.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/757: creat da/d18/d47/d4d/f106 x:0 0 0 2024-04-24T22:45:36.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/768: write d1/d2/d3/f46 [3156462,47812] 0 2024-04-24T22:45:36.620 INFO:tasks.workunit.client.0.smithi063.stdout:5/717: rmdir d5/d11/d23 39 2024-04-24T22:45:36.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/514: dwrite d0/d8/d29/d58/f5b [0,4194304] 0 2024-04-24T22:45:36.620 INFO:tasks.workunit.client.1.smithi192.stdout:5/457: chown d0/d1b/f25 597 1 2024-04-24T22:45:36.620 INFO:tasks.workunit.client.1.smithi192.stdout:2/706: truncate d4/f31 1213538 0 2024-04-24T22:45:36.621 INFO:tasks.workunit.client.0.smithi063.stdout:0/680: mkdir d6/d64/d21/d80/dea 0 2024-04-24T22:45:36.624 INFO:tasks.workunit.client.0.smithi063.stdout:1/794: fdatasync d3/da/da2/dba/d11/d5e/fa6 0 2024-04-24T22:45:36.624 INFO:tasks.workunit.client.1.smithi192.stdout:7/618: dread d2/f1a [4194304,4194304] 0 2024-04-24T22:45:36.626 INFO:tasks.workunit.client.1.smithi192.stdout:9/680: mkdir d3/dd/df2 0 2024-04-24T22:45:36.629 INFO:tasks.workunit.client.1.smithi192.stdout:0/578: dread d3/f55 [0,4194304] 0 2024-04-24T22:45:36.633 INFO:tasks.workunit.client.1.smithi192.stdout:8/574: creat db/dc/d22/d31/d33/d81/fb1 x:0 0 0 2024-04-24T22:45:36.634 INFO:tasks.workunit.client.0.smithi063.stdout:4/710: link d2/d19/d33/dc1/d50/f5d d2/d19/d33/dc1/d50/fe7 0 2024-04-24T22:45:36.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/711: write d2/d8/d40/f68 [1643687,40676] 0 2024-04-24T22:45:36.636 INFO:tasks.workunit.client.1.smithi192.stdout:0/579: dwrite d3/d27/f43 [0,4194304] 0 2024-04-24T22:45:36.638 INFO:tasks.workunit.client.1.smithi192.stdout:8/575: write db/d43/f69 [904573,22988] 0 2024-04-24T22:45:36.640 INFO:tasks.workunit.client.1.smithi192.stdout:1/488: dwrite d9/f1a [0,4194304] 0 2024-04-24T22:45:36.642 INFO:tasks.workunit.client.1.smithi192.stdout:2/707: rename d4/d9/d10/d6c/d95/caa to d4/d9/d10/d6c/d8c/ce4 0 2024-04-24T22:45:36.650 INFO:tasks.workunit.client.0.smithi063.stdout:2/831: write d0/d2/d2d/d31/d89/d7/f9d [628346,32384] 0 2024-04-24T22:45:36.652 INFO:tasks.workunit.client.1.smithi192.stdout:5/458: getdents d0/d1b/d62 0 2024-04-24T22:45:36.657 INFO:tasks.workunit.client.1.smithi192.stdout:6/515: creat d0/d8/d29/d4c/db0/fb5 x:0 0 0 2024-04-24T22:45:36.660 INFO:tasks.workunit.client.1.smithi192.stdout:8/576: write db/d5d/f84 [205261,89752] 0 2024-04-24T22:45:36.661 INFO:tasks.workunit.client.0.smithi063.stdout:9/747: creat d6/d2f/dcd/d1f/ddb/ff5 x:0 0 0 2024-04-24T22:45:36.662 INFO:tasks.workunit.client.1.smithi192.stdout:1/489: rename d9/d15/c50 to d9/d15/d2d/d48/d93/ca5 0 2024-04-24T22:45:36.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/708: fsync d4/d9/d10/d6c/d95/db7/fc3 0 2024-04-24T22:45:36.664 INFO:tasks.workunit.client.0.smithi063.stdout:6/792: dwrite db/d1d/d20/f50 [0,4194304] 0 2024-04-24T22:45:36.665 INFO:tasks.workunit.client.1.smithi192.stdout:3/637: dread d2/d4/d12/fb4 [0,4194304] 0 2024-04-24T22:45:36.669 INFO:tasks.workunit.client.0.smithi063.stdout:0/681: mkdir d6/d64/da0/d33/deb 0 2024-04-24T22:45:36.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/681: write d3/dd/de/d17/d89/d8b/f82 [342160,47965] 0 2024-04-24T22:45:36.672 INFO:tasks.workunit.client.0.smithi063.stdout:2/832: symlink d0/dd/d72/d77/dda/l103 0 2024-04-24T22:45:36.675 INFO:tasks.workunit.client.0.smithi063.stdout:7/677: creat d0/d4/fe1 x:0 0 0 2024-04-24T22:45:36.676 INFO:tasks.workunit.client.0.smithi063.stdout:7/678: dread - d0/d4/de/d17/d35/d69/db1/fdb zero size 2024-04-24T22:45:36.676 INFO:tasks.workunit.client.1.smithi192.stdout:8/577: creat db/dc/d22/d31/d33/d8f/da6/fb2 x:0 0 0 2024-04-24T22:45:36.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/712: fsync d2/f35 0 2024-04-24T22:45:36.678 INFO:tasks.workunit.client.1.smithi192.stdout:8/578: write db/dc/f5b [1822186,111871] 0 2024-04-24T22:45:36.679 INFO:tasks.workunit.client.0.smithi063.stdout:9/748: rmdir d6/d2f/d42 39 2024-04-24T22:45:36.682 INFO:tasks.workunit.client.1.smithi192.stdout:6/516: rename d0/d8/d29/d58/f5b to d0/d8/d29/d5a/fb6 0 2024-04-24T22:45:36.685 INFO:tasks.workunit.client.0.smithi063.stdout:6/793: mkdir db/d1d/d20/d29/dab/dd0/d105 0 2024-04-24T22:45:36.686 INFO:tasks.workunit.client.1.smithi192.stdout:2/709: truncate d4/d29/d60/f89 1143335 0 2024-04-24T22:45:36.686 INFO:tasks.workunit.client.0.smithi063.stdout:1/795: dread d3/f23 [0,4194304] 0 2024-04-24T22:45:36.687 INFO:tasks.workunit.client.0.smithi063.stdout:1/796: chown d3/db5/d21/d9f/daa/ld8 681890 1 2024-04-24T22:45:36.688 INFO:tasks.workunit.client.1.smithi192.stdout:7/619: link d2/db/d21/l9d d2/db/d21/d42/d67/d9b/d6e/db5/lc2 0 2024-04-24T22:45:36.689 INFO:tasks.workunit.client.1.smithi192.stdout:5/459: write d0/d13/d19/d64/d4f/d52/f7e [1451747,58454] 0 2024-04-24T22:45:36.690 INFO:tasks.workunit.client.1.smithi192.stdout:5/460: fsync d0/d13/d19/f1a 0 2024-04-24T22:45:36.690 INFO:tasks.workunit.client.0.smithi063.stdout:1/797: dread d3/da/d34/d6d/f92 [0,4194304] 0 2024-04-24T22:45:36.694 INFO:tasks.workunit.client.1.smithi192.stdout:9/682: creat d3/dd/d53/ff3 x:0 0 0 2024-04-24T22:45:36.696 INFO:tasks.workunit.client.1.smithi192.stdout:0/580: dwrite d3/d27/d6a/f87 [0,4194304] 0 2024-04-24T22:45:36.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/713: rmdir d2/d8/dd4/dda/dc6 39 2024-04-24T22:45:36.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/769: getdents d1/d11/d32/d38/db7 0 2024-04-24T22:45:36.703 INFO:tasks.workunit.client.1.smithi192.stdout:9/683: dwrite d3/f31 [4194304,4194304] 0 2024-04-24T22:45:36.705 INFO:tasks.workunit.client.1.smithi192.stdout:1/490: mknod d9/d51/ca6 0 2024-04-24T22:45:36.706 INFO:tasks.workunit.client.1.smithi192.stdout:1/491: read - d9/d15/d5e/d67/fa2 zero size 2024-04-24T22:45:36.709 INFO:tasks.workunit.client.0.smithi063.stdout:3/758: sync 2024-04-24T22:45:36.710 INFO:tasks.workunit.client.0.smithi063.stdout:5/718: link d5/d24/d44/fa5 d5/d11/d23/d2d/d62/d73/d95/ff5 0 2024-04-24T22:45:36.713 INFO:tasks.workunit.client.0.smithi063.stdout:1/798: rename d3/da/da2/dba to d3/db5/d21/d9f/dd5/d106 0 2024-04-24T22:45:36.714 INFO:tasks.workunit.client.0.smithi063.stdout:0/682: dread d6/d64/d21/d25/d62/d63/f82 [0,4194304] 0 2024-04-24T22:45:36.714 INFO:tasks.workunit.client.0.smithi063.stdout:2/833: truncate d0/fa6 1329075 0 2024-04-24T22:45:36.716 INFO:tasks.workunit.client.1.smithi192.stdout:2/710: rename d4/d9/d10/d6c/d95/dc0/dce to d4/d9/d37/de5 0 2024-04-24T22:45:36.717 INFO:tasks.workunit.client.1.smithi192.stdout:3/638: mknod d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/cc8 0 2024-04-24T22:45:36.718 INFO:tasks.workunit.client.1.smithi192.stdout:3/639: fdatasync d2/d27/dc6/f60 0 2024-04-24T22:45:36.719 INFO:tasks.workunit.client.0.smithi063.stdout:2/834: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:36.721 INFO:tasks.workunit.client.0.smithi063.stdout:4/714: unlink d2/d19/d33/dc1/f9b 0 2024-04-24T22:45:36.721 INFO:tasks.workunit.client.1.smithi192.stdout:5/461: rmdir d0/d13/d19/d71 39 2024-04-24T22:45:36.721 INFO:tasks.workunit.client.1.smithi192.stdout:8/579: write db/dc/f1b [1400539,96403] 0 2024-04-24T22:45:36.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/770: symlink d1/d11/d32/d38/db7/dc1/lfb 0 2024-04-24T22:45:36.727 INFO:tasks.workunit.client.0.smithi063.stdout:6/794: creat db/d1d/d20/d84/d100/f106 x:0 0 0 2024-04-24T22:45:36.728 INFO:tasks.workunit.client.0.smithi063.stdout:6/795: chown db/d1d/d20/d3e/fe1 48836 1 2024-04-24T22:45:36.730 INFO:tasks.workunit.client.1.smithi192.stdout:1/492: mkdir d9/d15/d2d/d63/d68/da7 0 2024-04-24T22:45:36.731 INFO:tasks.workunit.client.0.smithi063.stdout:7/679: write d0/d4/f19 [935976,97713] 0 2024-04-24T22:45:36.731 INFO:tasks.workunit.client.0.smithi063.stdout:5/719: creat d5/d24/db6/ff6 x:0 0 0 2024-04-24T22:45:36.734 INFO:tasks.workunit.client.0.smithi063.stdout:1/799: creat d3/de0/f107 x:0 0 0 2024-04-24T22:45:36.736 INFO:tasks.workunit.client.1.smithi192.stdout:9/684: rename d3/dd/d53/c8e to d3/dd/df2/cf4 0 2024-04-24T22:45:36.738 INFO:tasks.workunit.client.1.smithi192.stdout:2/711: creat d4/d29/d60/fe6 x:0 0 0 2024-04-24T22:45:36.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/835: creat d0/d2/d2f/dba/f104 x:0 0 0 2024-04-24T22:45:36.742 INFO:tasks.workunit.client.1.smithi192.stdout:9/685: dwrite d3/dd/f6c [0,4194304] 0 2024-04-24T22:45:36.743 INFO:tasks.workunit.client.0.smithi063.stdout:4/715: rename d2/d19/d33/dc1/f7f to d2/d19/d33/d9d/da8/fe8 0 2024-04-24T22:45:36.744 INFO:tasks.workunit.client.0.smithi063.stdout:4/716: chown d2/c7b 2 1 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:3/640: symlink d2/d4/d12/d1a/d1d/d55/d5f/dbd/lc9 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:3/641: dwrite d2/d27/dc6/f2c [0,4194304] 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/712: truncate d4/d9/d10/f5d 1418359 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:8/580: write db/d43/f4e [1595083,103344] 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:7/620: link d2/db/d21/d42/d5b/f6b d2/db/d21/d42/d67/d9b/dc1/fc3 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:7/621: chown d2/db/l1f 15 1 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:5/462: symlink d0/d90/l9b 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:1/493: symlink d9/la8 0 2024-04-24T22:45:36.912 INFO:tasks.workunit.client.1.smithi192.stdout:6/517: getdents d0/d8/d3d/d9d 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/581: link d3/d27/d28/f34 d3/d27/d28/d2e/d3f/fc8 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:2/713: write d4/f41 [5248224,12771] 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/582: stat d3/d6d/d79/f7a 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:3/642: unlink d2/d27/dc6/d4b/d54/ca6 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:3/643: fdatasync d2/d27/d28/dab/fc2 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/494: mkdir d9/d15/d5e/d90/da9 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:5/463: dwrite d0/d1b/f65 [0,4194304] 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/518: unlink d0/d8/d3d/d3e/d40/f47 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:9/686: creat d3/d59/ff5 x:0 0 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/583: rename d3/f14 to d3/d27/d6a/fc9 0 2024-04-24T22:45:36.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/519: read d0/d8/d29/f3b [771120,117759] 0 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:1/495: dwrite d9/dd/f41 [0,4194304] 0 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:3/644: mkdir d2/d27/d28/dab/dca 0 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:0/584: read d3/d40/f50 [353670,9325] 0 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:1/496: chown d9/d6f/f86 5653 1 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:3/645: read d2/d4/d12/d1a/d1d/d55/d95/f92 [106099,110355] 0 2024-04-24T22:45:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:3/646: chown d2/d27/dc6 69 1 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/680: creat d0/d4/de/d17/d35/d69/db1/fe2 x:0 0 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:3/759: getdents da/d18/d47/d4d/d9b/dde/df9 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/800: mkdir d3/db5/d21/d9f/daa/d108 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/749: rename d6/d2f/dcd/d53/dbe to d6/d2f/dcd/d53/df6 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/836: dread d0/d2/d39/fb7 [0,4194304] 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:5/720: symlink d5/d11/d76/lf7 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:3/760: write da/d18/d3a/f104 [880687,120531] 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/750: creat d6/d2f/dcd/d1f/d20/d2a/dcb/ff7 x:0 0 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/751: stat d6/d2f/dcd/db/d93/da0/fdf 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/717: mkdir d2/d8/d3c/de9 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:9/752: write d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/fe7 [118199,1523] 0 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:0/683: sync 2024-04-24T22:45:36.915 INFO:tasks.workunit.client.0.smithi063.stdout:8/771: sync 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/772: chown d1/d2/d3/l25 221239 1 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/773: readlink d1/d11/d3c/l40 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:2/837: rename d0/dd/d72/d77/dda/d7c/fd0 to d0/d2/d2d/d31/d89/d7/d86/d8e/f105 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:6/796: rmdir db/d1d/d2d/d48/dec 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:7/681: mknod d0/d4/de/d17/d35/d69/dc9/ce3 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:1/801: fdatasync d3/db5/fb2 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:3/761: write da/d18/d3a/d63/ffc [253109,112728] 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:9/753: creat d6/d2f/dcd/d1f/ddb/ff8 x:0 0 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/774: creat d1/d2/d3/db6/ffc x:0 0 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:6/797: truncate db/d1c/fbf 932450 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:0/684: dwrite d6/d64/d21/d25/d31/fdd [0,4194304] 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:3/762: write da/d18/d9e/da3/dc1/ff5 [3922139,31233] 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/718: mkdir d2/d8/dd4/dda/dc6/dea 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:2/838: symlink d0/d2/d2d/d31/d89/d7/d86/da0/deb/l106 0 2024-04-24T22:45:36.916 INFO:tasks.workunit.client.0.smithi063.stdout:2/839: chown d0/d2/d63 71 1 2024-04-24T22:45:36.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/719: dwrite d2/d80/f8a [4194304,4194304] 0 2024-04-24T22:45:36.917 INFO:tasks.workunit.client.0.smithi063.stdout:6/798: mknod db/d1d/d20/d3e/c107 0 2024-04-24T22:45:36.917 INFO:tasks.workunit.client.0.smithi063.stdout:1/802: symlink d3/db5/d21/d9f/dd5/d106/l109 0 2024-04-24T22:45:36.926 INFO:tasks.workunit.client.0.smithi063.stdout:2/840: rename d0/dd/d72/d77/dda/d7c/cae to d0/d2/d2d/d31/d89/d1c/d3f/c107 0 2024-04-24T22:45:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:36 smithi192 ceph-mon[33991]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 80 MiB/s rd, 226 MiB/s wr, 449 op/s 2024-04-24T22:45:36.934 INFO:tasks.workunit.client.0.smithi063.stdout:9/754: creat d6/d2f/dcd/db/d93/ff9 x:0 0 0 2024-04-24T22:45:36.935 INFO:tasks.workunit.client.0.smithi063.stdout:8/775: creat d1/d22/ffd x:0 0 0 2024-04-24T22:45:36.939 INFO:tasks.workunit.client.0.smithi063.stdout:2/841: dread - d0/dd/d72/d77/f78 zero size 2024-04-24T22:45:36.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/799: fsync db/f37 0 2024-04-24T22:45:36.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/720: dread d2/d8/f10 [0,4194304] 0 2024-04-24T22:45:36.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/685: write d6/d64/d21/f86 [2136228,27045] 0 2024-04-24T22:45:36.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/686: fdatasync d6/d64/d21/d9c/fbd 0 2024-04-24T22:45:36.946 INFO:tasks.workunit.client.0.smithi063.stdout:4/721: dread d2/d8/d36/fb6 [0,4194304] 0 2024-04-24T22:45:36.947 INFO:tasks.workunit.client.0.smithi063.stdout:2/842: dwrite d0/d2/d2f/ddc/ff3 [0,4194304] 0 2024-04-24T22:45:36.948 INFO:tasks.workunit.client.0.smithi063.stdout:2/843: readlink d0/dd/l66 0 2024-04-24T22:45:36.954 INFO:tasks.workunit.client.0.smithi063.stdout:6/800: symlink db/d1c/dc3/l108 0 2024-04-24T22:45:36.954 INFO:tasks.workunit.client.0.smithi063.stdout:2/844: dwrite d0/f101 [0,4194304] 0 2024-04-24T22:45:36.956 INFO:tasks.workunit.client.0.smithi063.stdout:2/845: truncate d0/d2/d2d/d31/d89/f80 543101 0 2024-04-24T22:45:36.958 INFO:tasks.workunit.client.0.smithi063.stdout:5/721: link d5/d24/d36/c4d d5/d11/d76/cf8 0 2024-04-24T22:45:36.959 INFO:tasks.workunit.client.0.smithi063.stdout:7/682: sync 2024-04-24T22:45:36.963 INFO:tasks.workunit.client.0.smithi063.stdout:1/803: write d3/f19 [1679696,88267] 0 2024-04-24T22:45:36.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/801: dwrite db/d1d/d20/d29/dab/dd0/fe3 [0,4194304] 0 2024-04-24T22:45:36.964 INFO:tasks.workunit.client.0.smithi063.stdout:3/763: getdents da/d18/d47/d4d/d9b/dde 0 2024-04-24T22:45:36.965 INFO:tasks.workunit.client.0.smithi063.stdout:1/804: fsync d3/da/d34/d6d/fb6 0 2024-04-24T22:45:36.967 INFO:tasks.workunit.client.1.smithi192.stdout:5/464: rename d0/f8e to d0/d13/d19/d64/d4f/d6c/f9c 0 2024-04-24T22:45:36.971 INFO:tasks.workunit.client.1.smithi192.stdout:9/687: truncate d3/dd/d62/d6d/fc5 693241 0 2024-04-24T22:45:36.976 INFO:tasks.workunit.client.1.smithi192.stdout:1/497: unlink d9/d51/f82 0 2024-04-24T22:45:36.976 INFO:tasks.workunit.client.1.smithi192.stdout:1/498: stat d9/d6f 0 2024-04-24T22:45:36.978 INFO:tasks.workunit.client.1.smithi192.stdout:9/688: dwrite d3/dd/de/d17/f1d [0,4194304] 0 2024-04-24T22:45:36.978 INFO:tasks.workunit.client.1.smithi192.stdout:0/585: truncate d3/d27/d28/d2e/d4a/f72 1949436 0 2024-04-24T22:45:36.980 INFO:tasks.workunit.client.1.smithi192.stdout:0/586: write d3/d27/d28/d31/f3a [374217,74059] 0 2024-04-24T22:45:36.980 INFO:tasks.workunit.client.1.smithi192.stdout:9/689: truncate d3/dd/d53/f5c 616969 0 2024-04-24T22:45:36.981 INFO:tasks.workunit.client.1.smithi192.stdout:3/647: mknod d2/d4/d12/d1a/d1d/d7b/ccb 0 2024-04-24T22:45:36.983 INFO:tasks.workunit.client.1.smithi192.stdout:7/622: getdents d2/db/d25 0 2024-04-24T22:45:36.984 INFO:tasks.workunit.client.0.smithi063.stdout:7/683: mkdir d0/d4/de/d17/d35/de4 0 2024-04-24T22:45:36.986 INFO:tasks.workunit.client.0.smithi063.stdout:5/722: dwrite d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:36.987 INFO:tasks.workunit.client.1.smithi192.stdout:6/520: symlink d0/lb7 0 2024-04-24T22:45:36.988 INFO:tasks.workunit.client.1.smithi192.stdout:6/521: stat d0/d8/d3d/d3e 0 2024-04-24T22:45:36.989 INFO:tasks.workunit.client.1.smithi192.stdout:4/582: sync 2024-04-24T22:45:36.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/722: dread d2/d19/d33/dc1/d50/f77 [0,4194304] 0 2024-04-24T22:45:36.992 INFO:tasks.workunit.client.1.smithi192.stdout:7/623: dwrite d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb9 [0,4194304] 0 2024-04-24T22:45:36.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/805: dread - d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/fd0 zero size 2024-04-24T22:45:36.997 INFO:tasks.workunit.client.0.smithi063.stdout:9/755: mknod d6/d2f/d42/dda/cfa 0 2024-04-24T22:45:37.001 INFO:tasks.workunit.client.0.smithi063.stdout:8/776: rmdir d1/d11/d32/d38/db1/de3 0 2024-04-24T22:45:37.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/499: truncate f3 2785335 0 2024-04-24T22:45:37.007 INFO:tasks.workunit.client.1.smithi192.stdout:0/587: mknod d3/d27/d6a/cca 0 2024-04-24T22:45:37.009 INFO:tasks.workunit.client.0.smithi063.stdout:7/684: creat d0/d4/de/d6f/d5c/d8b/fe5 x:0 0 0 2024-04-24T22:45:37.011 INFO:tasks.workunit.client.1.smithi192.stdout:3/648: mknod d2/d4/d12/d1a/d1d/d55/d5f/db1/ccc 0 2024-04-24T22:45:37.014 INFO:tasks.workunit.client.0.smithi063.stdout:0/687: truncate d6/d64/d21/f48 1951469 0 2024-04-24T22:45:37.015 INFO:tasks.workunit.client.1.smithi192.stdout:2/714: link d4/d29/d60/f47 d4/d9/fe7 0 2024-04-24T22:45:37.017 INFO:tasks.workunit.client.0.smithi063.stdout:4/723: unlink d2/d19/d33/dc1/f7a 0 2024-04-24T22:45:37.017 INFO:tasks.workunit.client.1.smithi192.stdout:6/522: unlink d0/d8/d18/d2e/laa 0 2024-04-24T22:45:37.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/688: dwrite d6/d64/da0/d33/fc3 [0,4194304] 0 2024-04-24T22:45:37.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/685: unlink d0/d4/de/d17/d35/c8c 0 2024-04-24T22:45:37.025 INFO:tasks.workunit.client.0.smithi063.stdout:5/723: truncate d5/d1d/f93 47387 0 2024-04-24T22:45:37.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/689: dwrite d6/d64/f92 [0,4194304] 0 2024-04-24T22:45:37.027 INFO:tasks.workunit.client.1.smithi192.stdout:2/715: symlink d4/d9/d10/d6c/d8c/da5/le8 0 2024-04-24T22:45:37.031 INFO:tasks.workunit.client.1.smithi192.stdout:6/523: mknod d0/d8/d22/cb8 0 2024-04-24T22:45:37.033 INFO:tasks.workunit.client.0.smithi063.stdout:7/686: dwrite d0/d4/de/d17/d35/d69/db1/fc1 [0,4194304] 0 2024-04-24T22:45:37.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/581: sync 2024-04-24T22:45:37.035 INFO:tasks.workunit.client.1.smithi192.stdout:5/465: rename d0/d13/d19/d64/l92 to d0/d13/l9d 0 2024-04-24T22:45:37.035 INFO:tasks.workunit.client.1.smithi192.stdout:5/466: readlink d0/l98 0 2024-04-24T22:45:37.037 INFO:tasks.workunit.client.1.smithi192.stdout:7/624: symlink d2/db/lc4 0 2024-04-24T22:45:37.042 INFO:tasks.workunit.client.1.smithi192.stdout:0/588: rmdir d3/d27/d28/d2e/d4a/d5c/d80/dac 39 2024-04-24T22:45:37.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/467: dwrite d0/d1b/f65 [4194304,4194304] 0 2024-04-24T22:45:37.045 INFO:tasks.workunit.client.0.smithi063.stdout:2/846: dwrite d0/d2/d2d/d31/d89/d7/d86/d8e/fbe [0,4194304] 0 2024-04-24T22:45:37.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/625: dwrite d2/db/d21/d42/d5b/f75 [0,4194304] 0 2024-04-24T22:45:37.048 INFO:tasks.workunit.client.1.smithi192.stdout:9/690: link d3/dd/de/dba/f96 d3/d59/ff6 0 2024-04-24T22:45:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:36 smithi063 ceph-mon[26912]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 80 MiB/s rd, 226 MiB/s wr, 449 op/s 2024-04-24T22:45:37.057 INFO:tasks.workunit.client.0.smithi063.stdout:6/802: write db/d1d/d20/d3e/f45 [376418,55335] 0 2024-04-24T22:45:37.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/716: truncate d4/d9/d37/f3b 1681512 0 2024-04-24T22:45:37.060 INFO:tasks.workunit.client.0.smithi063.stdout:0/690: symlink d6/d64/d24/d42/dd4/lec 0 2024-04-24T22:45:37.062 INFO:tasks.workunit.client.1.smithi192.stdout:6/524: symlink d0/d8/d3d/d3e/d40/d44/lb9 0 2024-04-24T22:45:37.063 INFO:tasks.workunit.client.0.smithi063.stdout:5/724: dread d5/f6f [0,4194304] 0 2024-04-24T22:45:37.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/500: write d9/d15/f49 [364904,95072] 0 2024-04-24T22:45:37.068 INFO:tasks.workunit.client.1.smithi192.stdout:4/583: rename d2/db/d75/da1/d36/d70/f85 to d2/db/d75/da1/d53/fc5 0 2024-04-24T22:45:37.068 INFO:tasks.workunit.client.1.smithi192.stdout:4/584: stat d2/d9/d6b/l6d 0 2024-04-24T22:45:37.069 INFO:tasks.workunit.client.1.smithi192.stdout:4/585: fsync d2/db/d75/da1/d36/d5b/f24 0 2024-04-24T22:45:37.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/586: readlink d2/db/d10/l1a 0 2024-04-24T22:45:37.071 INFO:tasks.workunit.client.0.smithi063.stdout:1/806: dwrite d3/db5/d21/d49/fe2 [0,4194304] 0 2024-04-24T22:45:37.072 INFO:tasks.workunit.client.0.smithi063.stdout:9/756: dwrite d6/d2f/dcd/d53/f82 [0,4194304] 0 2024-04-24T22:45:37.073 INFO:tasks.workunit.client.0.smithi063.stdout:9/757: chown d6/d2f/l73 113877 1 2024-04-24T22:45:37.074 INFO:tasks.workunit.client.0.smithi063.stdout:7/687: creat d0/d4/de/d17/d35/d69/da0/fe6 x:0 0 0 2024-04-24T22:45:37.077 INFO:tasks.workunit.client.0.smithi063.stdout:9/758: read - d6/d2f/dcd/d1f/ddb/ff5 zero size 2024-04-24T22:45:37.080 INFO:tasks.workunit.client.1.smithi192.stdout:0/589: mknod d3/d6d/ccb 0 2024-04-24T22:45:37.081 INFO:tasks.workunit.client.0.smithi063.stdout:3/764: link da/d39/c89 da/d18/d9e/c107 0 2024-04-24T22:45:37.083 INFO:tasks.workunit.client.1.smithi192.stdout:5/468: symlink d0/d13/d19/d64/d4f/d6c/l9e 0 2024-04-24T22:45:37.085 INFO:tasks.workunit.client.0.smithi063.stdout:2/847: rename d0/d2/d2d/d31/d89/d1c/ff4 to d0/d2/d2d/d31/d89/d7/d86/d8e/f108 0 2024-04-24T22:45:37.087 INFO:tasks.workunit.client.0.smithi063.stdout:2/848: fdatasync d0/d2/d2d/d31/d89/d7/f53 0 2024-04-24T22:45:37.088 INFO:tasks.workunit.client.1.smithi192.stdout:9/691: mkdir d3/dd/de/d17/d89/de1/df7 0 2024-04-24T22:45:37.090 INFO:tasks.workunit.client.0.smithi063.stdout:0/691: mknod d6/dd/ced 0 2024-04-24T22:45:37.091 INFO:tasks.workunit.client.1.smithi192.stdout:3/649: truncate d2/d27/f4a 3931479 0 2024-04-24T22:45:37.093 INFO:tasks.workunit.client.0.smithi063.stdout:2/849: dwrite d0/d2/d2d/d31/d89/d7/d86/da4/dd9/fea [0,4194304] 0 2024-04-24T22:45:37.098 INFO:tasks.workunit.client.0.smithi063.stdout:5/725: creat d5/d11/d23/d2d/ff9 x:0 0 0 2024-04-24T22:45:37.098 INFO:tasks.workunit.client.1.smithi192.stdout:3/650: dwrite d2/d4/d12/f88 [0,4194304] 0 2024-04-24T22:45:37.100 INFO:tasks.workunit.client.1.smithi192.stdout:1/501: chown d9/dd/l60 190397066 1 2024-04-24T22:45:37.109 INFO:tasks.workunit.client.0.smithi063.stdout:4/724: truncate d2/d19/d82/fb8 1029710 0 2024-04-24T22:45:37.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/777: link d1/d22/l64 d1/d11/d32/lfe 0 2024-04-24T22:45:37.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/469: mknod d0/d13/d53/c9f 0 2024-04-24T22:45:37.114 INFO:tasks.workunit.client.0.smithi063.stdout:9/759: rename d6/d2f/dcd/c86 to d6/d2f/dcd/db/d6b/d4f/cfb 0 2024-04-24T22:45:37.115 INFO:tasks.workunit.client.0.smithi063.stdout:8/778: dwrite d1/d11/f63 [0,4194304] 0 2024-04-24T22:45:37.117 INFO:tasks.workunit.client.1.smithi192.stdout:9/692: unlink d3/d5d/da1/d85/lb3 0 2024-04-24T22:45:37.125 INFO:tasks.workunit.client.0.smithi063.stdout:2/850: creat d0/dd/d72/d77/dda/d7c/f109 x:0 0 0 2024-04-24T22:45:37.126 INFO:tasks.workunit.client.0.smithi063.stdout:5/726: symlink d5/d11/d23/d2d/d62/d73/d95/lfa 0 2024-04-24T22:45:37.129 INFO:tasks.workunit.client.1.smithi192.stdout:7/626: write d2/db/d21/d42/d5b/f6b [119463,117108] 0 2024-04-24T22:45:37.133 INFO:tasks.workunit.client.1.smithi192.stdout:5/470: symlink d0/d13/d19/d64/d4f/la0 0 2024-04-24T22:45:37.135 INFO:tasks.workunit.client.0.smithi063.stdout:4/725: unlink d2/d19/d33/l43 0 2024-04-24T22:45:37.135 INFO:tasks.workunit.client.1.smithi192.stdout:9/693: rmdir d3/dd/d62 39 2024-04-24T22:45:37.136 INFO:tasks.workunit.client.1.smithi192.stdout:9/694: dread - d3/dd/d53/fd5 zero size 2024-04-24T22:45:37.138 INFO:tasks.workunit.client.1.smithi192.stdout:2/717: creat d4/d29/fe9 x:0 0 0 2024-04-24T22:45:37.139 INFO:tasks.workunit.client.0.smithi063.stdout:3/765: mkdir da/d39/d94/db3/df3/d108 0 2024-04-24T22:45:37.139 INFO:tasks.workunit.client.1.smithi192.stdout:4/587: write d2/d9/d25/f8e [213385,79868] 0 2024-04-24T22:45:37.140 INFO:tasks.workunit.client.1.smithi192.stdout:8/582: getdents db/d43 0 2024-04-24T22:45:37.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/588: write d2/f17 [485195,115145] 0 2024-04-24T22:45:37.143 INFO:tasks.workunit.client.1.smithi192.stdout:1/502: truncate d9/d51/d70/f94 4558294 0 2024-04-24T22:45:37.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/627: fdatasync d2/db/f6f 0 2024-04-24T22:45:37.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/803: creat db/d1d/d2d/f109 x:0 0 0 2024-04-24T22:45:37.149 INFO:tasks.workunit.client.1.smithi192.stdout:1/503: dwrite d9/f14 [0,4194304] 0 2024-04-24T22:45:37.153 INFO:tasks.workunit.client.1.smithi192.stdout:0/590: rename d3/d27/d28/d2e/d4a to d3/d27/d28/dcc 0 2024-04-24T22:45:37.154 INFO:tasks.workunit.client.1.smithi192.stdout:6/525: truncate d0/d8/d29/d58/fab 793810 0 2024-04-24T22:45:37.159 INFO:tasks.workunit.client.0.smithi063.stdout:1/807: truncate d3/de0/d79/f2e 3138611 0 2024-04-24T22:45:37.161 INFO:tasks.workunit.client.0.smithi063.stdout:7/688: write d0/f7 [1226731,68731] 0 2024-04-24T22:45:37.162 INFO:tasks.workunit.client.0.smithi063.stdout:1/808: chown d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/cdb 0 1 2024-04-24T22:45:37.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/583: unlink db/dc/f18 0 2024-04-24T22:45:37.170 INFO:tasks.workunit.client.0.smithi063.stdout:7/689: dread d0/f7 [0,4194304] 0 2024-04-24T22:45:37.176 INFO:tasks.workunit.client.0.smithi063.stdout:6/804: symlink db/d1d/d20/d29/l10a 0 2024-04-24T22:45:37.177 INFO:tasks.workunit.client.1.smithi192.stdout:7/628: read - d2/db/fa6 zero size 2024-04-24T22:45:37.179 INFO:tasks.workunit.client.0.smithi063.stdout:0/692: rmdir d6/d64/da0/d33/db5 0 2024-04-24T22:45:37.183 INFO:tasks.workunit.client.1.smithi192.stdout:0/591: truncate d3/d27/d28/dcc/d5c/f3d 5088242 0 2024-04-24T22:45:37.183 INFO:tasks.workunit.client.0.smithi063.stdout:6/805: dwrite db/d1d/d2d/f109 [0,4194304] 0 2024-04-24T22:45:37.185 INFO:tasks.workunit.client.1.smithi192.stdout:8/584: dread db/dc/f17 [0,4194304] 0 2024-04-24T22:45:37.185 INFO:tasks.workunit.client.1.smithi192.stdout:7/629: dwrite d2/f91 [0,4194304] 0 2024-04-24T22:45:37.187 INFO:tasks.workunit.client.1.smithi192.stdout:6/526: rename d0/d8/d22/cb8 to d0/d8/d3d/d9d/cba 0 2024-04-24T22:45:37.189 INFO:tasks.workunit.client.0.smithi063.stdout:2/851: dwrite d0/dd/d72/d77/f78 [0,4194304] 0 2024-04-24T22:45:37.190 INFO:tasks.workunit.client.0.smithi063.stdout:0/693: dwrite d6/d64/d21/fb3 [0,4194304] 0 2024-04-24T22:45:37.192 INFO:tasks.workunit.client.0.smithi063.stdout:4/726: unlink d2/d8/c5a 0 2024-04-24T22:45:37.194 INFO:tasks.workunit.client.0.smithi063.stdout:8/779: truncate d1/d21/f3a 696304 0 2024-04-24T22:45:37.196 INFO:tasks.workunit.client.0.smithi063.stdout:3/766: dwrite da/d18/f2a [4194304,4194304] 0 2024-04-24T22:45:37.198 INFO:tasks.workunit.client.1.smithi192.stdout:6/527: dwrite d0/d8/d22/fb4 [0,4194304] 0 2024-04-24T22:45:37.198 INFO:tasks.workunit.client.1.smithi192.stdout:4/589: dwrite d2/db/d75/da1/d36/d5b/d28/d9f/fa0 [4194304,4194304] 0 2024-04-24T22:45:37.201 INFO:tasks.workunit.client.1.smithi192.stdout:9/695: read d3/dd/d62/d6d/fc5 [311198,53978] 0 2024-04-24T22:45:37.202 INFO:tasks.workunit.client.1.smithi192.stdout:9/696: chown d3/dd/de/dba/dbc/ddb/lee 694373990 1 2024-04-24T22:45:37.204 INFO:tasks.workunit.client.1.smithi192.stdout:2/718: creat d4/d9/dd4/fea x:0 0 0 2024-04-24T22:45:37.210 INFO:tasks.workunit.client.1.smithi192.stdout:4/590: dwrite d2/db/d75/da1/d36/d5b/d28/f45 [0,4194304] 0 2024-04-24T22:45:37.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/760: link d6/d2f/dcd/db/d6b/d4f/dad/l64 d6/d2f/dcd/d8a/lfc 0 2024-04-24T22:45:37.216 INFO:tasks.workunit.client.1.smithi192.stdout:3/651: getdents d2/d4/d12/d39/d97 0 2024-04-24T22:45:37.217 INFO:tasks.workunit.client.0.smithi063.stdout:9/761: dread d6/d2f/dcd/db/d6b/d4f/d59/f9b [0,4194304] 0 2024-04-24T22:45:37.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/719: dwrite d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:37.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/809: dwrite d3/f23 [0,4194304] 0 2024-04-24T22:45:37.223 INFO:tasks.workunit.client.1.smithi192.stdout:2/720: dread - d4/d9/d9f/fd7 zero size 2024-04-24T22:45:37.223 INFO:tasks.workunit.client.1.smithi192.stdout:2/721: write d4/d9/d9f/fd7 [142142,19938] 0 2024-04-24T22:45:37.223 INFO:tasks.workunit.client.1.smithi192.stdout:3/652: read d2/d27/dc6/d4f/f8c [365575,98282] 0 2024-04-24T22:45:37.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/722: readlink d4/d29/l3d 0 2024-04-24T22:45:37.227 INFO:tasks.workunit.client.0.smithi063.stdout:2/852: rename d0/d2/d2d/d31/d89/d1c/d3f/f42 to d0/dd/d72/f10a 0 2024-04-24T22:45:37.229 INFO:tasks.workunit.client.0.smithi063.stdout:4/727: creat d2/d19/d33/dc1/d50/feb x:0 0 0 2024-04-24T22:45:37.230 INFO:tasks.workunit.client.1.smithi192.stdout:4/591: dread d2/d5c/f86 [0,4194304] 0 2024-04-24T22:45:37.231 INFO:tasks.workunit.client.1.smithi192.stdout:8/585: mknod db/d43/d7c/cb3 0 2024-04-24T22:45:37.234 INFO:tasks.workunit.client.1.smithi192.stdout:8/586: truncate db/fae 645381 0 2024-04-24T22:45:37.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/690: mkdir d0/d4/de/d17/d28/d31/dad/de7 0 2024-04-24T22:45:37.235 INFO:tasks.workunit.client.1.smithi192.stdout:3/653: dwrite d2/d4/d12/d1a/d1d/f3f [0,4194304] 0 2024-04-24T22:45:37.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/767: mkdir da/d18/d47/d4d/d62/d109 0 2024-04-24T22:45:37.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/587: write db/dc/d29/f2b [2037575,109208] 0 2024-04-24T22:45:37.238 INFO:tasks.workunit.client.0.smithi063.stdout:3/768: truncate da/d18/d9e/da3/dc1/ff6 68396 0 2024-04-24T22:45:37.238 INFO:tasks.workunit.client.0.smithi063.stdout:3/769: chown da/d18/f58 5431807 1 2024-04-24T22:45:37.242 INFO:tasks.workunit.client.0.smithi063.stdout:9/762: readlink d6/d2f/dcd/db/d1c/lf4 0 2024-04-24T22:45:37.245 INFO:tasks.workunit.client.0.smithi063.stdout:6/806: symlink db/d1c/l10b 0 2024-04-24T22:45:37.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/728: mkdir d2/d19/d33/d9d/da8/dec 0 2024-04-24T22:45:37.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/729: dread - d2/d8/d40/d51/f66 zero size 2024-04-24T22:45:37.250 INFO:tasks.workunit.client.0.smithi063.stdout:7/691: truncate d0/d4/de/d17/d35/d6c/f9e 395672 0 2024-04-24T22:45:37.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/592: mknod d3/d27/d28/dcc/d5c/d80/ccd 0 2024-04-24T22:45:37.254 INFO:tasks.workunit.client.0.smithi063.stdout:1/810: mkdir d3/de0/d79/d96/df8/d10a 0 2024-04-24T22:45:37.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/630: mknod d2/db/d21/d42/d67/dae/dbb/cc5 0 2024-04-24T22:45:37.255 INFO:tasks.workunit.client.1.smithi192.stdout:7/631: fsync d2/f91 0 2024-04-24T22:45:37.256 INFO:tasks.workunit.client.0.smithi063.stdout:1/811: read d3/da/d34/d68/fa8 [2837125,115595] 0 2024-04-24T22:45:37.256 INFO:tasks.workunit.client.1.smithi192.stdout:7/632: truncate d2/f91 5097472 0 2024-04-24T22:45:37.257 INFO:tasks.workunit.client.0.smithi063.stdout:7/692: dwrite d0/d4/de/d6f/d5c/d9c/fd8 [0,4194304] 0 2024-04-24T22:45:37.258 INFO:tasks.workunit.client.0.smithi063.stdout:7/693: stat d0/d4/de/f25 0 2024-04-24T22:45:37.259 INFO:tasks.workunit.client.0.smithi063.stdout:8/780: creat d1/d2/fff x:0 0 0 2024-04-24T22:45:37.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/807: dread db/d1d/d20/d3f/d7f/d68/d70/fad [0,4194304] 0 2024-04-24T22:45:37.259 INFO:tasks.workunit.client.1.smithi192.stdout:3/654: creat d2/d4/d12/d1a/d1d/d7b/fcd x:0 0 0 2024-04-24T22:45:37.260 INFO:tasks.workunit.client.1.smithi192.stdout:6/528: fdatasync d0/d8/d29/d58/fab 0 2024-04-24T22:45:37.262 INFO:tasks.workunit.client.1.smithi192.stdout:6/529: write d0/d8/d18/d2e/fa8 [335101,53558] 0 2024-04-24T22:45:37.270 INFO:tasks.workunit.client.0.smithi063.stdout:9/763: creat d6/d2f/dcd/d1f/d20/d2a/d4b/de0/ffd x:0 0 0 2024-04-24T22:45:37.272 INFO:tasks.workunit.client.0.smithi063.stdout:5/727: write d5/d1d/f93 [681052,4976] 0 2024-04-24T22:45:37.274 INFO:tasks.workunit.client.1.smithi192.stdout:5/471: getdents d0/d13/d19/d71 0 2024-04-24T22:45:37.274 INFO:tasks.workunit.client.0.smithi063.stdout:5/728: write d5/d1d/f93 [1569983,57142] 0 2024-04-24T22:45:37.275 INFO:tasks.workunit.client.0.smithi063.stdout:4/730: mkdir d2/d19/d33/d9d/da8/dec/ded 0 2024-04-24T22:45:37.276 INFO:tasks.workunit.client.1.smithi192.stdout:7/633: write d2/db/d21/f84 [1674428,11304] 0 2024-04-24T22:45:37.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/781: creat d1/d11/d32/f100 x:0 0 0 2024-04-24T22:45:37.278 INFO:tasks.workunit.client.0.smithi063.stdout:2/853: dread d0/d2/d63/f71 [0,4194304] 0 2024-04-24T22:45:37.280 INFO:tasks.workunit.client.1.smithi192.stdout:1/504: truncate d9/dd/f19 774229 0 2024-04-24T22:45:37.282 INFO:tasks.workunit.client.1.smithi192.stdout:0/593: symlink d3/d27/d28/lce 0 2024-04-24T22:45:37.285 INFO:tasks.workunit.client.1.smithi192.stdout:2/723: rename d4/d9/d37/f55 to d4/d9/d10/d6c/d6f/feb 0 2024-04-24T22:45:37.289 INFO:tasks.workunit.client.1.smithi192.stdout:7/634: creat d2/db/d21/fc6 x:0 0 0 2024-04-24T22:45:37.289 INFO:tasks.workunit.client.1.smithi192.stdout:5/472: dread d0/d1b/f45 [0,4194304] 0 2024-04-24T22:45:37.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/694: dread d0/d4/de/d6f/d5c/fcd [4194304,4194304] 0 2024-04-24T22:45:37.291 INFO:tasks.workunit.client.1.smithi192.stdout:3/655: truncate d2/d27/dc6/f47 2472 0 2024-04-24T22:45:37.292 INFO:tasks.workunit.client.0.smithi063.stdout:0/694: sync 2024-04-24T22:45:37.294 INFO:tasks.workunit.client.1.smithi192.stdout:6/530: mknod d0/d8/d29/d4c/d4d/d6a/cbb 0 2024-04-24T22:45:37.295 INFO:tasks.workunit.client.0.smithi063.stdout:0/695: truncate d6/d64/d24/d42/dd4/f96 889229 0 2024-04-24T22:45:37.296 INFO:tasks.workunit.client.1.smithi192.stdout:1/505: mknod d9/dd/d74/caa 0 2024-04-24T22:45:37.297 INFO:tasks.workunit.client.0.smithi063.stdout:1/812: dread d3/de0/d79/f2e [0,4194304] 0 2024-04-24T22:45:37.298 INFO:tasks.workunit.client.1.smithi192.stdout:0/594: unlink d3/f55 0 2024-04-24T22:45:37.303 INFO:tasks.workunit.client.1.smithi192.stdout:4/592: rename d2/faa to d2/db/d10/fc6 0 2024-04-24T22:45:37.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/724: creat d4/d9/dd4/fec x:0 0 0 2024-04-24T22:45:37.311 INFO:tasks.workunit.client.1.smithi192.stdout:6/531: stat d0/d8/d29/f7c 0 2024-04-24T22:45:37.313 INFO:tasks.workunit.client.1.smithi192.stdout:8/588: dwrite db/dc/d22/f5e [0,4194304] 0 2024-04-24T22:45:37.315 INFO:tasks.workunit.client.1.smithi192.stdout:0/595: mknod d3/d27/d6a/ccf 0 2024-04-24T22:45:37.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/596: write d3/d27/d28/fc3 [919309,68301] 0 2024-04-24T22:45:37.320 INFO:tasks.workunit.client.1.smithi192.stdout:7/635: rename d2/db/d2a/d57/da9 to d2/db/d21/d42/d67/da8/dc7 0 2024-04-24T22:45:37.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/593: symlink d2/db/d75/da1/d36/d5b/d28/lc7 0 2024-04-24T22:45:37.326 INFO:tasks.workunit.client.1.smithi192.stdout:0/597: dwrite d3/d27/d3c/d6c/fae [0,4194304] 0 2024-04-24T22:45:37.329 INFO:tasks.workunit.client.1.smithi192.stdout:5/473: mknod d0/d13/d19/d71/d7f/d9a/ca1 0 2024-04-24T22:45:37.333 INFO:tasks.workunit.client.1.smithi192.stdout:3/656: creat d2/d27/dc6/da0/fce x:0 0 0 2024-04-24T22:45:37.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/598: dwrite d3/d27/d3c/d6c/fae [0,4194304] 0 2024-04-24T22:45:37.338 INFO:tasks.workunit.client.1.smithi192.stdout:6/532: symlink d0/d8/d29/d4c/d4d/lbc 0 2024-04-24T22:45:37.339 INFO:tasks.workunit.client.1.smithi192.stdout:6/533: chown d0/d8/d3d/d3e/d40/d44/c98 20240 1 2024-04-24T22:45:37.341 INFO:tasks.workunit.client.1.smithi192.stdout:1/506: getdents d9/d15/d5e/d90/da9 0 2024-04-24T22:45:37.343 INFO:tasks.workunit.client.1.smithi192.stdout:8/589: mkdir db/d43/d7c/d85/db4 0 2024-04-24T22:45:37.348 INFO:tasks.workunit.client.1.smithi192.stdout:7/636: creat d2/db/d21/d42/d67/dae/fc8 x:0 0 0 2024-04-24T22:45:37.349 INFO:tasks.workunit.client.1.smithi192.stdout:7/637: readlink d2/db/d2a/d57/l94 0 2024-04-24T22:45:37.354 INFO:tasks.workunit.client.0.smithi063.stdout:9/764: unlink d6/d2f/l73 0 2024-04-24T22:45:37.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/474: rmdir d0/d13/d19/d71 39 2024-04-24T22:45:37.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/475: chown d0/l98 596627315 1 2024-04-24T22:45:37.357 INFO:tasks.workunit.client.0.smithi063.stdout:5/729: creat d5/d1d/d7d/ffb x:0 0 0 2024-04-24T22:45:37.358 INFO:tasks.workunit.client.0.smithi063.stdout:9/765: read d6/d2f/dcd/db/d1c/f69 [241939,59974] 0 2024-04-24T22:45:37.359 INFO:tasks.workunit.client.0.smithi063.stdout:5/730: write d5/d24/d49/d3b/dc5/feb [484701,107411] 0 2024-04-24T22:45:37.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/766: stat d6/dbd/dcf 0 2024-04-24T22:45:37.359 INFO:tasks.workunit.client.0.smithi063.stdout:4/731: fsync d2/d8/d40/d51/f69 0 2024-04-24T22:45:37.360 INFO:tasks.workunit.client.0.smithi063.stdout:6/808: creat db/d1d/d8d/dfc/f10c x:0 0 0 2024-04-24T22:45:37.361 INFO:tasks.workunit.client.0.smithi063.stdout:4/732: readlink d2/d8/d40/d51/dd5/ldc 0 2024-04-24T22:45:37.362 INFO:tasks.workunit.client.0.smithi063.stdout:2/854: stat d0/fa6 0 2024-04-24T22:45:37.363 INFO:tasks.workunit.client.1.smithi192.stdout:8/590: read db/dc/d22/d31/f50 [99773,21463] 0 2024-04-24T22:45:37.364 INFO:tasks.workunit.client.0.smithi063.stdout:8/782: truncate d1/d21/f83 1422020 0 2024-04-24T22:45:37.365 INFO:tasks.workunit.client.1.smithi192.stdout:2/725: write d4/d9/d10/d6c/d95/db7/fc3 [575348,27607] 0 2024-04-24T22:45:37.367 INFO:tasks.workunit.client.0.smithi063.stdout:3/770: rename da/f3b to da/d18/d9e/f10a 0 2024-04-24T22:45:37.369 INFO:tasks.workunit.client.1.smithi192.stdout:7/638: rename d2/db/d25/l31 to d2/db/d21/d42/d5b/d63/lc9 0 2024-04-24T22:45:37.370 INFO:tasks.workunit.client.0.smithi063.stdout:7/695: unlink d0/d2f/c30 0 2024-04-24T22:45:37.370 INFO:tasks.workunit.client.1.smithi192.stdout:3/657: creat d2/d27/d28/dab/dca/fcf x:0 0 0 2024-04-24T22:45:37.371 INFO:tasks.workunit.client.1.smithi192.stdout:3/658: stat d2/d4/d12/dbc/db0 0 2024-04-24T22:45:37.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/696: chown d6/d64/c70 3017284 1 2024-04-24T22:45:37.375 INFO:tasks.workunit.client.1.smithi192.stdout:0/599: creat d3/d27/d28/d2e/dc7/fd0 x:0 0 0 2024-04-24T22:45:37.377 INFO:tasks.workunit.client.0.smithi063.stdout:0/697: dwrite d6/d64/d24/d42/fbe [0,4194304] 0 2024-04-24T22:45:37.384 INFO:tasks.workunit.client.1.smithi192.stdout:2/726: creat d4/d9/d10/d6c/d95/db7/fed x:0 0 0 2024-04-24T22:45:37.385 INFO:tasks.workunit.client.1.smithi192.stdout:2/727: write d4/d9/f52 [1181308,128824] 0 2024-04-24T22:45:37.386 INFO:tasks.workunit.client.0.smithi063.stdout:9/767: dwrite d6/d2f/dcd/d1f/d20/d22/f47 [0,4194304] 0 2024-04-24T22:45:37.388 INFO:tasks.workunit.client.0.smithi063.stdout:6/809: creat db/d1d/d2d/d69/dd4/f10d x:0 0 0 2024-04-24T22:45:37.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/639: dread - d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fab zero size 2024-04-24T22:45:37.390 INFO:tasks.workunit.client.1.smithi192.stdout:7/640: truncate d2/db/d49/f70 5007274 0 2024-04-24T22:45:37.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/855: unlink d0/dd/d72/dc9/cf6 0 2024-04-24T22:45:37.393 INFO:tasks.workunit.client.0.smithi063.stdout:8/783: read d1/d2/d3/db6/fbb [347275,109870] 0 2024-04-24T22:45:37.394 INFO:tasks.workunit.client.1.smithi192.stdout:5/476: symlink d0/d13/d19/d71/d88/la2 0 2024-04-24T22:45:37.397 INFO:tasks.workunit.client.1.smithi192.stdout:0/600: creat d3/d27/d3c/d3b/fd1 x:0 0 0 2024-04-24T22:45:37.398 INFO:tasks.workunit.client.1.smithi192.stdout:0/601: stat d3/d27/d28/dcc/d5c/f9a 0 2024-04-24T22:45:37.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/733: mkdir d2/d8/dee 0 2024-04-24T22:45:37.407 INFO:tasks.workunit.client.1.smithi192.stdout:4/594: getdents d2/db/d5d/d77/d82 0 2024-04-24T22:45:37.408 INFO:tasks.workunit.client.0.smithi063.stdout:1/813: dwrite d3/db5/d21/d9f/dd5/d106/d11/d5e/fa6 [0,4194304] 0 2024-04-24T22:45:37.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/734: truncate d2/d19/d82/d96/fa9 4830872 0 2024-04-24T22:45:37.412 INFO:tasks.workunit.client.1.smithi192.stdout:2/728: symlink d4/d9/dd4/lee 0 2024-04-24T22:45:37.414 INFO:tasks.workunit.client.0.smithi063.stdout:1/814: dread d3/db5/d21/d9f/daa/feb [0,4194304] 0 2024-04-24T22:45:37.414 INFO:tasks.workunit.client.1.smithi192.stdout:7/641: mkdir d2/db/d49/dca 0 2024-04-24T22:45:37.419 INFO:tasks.workunit.client.1.smithi192.stdout:3/659: chown d2/d27/dc6/f47 254 1 2024-04-24T22:45:37.420 INFO:tasks.workunit.client.0.smithi063.stdout:6/810: dread db/d1d/d20/d3e/f4f [0,4194304] 0 2024-04-24T22:45:37.420 INFO:tasks.workunit.client.0.smithi063.stdout:2/856: truncate d0/d2/d39/f65 741568 0 2024-04-24T22:45:37.423 INFO:tasks.workunit.client.1.smithi192.stdout:6/534: getdents d0/d8/d18/d2e 0 2024-04-24T22:45:37.423 INFO:tasks.workunit.client.1.smithi192.stdout:0/602: dread d3/d27/d28/f56 [0,4194304] 0 2024-04-24T22:45:37.424 INFO:tasks.workunit.client.1.smithi192.stdout:0/603: readlink d3/d27/d28/d2e/d3f/l9f 0 2024-04-24T22:45:37.424 INFO:tasks.workunit.client.1.smithi192.stdout:1/507: getdents d9/d51 0 2024-04-24T22:45:37.424 INFO:tasks.workunit.client.1.smithi192.stdout:1/508: dread - d9/d15/d2d/d4a/f8b zero size 2024-04-24T22:45:37.425 INFO:tasks.workunit.client.1.smithi192.stdout:8/591: link db/d43/c68 db/dc/d29/cb5 0 2024-04-24T22:45:37.425 INFO:tasks.workunit.client.0.smithi063.stdout:5/731: truncate d5/d11/d23/fd5 338830 0 2024-04-24T22:45:37.427 INFO:tasks.workunit.client.0.smithi063.stdout:7/696: rename d0/d4/de/d6f/c3f to d0/d4/de/d6f/d2e/d55/ce8 0 2024-04-24T22:45:37.428 INFO:tasks.workunit.client.0.smithi063.stdout:7/697: chown d0/d4/de/d17/d28/fb8 2 1 2024-04-24T22:45:37.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/698: readlink d0/d4/de/d17/d28/d31/lcc 0 2024-04-24T22:45:37.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/735: unlink d2/d8/d3c/f62 0 2024-04-24T22:45:37.431 INFO:tasks.workunit.client.0.smithi063.stdout:9/768: write d6/d2f/d42/fb0 [636565,99966] 0 2024-04-24T22:45:37.433 INFO:tasks.workunit.client.1.smithi192.stdout:6/535: creat d0/d8/d3d/d3e/d40/d44/fbd x:0 0 0 2024-04-24T22:45:37.433 INFO:tasks.workunit.client.1.smithi192.stdout:0/604: read - d3/d27/d3c/f4e zero size 2024-04-24T22:45:37.434 INFO:tasks.workunit.client.0.smithi063.stdout:2/857: symlink d0/d2/d35/dc6/de2/l10b 0 2024-04-24T22:45:37.435 INFO:tasks.workunit.client.1.smithi192.stdout:8/592: fsync f9 0 2024-04-24T22:45:37.437 INFO:tasks.workunit.client.1.smithi192.stdout:8/593: chown db/dc/f44 85334 1 2024-04-24T22:45:37.437 INFO:tasks.workunit.client.0.smithi063.stdout:5/732: rename d5/d24/d49/ff3 to d5/d11/dec/ffc 0 2024-04-24T22:45:37.438 INFO:tasks.workunit.client.0.smithi063.stdout:0/698: creat d6/d3a/fee x:0 0 0 2024-04-24T22:45:37.439 INFO:tasks.workunit.client.0.smithi063.stdout:8/784: dread d1/d2/fe [0,4194304] 0 2024-04-24T22:45:37.440 INFO:tasks.workunit.client.1.smithi192.stdout:0/605: dwrite d3/d27/d28/dcc/f6b [0,4194304] 0 2024-04-24T22:45:37.445 INFO:tasks.workunit.client.1.smithi192.stdout:0/606: read d3/d6d/f91 [2650173,30978] 0 2024-04-24T22:45:37.445 INFO:tasks.workunit.client.1.smithi192.stdout:0/607: fsync d3/fc0 0 2024-04-24T22:45:37.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/769: unlink d6/d2f/dcd/d53/f82 0 2024-04-24T22:45:37.454 INFO:tasks.workunit.client.0.smithi063.stdout:6/811: symlink db/d1d/d2d/d48/de5/l10e 0 2024-04-24T22:45:37.455 INFO:tasks.workunit.client.1.smithi192.stdout:4/595: truncate d2/d9/f3f 1354434 0 2024-04-24T22:45:37.456 INFO:tasks.workunit.client.0.smithi063.stdout:2/858: symlink d0/d2/d35/dc6/l10c 0 2024-04-24T22:45:37.457 INFO:tasks.workunit.client.0.smithi063.stdout:2/859: chown d0/dd/d72/d77/f78 0 1 2024-04-24T22:45:37.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/771: sync 2024-04-24T22:45:37.459 INFO:tasks.workunit.client.1.smithi192.stdout:8/594: rmdir db/d5d 39 2024-04-24T22:45:37.460 INFO:tasks.workunit.client.0.smithi063.stdout:5/733: mknod d5/d11/d76/d92/cfd 0 2024-04-24T22:45:37.462 INFO:tasks.workunit.client.0.smithi063.stdout:0/699: creat d6/d64/d21/d80/fef x:0 0 0 2024-04-24T22:45:37.464 INFO:tasks.workunit.client.1.smithi192.stdout:6/536: dread d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:37.465 INFO:tasks.workunit.client.0.smithi063.stdout:0/700: truncate d6/d3a/fee 983632 0 2024-04-24T22:45:37.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/701: readlink d6/l56 0 2024-04-24T22:45:37.466 INFO:tasks.workunit.client.0.smithi063.stdout:8/785: chown d1/d11/d32/fb2 113 1 2024-04-24T22:45:37.466 INFO:tasks.workunit.client.0.smithi063.stdout:5/734: dwrite d5/d1d/f93 [0,4194304] 0 2024-04-24T22:45:37.468 INFO:tasks.workunit.client.0.smithi063.stdout:8/786: readlink d1/d22/d33/da6/d54/l94 0 2024-04-24T22:45:37.468 INFO:tasks.workunit.client.0.smithi063.stdout:7/699: symlink d0/d4/de/d17/d28/d31/d99/le9 0 2024-04-24T22:45:37.469 INFO:tasks.workunit.client.0.smithi063.stdout:5/735: write d5/d11/f6a [1018070,95226] 0 2024-04-24T22:45:37.469 INFO:tasks.workunit.client.0.smithi063.stdout:8/787: fsync d1/f5 0 2024-04-24T22:45:37.474 INFO:tasks.workunit.client.1.smithi192.stdout:1/509: creat d9/d15/d2d/d31/fab x:0 0 0 2024-04-24T22:45:37.475 INFO:tasks.workunit.client.0.smithi063.stdout:5/736: dread d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:37.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/510: chown d9/dd/f41 7 1 2024-04-24T22:45:37.476 INFO:tasks.workunit.client.0.smithi063.stdout:6/812: rename db/d1d/d2d/d48/de5 to db/d85/db5/d10f 0 2024-04-24T22:45:37.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/737: chown d5/d11/f15 66596 1 2024-04-24T22:45:37.477 INFO:tasks.workunit.client.0.smithi063.stdout:8/788: dwrite d1/d11/d32/dd3/fe8 [0,4194304] 0 2024-04-24T22:45:37.477 INFO:tasks.workunit.client.1.smithi192.stdout:1/511: dread - d9/f4f zero size 2024-04-24T22:45:37.477 INFO:tasks.workunit.client.1.smithi192.stdout:8/595: symlink db/dc/d29/d9f/lb6 0 2024-04-24T22:45:37.477 INFO:tasks.workunit.client.0.smithi063.stdout:1/815: sync 2024-04-24T22:45:37.479 INFO:tasks.workunit.client.1.smithi192.stdout:1/512: write d9/d6f/f86 [1442618,83203] 0 2024-04-24T22:45:37.480 INFO:tasks.workunit.client.1.smithi192.stdout:0/608: dwrite d3/f1c [0,4194304] 0 2024-04-24T22:45:37.483 INFO:tasks.workunit.client.1.smithi192.stdout:2/729: getdents d4/d9/d10/d6c/d8c 0 2024-04-24T22:45:37.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/537: fsync d0/d8/d29/f3b 0 2024-04-24T22:45:37.486 INFO:tasks.workunit.client.0.smithi063.stdout:4/736: creat d2/d8/dd4/dda/fef x:0 0 0 2024-04-24T22:45:37.487 INFO:tasks.workunit.client.1.smithi192.stdout:3/660: link d2/d27/dc6/f47 d2/d4/d12/d39/fd0 0 2024-04-24T22:45:37.489 INFO:tasks.workunit.client.1.smithi192.stdout:3/661: write d2/d62/db2/f85 [782808,26627] 0 2024-04-24T22:45:37.496 INFO:tasks.workunit.client.1.smithi192.stdout:8/596: fsync db/dc/d29/f86 0 2024-04-24T22:45:37.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/513: mkdir d9/d15/d2d/d78/dac 0 2024-04-24T22:45:37.499 INFO:tasks.workunit.client.1.smithi192.stdout:0/609: truncate d3/d27/d28/d2e/f4c 984596 0 2024-04-24T22:45:37.500 INFO:tasks.workunit.client.1.smithi192.stdout:0/610: chown d3/l2f 6527 1 2024-04-24T22:45:37.501 INFO:tasks.workunit.client.0.smithi063.stdout:6/813: creat db/d1d/d20/d3e/f110 x:0 0 0 2024-04-24T22:45:37.501 INFO:tasks.workunit.client.1.smithi192.stdout:2/730: creat d4/d9/d10/d6c/d6f/da8/fef x:0 0 0 2024-04-24T22:45:37.503 INFO:tasks.workunit.client.1.smithi192.stdout:6/538: creat d0/d8/d29/d58/fbe x:0 0 0 2024-04-24T22:45:37.504 INFO:tasks.workunit.client.1.smithi192.stdout:7/642: link d2/db/d21/f28 d2/fcb 0 2024-04-24T22:45:37.506 INFO:tasks.workunit.client.0.smithi063.stdout:6/814: dwrite db/d1c/f90 [0,4194304] 0 2024-04-24T22:45:37.507 INFO:tasks.workunit.client.0.smithi063.stdout:5/738: rename d5/d1d/dc0/f72 to d5/d11/d23/d2d/d62/ffe 0 2024-04-24T22:45:37.507 INFO:tasks.workunit.client.0.smithi063.stdout:6/815: write db/d1c/dc3/fc8 [659191,59166] 0 2024-04-24T22:45:37.508 INFO:tasks.workunit.client.1.smithi192.stdout:6/539: dread d0/d8/d29/d5a/fb6 [0,4194304] 0 2024-04-24T22:45:37.508 INFO:tasks.workunit.client.1.smithi192.stdout:4/596: creat d2/d9/fc8 x:0 0 0 2024-04-24T22:45:37.510 INFO:tasks.workunit.client.0.smithi063.stdout:2/860: mknod d0/d2/d2d/d31/d89/d1c/d3f/dca/c10d 0 2024-04-24T22:45:37.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/816: dwrite db/fdf [0,4194304] 0 2024-04-24T22:45:37.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/514: mknod d9/d15/d2d/d63/d68/cad 0 2024-04-24T22:45:37.521 INFO:tasks.workunit.client.1.smithi192.stdout:1/515: write d9/d15/d2d/d31/d2f/f3a [2893071,44761] 0 2024-04-24T22:45:37.521 INFO:tasks.workunit.client.1.smithi192.stdout:0/611: read - d3/d27/d28/dcc/f8c zero size 2024-04-24T22:45:37.522 INFO:tasks.workunit.client.1.smithi192.stdout:0/612: fsync d3/d40/f9b 0 2024-04-24T22:45:37.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/700: truncate d0/d4/de/f12 690492 0 2024-04-24T22:45:37.532 INFO:tasks.workunit.client.0.smithi063.stdout:1/816: unlink d3/c4 0 2024-04-24T22:45:37.534 INFO:tasks.workunit.client.0.smithi063.stdout:8/789: write d1/d2/d3/db6/fbb [1762865,46717] 0 2024-04-24T22:45:37.535 INFO:tasks.workunit.client.0.smithi063.stdout:8/790: chown d1/d11/d32/fc7 15 1 2024-04-24T22:45:37.539 INFO:tasks.workunit.client.0.smithi063.stdout:2/861: mkdir d0/d2/d2d/d31/d89/d7/d86/da0/deb/d10e 0 2024-04-24T22:45:37.541 INFO:tasks.workunit.client.0.smithi063.stdout:6/817: mknod db/d85/dd2/dd7/c111 0 2024-04-24T22:45:37.543 INFO:tasks.workunit.client.1.smithi192.stdout:8/597: write db/dc/d22/d31/f4c [814953,121215] 0 2024-04-24T22:45:37.544 INFO:tasks.workunit.client.0.smithi063.stdout:3/772: rename da/d18/d47/d4d/c5b to da/d18/dfb/c10b 0 2024-04-24T22:45:37.545 INFO:tasks.workunit.client.0.smithi063.stdout:3/773: write da/d18/d47/f6a [3073297,33364] 0 2024-04-24T22:45:37.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/662: dwrite d2/d62/f7d [4194304,4194304] 0 2024-04-24T22:45:37.548 INFO:tasks.workunit.client.1.smithi192.stdout:6/540: write d0/d8/f33 [863247,79241] 0 2024-04-24T22:45:37.549 INFO:tasks.workunit.client.1.smithi192.stdout:4/597: write d2/d9/d25/d2e/f66 [503977,98682] 0 2024-04-24T22:45:37.550 INFO:tasks.workunit.client.0.smithi063.stdout:9/770: getdents d6/d2f/d42/d81 0 2024-04-24T22:45:37.550 INFO:tasks.workunit.client.1.smithi192.stdout:6/541: chown d0/d8/d18/d2e/c4a 5 1 2024-04-24T22:45:37.551 INFO:tasks.workunit.client.1.smithi192.stdout:1/516: dread d9/f2c [0,4194304] 0 2024-04-24T22:45:37.552 INFO:tasks.workunit.client.1.smithi192.stdout:8/598: mkdir db/dc/d22/d31/d33/db7 0 2024-04-24T22:45:37.554 INFO:tasks.workunit.client.0.smithi063.stdout:1/817: unlink d3/da/d34/d6d/fb6 0 2024-04-24T22:45:37.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/771: dwrite d6/d2f/d42/d81/fc6 [0,4194304] 0 2024-04-24T22:45:37.559 INFO:tasks.workunit.client.0.smithi063.stdout:9/772: stat d6/d2f/dcd/l2e 0 2024-04-24T22:45:37.560 INFO:tasks.workunit.client.1.smithi192.stdout:7/643: truncate d2/db/d21/d42/f8c 4134672 0 2024-04-24T22:45:37.561 INFO:tasks.workunit.client.1.smithi192.stdout:6/542: mknod d0/d8/d29/d5a/cbf 0 2024-04-24T22:45:37.564 INFO:tasks.workunit.client.1.smithi192.stdout:1/517: mkdir d9/d15/d2d/d31/d2f/dae 0 2024-04-24T22:45:37.565 INFO:tasks.workunit.client.1.smithi192.stdout:8/599: read db/dc/f24 [1396018,6897] 0 2024-04-24T22:45:37.566 INFO:tasks.workunit.client.1.smithi192.stdout:1/518: chown d9/dd/c45 12666444 1 2024-04-24T22:45:37.567 INFO:tasks.workunit.client.0.smithi063.stdout:4/737: creat d2/ff0 x:0 0 0 2024-04-24T22:45:37.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/702: getdents d6/d64/da0/d98 0 2024-04-24T22:45:37.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/703: stat d6/d64/d21/d9c/cbf 0 2024-04-24T22:45:37.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/704: write d6/d3a/fee [102023,91458] 0 2024-04-24T22:45:37.579 INFO:tasks.workunit.client.0.smithi063.stdout:5/739: link d5/d1d/dc0/dbb/l4b d5/d1d/dc7/lff 0 2024-04-24T22:45:37.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/740: readlink d5/l8 0 2024-04-24T22:45:37.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/791: dread d1/d22/f77 [4194304,4194304] 0 2024-04-24T22:45:37.582 INFO:tasks.workunit.client.1.smithi192.stdout:3/663: dread d2/f76 [0,4194304] 0 2024-04-24T22:45:37.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/792: read d1/d22/d33/da6/d54/d7b/f7f [292136,81531] 0 2024-04-24T22:45:37.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/793: stat d1/d11/fe4 0 2024-04-24T22:45:37.586 INFO:tasks.workunit.client.0.smithi063.stdout:7/701: sync 2024-04-24T22:45:37.586 INFO:tasks.workunit.client.1.smithi192.stdout:1/519: truncate d9/d15/d2d/d31/d2f/f73 150037 0 2024-04-24T22:45:37.586 INFO:tasks.workunit.client.0.smithi063.stdout:7/702: chown d0/d4/de/d17/d35/d69/fbc 88 1 2024-04-24T22:45:37.590 INFO:tasks.workunit.client.0.smithi063.stdout:2/862: mknod d0/d2/d2d/d31/d89/c10f 0 2024-04-24T22:45:37.591 INFO:tasks.workunit.client.0.smithi063.stdout:3/774: write da/d18/d47/d4d/f8c [459670,18157] 0 2024-04-24T22:45:37.592 INFO:tasks.workunit.client.0.smithi063.stdout:6/818: symlink db/d1d/d8d/dce/l112 0 2024-04-24T22:45:37.593 INFO:tasks.workunit.client.0.smithi063.stdout:6/819: readlink db/d1c/dea/lbe 0 2024-04-24T22:45:37.594 INFO:tasks.workunit.client.1.smithi192.stdout:1/520: dwrite d9/d15/f49 [0,4194304] 0 2024-04-24T22:45:37.596 INFO:tasks.workunit.client.1.smithi192.stdout:3/664: mknod d2/d4/cd1 0 2024-04-24T22:45:37.597 INFO:tasks.workunit.client.1.smithi192.stdout:7/644: truncate d2/db/d2a/d57/fa3 2635501 0 2024-04-24T22:45:37.598 INFO:tasks.workunit.client.1.smithi192.stdout:6/543: dwrite d0/d8/f91 [0,4194304] 0 2024-04-24T22:45:37.598 INFO:tasks.workunit.client.1.smithi192.stdout:7/645: chown d2/lc0 17207753 1 2024-04-24T22:45:37.600 INFO:tasks.workunit.client.1.smithi192.stdout:8/600: link db/dc/d29/l4b db/d75/lb8 0 2024-04-24T22:45:37.604 INFO:tasks.workunit.client.1.smithi192.stdout:8/601: rename db/dc/d22 to db/dc/d22/d31/d33/db9 22 2024-04-24T22:45:37.605 INFO:tasks.workunit.client.1.smithi192.stdout:8/602: chown db/d43/f4e 4270 1 2024-04-24T22:45:37.608 INFO:tasks.workunit.client.0.smithi063.stdout:9/773: dwrite d6/d2f/dcd/db/d93/da0/fac [0,4194304] 0 2024-04-24T22:45:37.608 INFO:tasks.workunit.client.1.smithi192.stdout:6/544: dwrite d0/d8/d29/f3b [0,4194304] 0 2024-04-24T22:45:37.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/705: creat d6/d64/d21/d25/d31/da3/ff0 x:0 0 0 2024-04-24T22:45:37.611 INFO:tasks.workunit.client.0.smithi063.stdout:1/818: symlink d3/db5/d21/l10b 0 2024-04-24T22:45:37.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/521: mknod d9/dd/caf 0 2024-04-24T22:45:37.616 INFO:tasks.workunit.client.1.smithi192.stdout:0/613: dwrite d3/d27/d28/d2e/f4c [0,4194304] 0 2024-04-24T22:45:37.621 INFO:tasks.workunit.client.0.smithi063.stdout:6/820: mknod db/d85/dd2/dd7/c113 0 2024-04-24T22:45:37.622 INFO:tasks.workunit.client.0.smithi063.stdout:4/738: fsync d2/d19/d33/dc1/d50/f5f 0 2024-04-24T22:45:37.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/545: dread - d0/f7f zero size 2024-04-24T22:45:37.647 INFO:tasks.workunit.client.0.smithi063.stdout:9/774: creat d6/d2f/dcd/db/d1c/ffe x:0 0 0 2024-04-24T22:45:37.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/603: symlink db/d5d/d74/d96/lba 0 2024-04-24T22:45:37.656 INFO:tasks.workunit.client.1.smithi192.stdout:1/522: fdatasync d9/f42 0 2024-04-24T22:45:37.660 INFO:tasks.workunit.client.1.smithi192.stdout:6/546: dwrite d0/d8/f2b [0,4194304] 0 2024-04-24T22:45:37.660 INFO:tasks.workunit.client.0.smithi063.stdout:7/703: getdents d0/d4/de/d17/d35/de4 0 2024-04-24T22:45:37.661 INFO:tasks.workunit.client.0.smithi063.stdout:2/863: sync 2024-04-24T22:45:37.664 INFO:tasks.workunit.client.0.smithi063.stdout:3/775: creat da/d39/d94/dbc/f10c x:0 0 0 2024-04-24T22:45:37.669 INFO:tasks.workunit.client.0.smithi063.stdout:4/739: symlink d2/d8/d40/d51/dd5/lf1 0 2024-04-24T22:45:37.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/864: creat d0/d2/d2d/d31/d89/d1c/d3f/f110 x:0 0 0 2024-04-24T22:45:37.675 INFO:tasks.workunit.client.0.smithi063.stdout:5/741: dwrite d5/d24/d36/f8c [0,4194304] 0 2024-04-24T22:45:37.676 INFO:tasks.workunit.client.0.smithi063.stdout:9/775: mkdir d6/d2f/dcd/d53/dff 0 2024-04-24T22:45:37.678 INFO:tasks.workunit.client.0.smithi063.stdout:8/794: truncate d1/d11/d8b/fa9 781376 0 2024-04-24T22:45:37.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/704: truncate d0/d4/de/f25 3036259 0 2024-04-24T22:45:37.680 INFO:tasks.workunit.client.0.smithi063.stdout:5/742: chown d5/d11/d23/d2d/d62/d81/da4/fd4 486092 1 2024-04-24T22:45:37.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/740: dwrite d2/d8/d40/da0/fc2 [4194304,4194304] 0 2024-04-24T22:45:37.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/819: dread - d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/fd3 zero size 2024-04-24T22:45:37.683 INFO:tasks.workunit.client.0.smithi063.stdout:7/705: readlink d0/d4/de/d17/d35/l83 0 2024-04-24T22:45:37.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/776: unlink da/d18/d3a/f96 0 2024-04-24T22:45:37.684 INFO:tasks.workunit.client.0.smithi063.stdout:0/706: link d6/d3a/fb0 d6/d64/da0/ff1 0 2024-04-24T22:45:37.685 INFO:tasks.workunit.client.0.smithi063.stdout:2/865: symlink d0/dd/d72/d77/l111 0 2024-04-24T22:45:37.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/707: write d6/d64/d21/d25/d31/da3/ff0 [487324,46950] 0 2024-04-24T22:45:37.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/776: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d100 0 2024-04-24T22:45:37.689 INFO:tasks.workunit.client.0.smithi063.stdout:2/866: dread - d0/d2/d2d/d31/d89/d7/d86/d8e/f108 zero size 2024-04-24T22:45:37.691 INFO:tasks.workunit.client.0.smithi063.stdout:9/777: dread - d6/d2f/dcd/db/d6b/d4f/dad/d36/fe4 zero size 2024-04-24T22:45:37.692 INFO:tasks.workunit.client.0.smithi063.stdout:6/821: dwrite db/d1d/d2d/f76 [0,4194304] 0 2024-04-24T22:45:37.694 INFO:tasks.workunit.client.0.smithi063.stdout:4/741: creat d2/d8/d40/d51/dd5/ff2 x:0 0 0 2024-04-24T22:45:37.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/795: read d1/d22/d33/da6/fa2 [920605,19288] 0 2024-04-24T22:45:37.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/777: write da/d39/d94/dbc/fd3 [2056008,29395] 0 2024-04-24T22:45:37.697 INFO:tasks.workunit.client.0.smithi063.stdout:0/708: mknod d6/d64/d21/d9c/cf2 0 2024-04-24T22:45:37.699 INFO:tasks.workunit.client.0.smithi063.stdout:5/743: symlink d5/d11/d23/l100 0 2024-04-24T22:45:37.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/867: rename d0/fbc to d0/d2/d35/dc6/de2/de4/f112 0 2024-04-24T22:45:37.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/778: creat d6/d2f/dcd/db/d1c/d78/d9a/daa/f101 x:0 0 0 2024-04-24T22:45:37.702 INFO:tasks.workunit.client.0.smithi063.stdout:7/706: mknod d0/cea 0 2024-04-24T22:45:37.704 INFO:tasks.workunit.client.0.smithi063.stdout:7/707: chown d0/d4/de/d17/d35/d69/db1/fd3 3 1 2024-04-24T22:45:37.706 INFO:tasks.workunit.client.0.smithi063.stdout:9/779: symlink d6/d2f/dcd/db/dca/l102 0 2024-04-24T22:45:37.707 INFO:tasks.workunit.client.0.smithi063.stdout:8/796: link d1/d2/fba d1/d2/ddf/f101 0 2024-04-24T22:45:37.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/742: dread d2/d8/f1a [0,4194304] 0 2024-04-24T22:45:37.712 INFO:tasks.workunit.client.0.smithi063.stdout:5/744: getdents d5/d24/d44/dc9 0 2024-04-24T22:45:37.714 INFO:tasks.workunit.client.0.smithi063.stdout:3/778: dwrite da/d48/fd7 [4194304,4194304] 0 2024-04-24T22:45:37.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/743: write d2/d19/d33/dc1/d50/f9e [431665,6730] 0 2024-04-24T22:45:37.715 INFO:tasks.workunit.client.0.smithi063.stdout:0/709: creat d6/d64/d21/d25/d62/d63/d89/ff3 x:0 0 0 2024-04-24T22:45:37.716 INFO:tasks.workunit.client.0.smithi063.stdout:8/797: unlink d1/d22/d33/da6/d54/d79/ld5 0 2024-04-24T22:45:37.719 INFO:tasks.workunit.client.0.smithi063.stdout:9/780: truncate d6/d2f/d42/f6f 4512770 0 2024-04-24T22:45:37.719 INFO:tasks.workunit.client.0.smithi063.stdout:8/798: dread - d1/d11/fe4 zero size 2024-04-24T22:45:37.720 INFO:tasks.workunit.client.0.smithi063.stdout:6/822: rename db/d1d/d20/d3e/f43 to db/d1d/d2d/d69/f114 0 2024-04-24T22:45:37.722 INFO:tasks.workunit.client.0.smithi063.stdout:7/708: symlink d0/d4/de/d17/d28/d31/dad/de7/leb 0 2024-04-24T22:45:37.727 INFO:tasks.workunit.client.0.smithi063.stdout:5/745: dwrite d5/d24/d49/fdf [0,4194304] 0 2024-04-24T22:45:37.730 INFO:tasks.workunit.client.1.smithi192.stdout:1/523: dread - d9/d15/d2d/d63/d68/f76 zero size 2024-04-24T22:45:37.730 INFO:tasks.workunit.client.1.smithi192.stdout:8/604: dwrite db/dc/d22/d31/f45 [0,4194304] 0 2024-04-24T22:45:37.731 INFO:tasks.workunit.client.0.smithi063.stdout:1/820: dread d3/db5/d21/d9f/dd5/d106/d39/d5a/f8f [0,4194304] 0 2024-04-24T22:45:37.732 INFO:tasks.workunit.client.0.smithi063.stdout:7/709: write d0/d4/f11 [2312817,35663] 0 2024-04-24T22:45:37.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/781: fsync d6/d2f/dcd/d1f/d20/d2a/dcb/f68 0 2024-04-24T22:45:37.732 INFO:tasks.workunit.client.1.smithi192.stdout:6/547: write d0/d8/d29/d58/fbe [959416,54876] 0 2024-04-24T22:45:37.733 INFO:tasks.workunit.client.1.smithi192.stdout:1/524: fsync d9/d15/d2d/f6c 0 2024-04-24T22:45:37.735 INFO:tasks.workunit.client.0.smithi063.stdout:0/710: dwrite d6/d64/da0/d33/fc3 [0,4194304] 0 2024-04-24T22:45:37.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/779: mknod da/d18/d47/d4d/d9b/dde/df9/c10d 0 2024-04-24T22:45:37.736 INFO:tasks.workunit.client.0.smithi063.stdout:8/799: dwrite d1/d2/d3/d86/ff2 [0,4194304] 0 2024-04-24T22:45:37.736 INFO:tasks.workunit.client.0.smithi063.stdout:7/710: fsync d0/d4/de/d17/d28/f64 0 2024-04-24T22:45:37.737 INFO:tasks.workunit.client.1.smithi192.stdout:1/525: read - d9/d51/f9c zero size 2024-04-24T22:45:37.738 INFO:tasks.workunit.client.0.smithi063.stdout:8/800: chown d1/d21/c8f 1254 1 2024-04-24T22:45:37.742 INFO:tasks.workunit.client.1.smithi192.stdout:7/646: dwrite d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:37.745 INFO:tasks.workunit.client.0.smithi063.stdout:6/823: truncate db/d1d/d20/d3f/d7f/f97 183223 0 2024-04-24T22:45:37.746 INFO:tasks.workunit.client.0.smithi063.stdout:8/801: dwrite d1/d22/d33/da6/d54/f71 [4194304,4194304] 0 2024-04-24T22:45:37.746 INFO:tasks.workunit.client.0.smithi063.stdout:6/824: write db/fdf [5039004,84593] 0 2024-04-24T22:45:37.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/647: write d2/db/d21/d42/d67/dae/fc8 [677562,108039] 0 2024-04-24T22:45:37.748 INFO:tasks.workunit.client.1.smithi192.stdout:0/614: dread d3/d27/d28/f1d [0,4194304] 0 2024-04-24T22:45:37.751 INFO:tasks.workunit.client.1.smithi192.stdout:8/605: dwrite db/d43/d76/d92/f9b [0,4194304] 0 2024-04-24T22:45:37.751 INFO:tasks.workunit.client.0.smithi063.stdout:3/780: dread da/d18/d3a/f104 [0,4194304] 0 2024-04-24T22:45:37.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/781: write da/d18/d47/d4d/f8c [575090,84021] 0 2024-04-24T22:45:37.765 INFO:tasks.workunit.client.0.smithi063.stdout:5/746: mknod d5/d11/d23/d2d/d62/d81/da4/da6/c101 0 2024-04-24T22:45:37.768 INFO:tasks.workunit.client.1.smithi192.stdout:6/548: unlink d0/d8/d29/d4c/l5e 0 2024-04-24T22:45:37.770 INFO:tasks.workunit.client.1.smithi192.stdout:6/549: write d0/d8/d29/f3b [3284457,94551] 0 2024-04-24T22:45:37.770 INFO:tasks.workunit.client.1.smithi192.stdout:7/648: creat d2/db/d21/d42/d5b/d63/d80/fcc x:0 0 0 2024-04-24T22:45:37.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/782: mknod d6/d2f/dcd/d1f/d20/d2a/dcb/c103 0 2024-04-24T22:45:37.776 INFO:tasks.workunit.client.0.smithi063.stdout:9/783: write d6/d2f/dcd/d1f/d20/fbc [4358360,433] 0 2024-04-24T22:45:37.776 INFO:tasks.workunit.client.1.smithi192.stdout:1/526: truncate d9/f38 46624 0 2024-04-24T22:45:37.777 INFO:tasks.workunit.client.0.smithi063.stdout:1/821: dwrite d3/db5/d21/d9f/dd5/d106/f100 [4194304,4194304] 0 2024-04-24T22:45:37.781 INFO:tasks.workunit.client.1.smithi192.stdout:0/615: mknod d3/d27/d28/dcc/d5c/d80/cd2 0 2024-04-24T22:45:37.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/606: symlink db/dc/d9a/lbb 0 2024-04-24T22:45:37.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/802: unlink d1/d2/fbc 0 2024-04-24T22:45:37.791 INFO:tasks.workunit.client.0.smithi063.stdout:3/782: mkdir da/d39/d94/db3/df3/d10e 0 2024-04-24T22:45:37.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/607: creat db/dc/d22/d31/d33/fbc x:0 0 0 2024-04-24T22:45:37.793 INFO:tasks.workunit.client.0.smithi063.stdout:5/747: symlink d5/d24/d49/d3b/dc5/dda/l102 0 2024-04-24T22:45:37.796 INFO:tasks.workunit.client.1.smithi192.stdout:1/527: rename d9/d15/d5e/l8a to d9/d15/d2d/lb0 0 2024-04-24T22:45:37.797 INFO:tasks.workunit.client.1.smithi192.stdout:1/528: stat d9/d51/d70 0 2024-04-24T22:45:37.797 INFO:tasks.workunit.client.1.smithi192.stdout:8/608: mknod db/dc/cbd 0 2024-04-24T22:45:37.799 INFO:tasks.workunit.client.0.smithi063.stdout:9/784: unlink d6/d2f/dcd/d1f/c60 0 2024-04-24T22:45:37.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/609: dread db/d43/d76/f94 [0,4194304] 0 2024-04-24T22:45:37.803 INFO:tasks.workunit.client.0.smithi063.stdout:8/803: fdatasync d1/d2/fba 0 2024-04-24T22:45:37.806 INFO:tasks.workunit.client.0.smithi063.stdout:5/748: rename d5/d11/d23/d2d/d62/d73/dcd to d5/d11/d23/d2d/d62/d81/da4/da6/d103 0 2024-04-24T22:45:37.807 INFO:tasks.workunit.client.1.smithi192.stdout:8/610: mknod db/d43/d76/d92/dad/cbe 0 2024-04-24T22:45:37.809 INFO:tasks.workunit.client.0.smithi063.stdout:4/744: link d2/d19/dce/lcf d2/d19/lf3 0 2024-04-24T22:45:37.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/804: creat d1/dc0/f102 x:0 0 0 2024-04-24T22:45:37.812 INFO:tasks.workunit.client.0.smithi063.stdout:2/868: sync 2024-04-24T22:45:37.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/711: dread d6/d64/d21/d25/d62/fc8 [0,4194304] 0 2024-04-24T22:45:37.813 INFO:tasks.workunit.client.0.smithi063.stdout:4/745: creat d2/d8/dd4/dda/dc6/ff4 x:0 0 0 2024-04-24T22:45:37.814 INFO:tasks.workunit.client.1.smithi192.stdout:8/611: getdents db/dc/d22/d31/d33/d8f 0 2024-04-24T22:45:37.815 INFO:tasks.workunit.client.1.smithi192.stdout:8/612: chown db/d43/c8d 585253681 1 2024-04-24T22:45:37.816 INFO:tasks.workunit.client.0.smithi063.stdout:8/805: truncate d1/d11/d32/fb2 420016 0 2024-04-24T22:45:37.816 INFO:tasks.workunit.client.0.smithi063.stdout:2/869: write d0/d2/d2d/d31/d89/d7/d86/da4/dd9/fea [5234871,29394] 0 2024-04-24T22:45:37.817 INFO:tasks.workunit.client.0.smithi063.stdout:2/870: dread - d0/d2/d2d/d31/d89/d1c/d3f/fff zero size 2024-04-24T22:45:37.817 INFO:tasks.workunit.client.0.smithi063.stdout:0/712: creat d6/d64/d24/d42/dd4/ff4 x:0 0 0 2024-04-24T22:45:37.819 INFO:tasks.workunit.client.0.smithi063.stdout:2/871: chown d0/d2/d2d/d31/d89/d7/d86/d8e 203477221 1 2024-04-24T22:45:37.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/713: creat d6/d64/d24/d42/dd2/ff5 x:0 0 0 2024-04-24T22:45:37.825 INFO:tasks.workunit.client.0.smithi063.stdout:4/746: truncate d2/d8/d5c/fe2 151098 0 2024-04-24T22:45:37.826 INFO:tasks.workunit.client.0.smithi063.stdout:4/747: write d2/d8/dd4/dda/fef [293564,76196] 0 2024-04-24T22:45:37.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/714: unlink d6/d64/da0/c3e 0 2024-04-24T22:45:37.827 INFO:tasks.workunit.client.0.smithi063.stdout:0/715: readlink d6/da/d2b/l37 0 2024-04-24T22:45:37.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/806: dread d1/d11/d32/d38/f3f [0,4194304] 0 2024-04-24T22:45:37.829 INFO:tasks.workunit.client.1.smithi192.stdout:7/649: write d2/db/d21/f39 [1167535,1050] 0 2024-04-24T22:45:37.833 INFO:tasks.workunit.client.1.smithi192.stdout:7/650: creat d2/db/d21/d42/d67/d9b/fcd x:0 0 0 2024-04-24T22:45:37.835 INFO:tasks.workunit.client.1.smithi192.stdout:0/616: write d3/d27/f30 [1158123,36381] 0 2024-04-24T22:45:37.838 INFO:tasks.workunit.client.1.smithi192.stdout:0/617: write d3/d27/d3c/d6c/db3/f86 [1367397,51370] 0 2024-04-24T22:45:37.842 INFO:tasks.workunit.client.1.smithi192.stdout:1/529: write d9/d51/f54 [1115209,78204] 0 2024-04-24T22:45:37.846 INFO:tasks.workunit.client.1.smithi192.stdout:1/530: rename d9/d15/d2d/d63/d68 to d9/d15/d2d/d63/d68/d9d/db1 22 2024-04-24T22:45:37.846 INFO:tasks.workunit.client.1.smithi192.stdout:0/618: dwrite d3/d27/d28/dcc/fbb [0,4194304] 0 2024-04-24T22:45:37.850 INFO:tasks.workunit.client.0.smithi063.stdout:4/748: creat d2/d19/d33/d9d/da8/dec/ded/ff5 x:0 0 0 2024-04-24T22:45:37.852 INFO:tasks.workunit.client.0.smithi063.stdout:9/785: dread d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/fc7 [4194304,4194304] 0 2024-04-24T22:45:37.854 INFO:tasks.workunit.client.1.smithi192.stdout:1/531: creat d9/d15/d2d/d63/fb2 x:0 0 0 2024-04-24T22:45:37.856 INFO:tasks.workunit.client.1.smithi192.stdout:0/619: dwrite d3/d27/d3c/d6c/db3/f86 [0,4194304] 0 2024-04-24T22:45:37.857 INFO:tasks.workunit.client.1.smithi192.stdout:0/620: chown d3/d27/d3c 260670 1 2024-04-24T22:45:37.861 INFO:tasks.workunit.client.1.smithi192.stdout:1/532: mknod d9/d15/d5e/d90/cb3 0 2024-04-24T22:45:37.861 INFO:tasks.workunit.client.0.smithi063.stdout:9/786: write d6/d2f/dcd/d53/f7a [1420515,82166] 0 2024-04-24T22:45:37.861 INFO:tasks.workunit.client.1.smithi192.stdout:8/613: dwrite db/d43/d71/f99 [0,4194304] 0 2024-04-24T22:45:37.864 INFO:tasks.workunit.client.1.smithi192.stdout:7/651: write d2/f2c [5016372,1002] 0 2024-04-24T22:45:37.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/621: creat d3/d27/d6a/dc4/fd3 x:0 0 0 2024-04-24T22:45:37.869 INFO:tasks.workunit.client.0.smithi063.stdout:1/822: dwrite d3/de0/d79/ff0 [0,4194304] 0 2024-04-24T22:45:37.869 INFO:tasks.workunit.client.0.smithi063.stdout:3/783: dwrite da/d18/d47/f6c [0,4194304] 0 2024-04-24T22:45:37.870 INFO:tasks.workunit.client.1.smithi192.stdout:7/652: mkdir d2/db/d21/d42/d67/dae/dce 0 2024-04-24T22:45:37.870 INFO:tasks.workunit.client.1.smithi192.stdout:7/653: stat d2/db/l56 0 2024-04-24T22:45:37.872 INFO:tasks.workunit.client.1.smithi192.stdout:7/654: dread - d2/db/d21/fc6 zero size 2024-04-24T22:45:37.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/749: dwrite d2/f6 [4194304,4194304] 0 2024-04-24T22:45:37.880 INFO:tasks.workunit.client.0.smithi063.stdout:9/787: creat d6/d2f/dcd/d53/dd5/f104 x:0 0 0 2024-04-24T22:45:37.881 INFO:tasks.workunit.client.0.smithi063.stdout:0/716: dread d6/d64/d24/d42/dd4/fa5 [0,4194304] 0 2024-04-24T22:45:37.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/711: sync 2024-04-24T22:45:37.882 INFO:tasks.workunit.client.1.smithi192.stdout:7/655: mknod d2/db/d21/d42/d67/d9b/d6e/db5/db6/ccf 0 2024-04-24T22:45:37.882 INFO:tasks.workunit.client.0.smithi063.stdout:1/823: fdatasync d3/db5/d21/d9f/dd5/d106/f3a 0 2024-04-24T22:45:37.885 INFO:tasks.workunit.client.0.smithi063.stdout:8/807: dread d1/d11/d8b/dd6/ff0 [0,4194304] 0 2024-04-24T22:45:37.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/750: rename d2/d8/dd4/dda/dc6/dea to d2/d8/d40/d51/dd5/df6 0 2024-04-24T22:45:37.890 INFO:tasks.workunit.client.0.smithi063.stdout:0/717: chown d6/d3a/fb0 433 1 2024-04-24T22:45:37.890 INFO:tasks.workunit.client.1.smithi192.stdout:7/656: dwrite d2/f2c [4194304,4194304] 0 2024-04-24T22:45:37.891 INFO:tasks.workunit.client.0.smithi063.stdout:0/718: chown d6/d64/d24/d42/l4b 236382904 1 2024-04-24T22:45:37.895 INFO:tasks.workunit.client.0.smithi063.stdout:9/788: creat d6/d2f/dcd/db/d6b/d4f/dad/f105 x:0 0 0 2024-04-24T22:45:37.899 INFO:tasks.workunit.client.1.smithi192.stdout:7/657: fdatasync d2/db/d36/f64 0 2024-04-24T22:45:37.901 INFO:tasks.workunit.client.1.smithi192.stdout:7/658: read d2/db/d21/d42/f4d [175213,38584] 0 2024-04-24T22:45:37.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/614: dread db/dc/d29/f47 [0,4194304] 0 2024-04-24T22:45:37.902 INFO:tasks.workunit.client.1.smithi192.stdout:8/615: stat db/dc/d29/f2b 0 2024-04-24T22:45:37.903 INFO:tasks.workunit.client.0.smithi063.stdout:8/808: creat d1/d2/d17/dd1/f103 x:0 0 0 2024-04-24T22:45:37.904 INFO:tasks.workunit.client.0.smithi063.stdout:8/809: dread - d1/d2/fba zero size 2024-04-24T22:45:37.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/751: chown d2/d19/d33/dc1/d50/c5e 173 1 2024-04-24T22:45:37.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/752: chown d2/d8/dee 51825 1 2024-04-24T22:45:37.912 INFO:tasks.workunit.client.1.smithi192.stdout:1/533: dwrite d9/f3c [4194304,4194304] 0 2024-04-24T22:45:37.913 INFO:tasks.workunit.client.0.smithi063.stdout:9/789: creat d6/d2f/dcd/db/d93/da0/f106 x:0 0 0 2024-04-24T22:45:37.914 INFO:tasks.workunit.client.1.smithi192.stdout:0/622: dwrite d3/f39 [0,4194304] 0 2024-04-24T22:45:37.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/712: write d0/f48 [281942,121551] 0 2024-04-24T22:45:37.915 INFO:tasks.workunit.client.1.smithi192.stdout:1/534: truncate d9/d15/d2d/d63/fb2 109819 0 2024-04-24T22:45:37.928 INFO:tasks.workunit.client.0.smithi063.stdout:7/713: truncate d0/d4/de/d6f/d5c/fcd 2861734 0 2024-04-24T22:45:37.934 INFO:tasks.workunit.client.0.smithi063.stdout:8/810: write d1/d2/f39 [1635126,108710] 0 2024-04-24T22:45:37.934 INFO:tasks.workunit.client.0.smithi063.stdout:9/790: getdents d6/d2f/dcd/d1f/d20/de6 0 2024-04-24T22:45:37.935 INFO:tasks.workunit.client.1.smithi192.stdout:9/697: sync 2024-04-24T22:45:37.935 INFO:tasks.workunit.client.0.smithi063.stdout:3/784: getdents da/d18/d3a/d63 0 2024-04-24T22:45:37.937 INFO:tasks.workunit.client.0.smithi063.stdout:4/753: fdatasync d2/d8/d5c/fe2 0 2024-04-24T22:45:37.937 INFO:tasks.workunit.client.0.smithi063.stdout:6/825: sync 2024-04-24T22:45:37.938 INFO:tasks.workunit.client.0.smithi063.stdout:1/824: dwrite d3/db5/d21/d9f/dd5/d106/f8c [0,4194304] 0 2024-04-24T22:45:37.938 INFO:tasks.workunit.client.0.smithi063.stdout:7/714: fdatasync d0/d4/de/d6f/f5b 0 2024-04-24T22:45:37.939 INFO:tasks.workunit.client.0.smithi063.stdout:3/785: dread - da/d18/d47/d4d/f106 zero size 2024-04-24T22:45:37.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/826: stat db/d1d/d20/d29/l8a 0 2024-04-24T22:45:37.941 INFO:tasks.workunit.client.0.smithi063.stdout:0/719: getdents d6/d64/d24/d42 0 2024-04-24T22:45:37.942 INFO:tasks.workunit.client.0.smithi063.stdout:3/786: chown da/d18/f2a 42 1 2024-04-24T22:45:37.945 INFO:tasks.workunit.client.0.smithi063.stdout:8/811: mknod d1/d22/d33/c104 0 2024-04-24T22:45:37.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/754: dwrite d2/d19/d33/dc1/d50/f9e [4194304,4194304] 0 2024-04-24T22:45:37.951 INFO:tasks.workunit.client.0.smithi063.stdout:8/812: dwrite d1/d2/d3/db6/fbb [0,4194304] 0 2024-04-24T22:45:37.960 INFO:tasks.workunit.client.1.smithi192.stdout:7/659: chown d2/db/d21/d42/d67/da8/dc7/lba 7497574 1 2024-04-24T22:45:37.961 INFO:tasks.workunit.client.0.smithi063.stdout:7/715: dwrite d0/d4/de/d6f/f5b [0,4194304] 0 2024-04-24T22:45:37.963 INFO:tasks.workunit.client.0.smithi063.stdout:6/827: read db/d1d/d20/d3e/f5e [1057287,46459] 0 2024-04-24T22:45:37.964 INFO:tasks.workunit.client.1.smithi192.stdout:0/623: mkdir d3/d27/d28/d2e/dc7/dd4 0 2024-04-24T22:45:37.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/755: symlink d2/d8/d40/da0/lf7 0 2024-04-24T22:45:37.965 INFO:tasks.workunit.client.0.smithi063.stdout:6/828: write db/d1d/d20/d3e/f45 [1563953,82366] 0 2024-04-24T22:45:37.965 INFO:tasks.workunit.client.1.smithi192.stdout:9/698: mkdir d3/d5d/da1/df8 0 2024-04-24T22:45:37.967 INFO:tasks.workunit.client.0.smithi063.stdout:9/791: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d107 0 2024-04-24T22:45:37.971 INFO:tasks.workunit.client.1.smithi192.stdout:7/660: unlink d2/db/d21/d42/d5b/c74 0 2024-04-24T22:45:37.974 INFO:tasks.workunit.client.1.smithi192.stdout:0/624: creat d3/d6d/d79/fd5 x:0 0 0 2024-04-24T22:45:37.975 INFO:tasks.workunit.client.1.smithi192.stdout:0/625: dread - d3/d27/d3c/d3b/fd1 zero size 2024-04-24T22:45:37.976 INFO:tasks.workunit.client.1.smithi192.stdout:9/699: mknod d3/dd/de/d17/d89/db4/db9/cf9 0 2024-04-24T22:45:37.979 INFO:tasks.workunit.client.1.smithi192.stdout:7/661: symlink d2/db/d21/d42/d67/dae/ld0 0 2024-04-24T22:45:37.981 INFO:tasks.workunit.client.0.smithi063.stdout:2/872: sync 2024-04-24T22:45:37.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/749: sync 2024-04-24T22:45:37.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/720: mkdir d6/d64/d21/d25/d62/d63/d89/d95/df6 0 2024-04-24T22:45:37.983 INFO:tasks.workunit.client.1.smithi192.stdout:9/700: symlink d3/dd/de/da7/dae/lfa 0 2024-04-24T22:45:37.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/756: mkdir d2/d9f/df8 0 2024-04-24T22:45:37.989 INFO:tasks.workunit.client.0.smithi063.stdout:1/825: write d3/db5/d21/d9f/dd5/d106/d39/f66 [2584103,22587] 0 2024-04-24T22:45:37.992 INFO:tasks.workunit.client.0.smithi063.stdout:4/757: dread d2/d19/d33/d9d/fbf [0,4194304] 0 2024-04-24T22:45:37.993 INFO:tasks.workunit.client.0.smithi063.stdout:9/792: dread - d6/d2f/dcd/d1f/d20/d22/fb2 zero size 2024-04-24T22:45:37.993 INFO:tasks.workunit.client.0.smithi063.stdout:1/826: dwrite f1 [0,4194304] 0 2024-04-24T22:45:37.995 INFO:tasks.workunit.client.0.smithi063.stdout:8/813: fsync d1/d22/d33/da6/f7c 0 2024-04-24T22:45:37.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/827: readlink d3/db5/d21/d9f/dd5/d106/d11/dc6/lc9 0 2024-04-24T22:45:37.996 INFO:tasks.workunit.client.1.smithi192.stdout:0/626: link d3/d27/d3c/caf d3/d27/d28/dba/cd6 0 2024-04-24T22:45:37.997 INFO:tasks.workunit.client.1.smithi192.stdout:7/662: dread d2/db/d25/f69 [0,4194304] 0 2024-04-24T22:45:37.998 INFO:tasks.workunit.client.1.smithi192.stdout:7/663: chown d2/db/d21/d42/d67/d9b/d6e/db5/db6 860502081 1 2024-04-24T22:45:38.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/787: truncate da/f2c 983244 0 2024-04-24T22:45:38.001 INFO:tasks.workunit.client.0.smithi063.stdout:2/873: rmdir d0/dd/d72/d77/dda 39 2024-04-24T22:45:38.003 INFO:tasks.workunit.client.0.smithi063.stdout:8/814: dwrite d1/d22/d33/da6/d54/f71 [4194304,4194304] 0 2024-04-24T22:45:38.004 INFO:tasks.workunit.client.0.smithi063.stdout:3/788: dread da/d18/d47/d4d/d62/f80 [4194304,4194304] 0 2024-04-24T22:45:38.004 INFO:tasks.workunit.client.1.smithi192.stdout:0/627: read d3/d6d/d79/f8b [1576355,128503] 0 2024-04-24T22:45:38.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/628: fsync d3/d27/d6a/f87 0 2024-04-24T22:45:38.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/721: fsync d6/d64/d21/d25/f55 0 2024-04-24T22:45:38.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/716: mknod d0/d4/de/d17/cec 0 2024-04-24T22:45:38.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/664: read d2/db/f10 [1127295,73865] 0 2024-04-24T22:45:38.010 INFO:tasks.workunit.client.1.smithi192.stdout:9/701: write d3/dd/de/d17/d89/d8b/f99 [263839,129477] 0 2024-04-24T22:45:38.012 INFO:tasks.workunit.client.0.smithi063.stdout:6/829: dwrite db/d1d/d2d/f4d [0,4194304] 0 2024-04-24T22:45:38.033 INFO:tasks.workunit.client.1.smithi192.stdout:0/629: mkdir d3/d27/d28/d2e/d3f/dd7 0 2024-04-24T22:45:38.033 INFO:tasks.workunit.client.1.smithi192.stdout:0/630: chown d3/d27/d28/d2e 7739740 1 2024-04-24T22:45:38.033 INFO:tasks.workunit.client.1.smithi192.stdout:7/665: unlink d2/db/c81 0 2024-04-24T22:45:38.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/828: fsync d3/da/f6a 0 2024-04-24T22:45:38.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/666: truncate d2/db/d21/d42/f93 312557 0 2024-04-24T22:45:38.043 INFO:tasks.workunit.client.1.smithi192.stdout:0/631: creat d3/d27/d28/d31/dbc/fd8 x:0 0 0 2024-04-24T22:45:38.045 INFO:tasks.workunit.client.1.smithi192.stdout:7/667: unlink d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fbf 0 2024-04-24T22:45:38.046 INFO:tasks.workunit.client.0.smithi063.stdout:1/829: dwrite d3/db5/f22 [0,4194304] 0 2024-04-24T22:45:38.047 INFO:tasks.workunit.client.1.smithi192.stdout:7/668: write d2/db/d21/d42/d5b/d63/d80/fcc [970434,103085] 0 2024-04-24T22:45:38.048 INFO:tasks.workunit.client.1.smithi192.stdout:5/477: sync 2024-04-24T22:45:38.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/669: chown d2/db/d21/d42/d67/d9b/d6e/d77/l83 2044116876 1 2024-04-24T22:45:38.051 INFO:tasks.workunit.client.1.smithi192.stdout:7/670: fsync d2/db/d21/d42/d5b/f6b 0 2024-04-24T22:45:38.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/671: fdatasync d2/db/d21/d42/d67/d9b/d6e/db5/db6/f8d 0 2024-04-24T22:45:38.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/632: dwrite d3/d27/d28/f20 [4194304,4194304] 0 2024-04-24T22:45:38.060 INFO:tasks.workunit.client.1.smithi192.stdout:5/478: creat d0/d13/d19/d64/d5f/d2c/fa3 x:0 0 0 2024-04-24T22:45:38.065 INFO:tasks.workunit.client.1.smithi192.stdout:9/702: dwrite d3/d59/f5b [0,4194304] 0 2024-04-24T22:45:38.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/703: fdatasync d3/d59/ff5 0 2024-04-24T22:45:38.068 INFO:tasks.workunit.client.1.smithi192.stdout:7/672: creat d2/db/d21/d42/d67/d9b/d6e/da2/fd1 x:0 0 0 2024-04-24T22:45:38.069 INFO:tasks.workunit.client.1.smithi192.stdout:7/673: dread - d2/db/d21/fc6 zero size 2024-04-24T22:45:38.071 INFO:tasks.workunit.client.0.smithi063.stdout:4/758: sync 2024-04-24T22:45:38.071 INFO:tasks.workunit.client.0.smithi063.stdout:4/759: chown d2/f99 18495617 1 2024-04-24T22:45:38.073 INFO:tasks.workunit.client.1.smithi192.stdout:5/479: symlink d0/d13/d19/d71/d88/la4 0 2024-04-24T22:45:38.078 INFO:tasks.workunit.client.0.smithi063.stdout:2/874: creat d0/d2/d2f/de8/f113 x:0 0 0 2024-04-24T22:45:38.083 INFO:tasks.workunit.client.1.smithi192.stdout:9/704: rename d3/dd/de/d17/d89/de1/df7 to d3/d5d/da1/d5e/dfb 0 2024-04-24T22:45:38.084 INFO:tasks.workunit.client.0.smithi063.stdout:1/830: fsync d3/da/d34/fa1 0 2024-04-24T22:45:38.087 INFO:tasks.workunit.client.0.smithi063.stdout:3/789: creat da/d40/f10f x:0 0 0 2024-04-24T22:45:38.088 INFO:tasks.workunit.client.1.smithi192.stdout:7/674: mkdir d2/db/d21/d42/d67/d9b/d6e/d77/dd2 0 2024-04-24T22:45:38.088 INFO:tasks.workunit.client.0.smithi063.stdout:8/815: creat d1/d11/dcb/f105 x:0 0 0 2024-04-24T22:45:38.090 INFO:tasks.workunit.client.1.smithi192.stdout:5/480: creat d0/d13/d19/d71/d7f/d9a/fa5 x:0 0 0 2024-04-24T22:45:38.093 INFO:tasks.workunit.client.0.smithi063.stdout:0/722: unlink d6/da/dc/le4 0 2024-04-24T22:45:38.094 INFO:tasks.workunit.client.1.smithi192.stdout:9/705: read d3/dd/de/f1a [151351,38624] 0 2024-04-24T22:45:38.095 INFO:tasks.workunit.client.0.smithi063.stdout:7/717: mkdir d0/d4/de/ded 0 2024-04-24T22:45:38.096 INFO:tasks.workunit.client.1.smithi192.stdout:0/633: write d3/f47 [2127502,100919] 0 2024-04-24T22:45:38.097 INFO:tasks.workunit.client.0.smithi063.stdout:7/718: chown d0/d4/de/d17/d28/d31/dad/de7/leb 818 1 2024-04-24T22:45:38.097 INFO:tasks.workunit.client.1.smithi192.stdout:0/634: chown d3/d27/d6a/cca 0 1 2024-04-24T22:45:38.098 INFO:tasks.workunit.client.1.smithi192.stdout:0/635: fdatasync d3/d27/d3c/d64/f84 0 2024-04-24T22:45:38.101 INFO:tasks.workunit.client.0.smithi063.stdout:9/793: creat d6/d2f/dcd/f108 x:0 0 0 2024-04-24T22:45:38.103 INFO:tasks.workunit.client.1.smithi192.stdout:7/675: symlink d2/db/d21/d42/d67/d9b/dc1/ld3 0 2024-04-24T22:45:38.106 INFO:tasks.workunit.client.0.smithi063.stdout:5/750: link d5/d11/d23/d2d/cb7 d5/d11/c104 0 2024-04-24T22:45:38.107 INFO:tasks.workunit.client.0.smithi063.stdout:0/723: dread d6/d64/d24/f59 [4194304,4194304] 0 2024-04-24T22:45:38.107 INFO:tasks.workunit.client.1.smithi192.stdout:0/636: mkdir d3/d27/d28/dcc/d5c/d80/d83/dd9 0 2024-04-24T22:45:38.107 INFO:tasks.workunit.client.0.smithi063.stdout:5/751: chown d5/d11/d23/d2d/d62/la1 9606291 1 2024-04-24T22:45:38.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/637: chown d3/d27/d3c/d64/faa 1323035373 1 2024-04-24T22:45:38.109 INFO:tasks.workunit.client.0.smithi063.stdout:1/831: truncate d3/db5/d21/d9f/dd5/d106/d39/f65 4583155 0 2024-04-24T22:45:38.112 INFO:tasks.workunit.client.0.smithi063.stdout:3/790: creat da/d39/d94/f110 x:0 0 0 2024-04-24T22:45:38.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/638: readlink d3/d27/d28/d2e/d3f/lb2 0 2024-04-24T22:45:38.113 INFO:tasks.workunit.client.0.smithi063.stdout:3/791: fdatasync da/d18/d47/d4d/f8c 0 2024-04-24T22:45:38.115 INFO:tasks.workunit.client.0.smithi063.stdout:8/816: symlink d1/d11/d3c/l106 0 2024-04-24T22:45:38.116 INFO:tasks.workunit.client.0.smithi063.stdout:6/830: write db/d1d/d20/d29/f9f [1560662,114337] 0 2024-04-24T22:45:38.117 INFO:tasks.workunit.client.1.smithi192.stdout:5/481: write d0/d13/d19/d64/d5f/f20 [2514992,86659] 0 2024-04-24T22:45:38.121 INFO:tasks.workunit.client.1.smithi192.stdout:0/639: dwrite d3/d27/d28/d31/dbc/fd8 [0,4194304] 0 2024-04-24T22:45:38.121 INFO:tasks.workunit.client.1.smithi192.stdout:5/482: stat d0/d13/d53/f70 0 2024-04-24T22:45:38.122 INFO:tasks.workunit.client.0.smithi063.stdout:4/760: dread d2/d80/d8d/f76 [0,4194304] 0 2024-04-24T22:45:38.123 INFO:tasks.workunit.client.0.smithi063.stdout:0/724: mknod d6/d64/d24/d42/dd2/cf7 0 2024-04-24T22:45:38.125 INFO:tasks.workunit.client.0.smithi063.stdout:7/719: dwrite d0/d4/de/d17/f8a [0,4194304] 0 2024-04-24T22:45:38.126 INFO:tasks.workunit.client.1.smithi192.stdout:5/483: write d0/d5b/f7a [1181265,57383] 0 2024-04-24T22:45:38.130 INFO:tasks.workunit.client.1.smithi192.stdout:0/640: dwrite d3/fc0 [0,4194304] 0 2024-04-24T22:45:38.130 INFO:tasks.workunit.client.0.smithi063.stdout:3/792: mkdir da/d40/d111 0 2024-04-24T22:45:38.131 INFO:tasks.workunit.client.0.smithi063.stdout:1/832: dread d3/db5/d21/d9f/dd5/f102 [0,4194304] 0 2024-04-24T22:45:38.131 INFO:tasks.workunit.client.0.smithi063.stdout:4/761: dwrite d2/d8/d40/d51/f6c [0,4194304] 0 2024-04-24T22:45:38.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/484: truncate d0/d13/d19/d64/d4f/d52/f6a 413684 0 2024-04-24T22:45:38.138 INFO:tasks.workunit.client.0.smithi063.stdout:6/831: dread - db/d1d/d20/d3f/d7f/fac zero size 2024-04-24T22:45:38.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/832: chown db/d1c/dea/lbe 0 1 2024-04-24T22:45:38.140 INFO:tasks.workunit.client.1.smithi192.stdout:7/676: dwrite d2/db/d21/f22 [8388608,4194304] 0 2024-04-24T22:45:38.143 INFO:tasks.workunit.client.0.smithi063.stdout:5/752: creat d5/d1d/da3/f105 x:0 0 0 2024-04-24T22:45:38.146 INFO:tasks.workunit.client.1.smithi192.stdout:5/485: write d0/d13/d19/d64/d4f/d6c/f81 [999852,112397] 0 2024-04-24T22:45:38.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/720: write d0/fa [4033324,11957] 0 2024-04-24T22:45:38.148 INFO:tasks.workunit.client.0.smithi063.stdout:1/833: rmdir d3/db5/d21 39 2024-04-24T22:45:38.149 INFO:tasks.workunit.client.0.smithi063.stdout:8/817: write d1/d21/f5d [2575828,21634] 0 2024-04-24T22:45:38.151 INFO:tasks.workunit.client.1.smithi192.stdout:7/677: mkdir d2/db/d21/d42/d67/dd4 0 2024-04-24T22:45:38.152 INFO:tasks.workunit.client.0.smithi063.stdout:4/762: creat d2/d19/d33/dc1/d50/dc3/ff9 x:0 0 0 2024-04-24T22:45:38.152 INFO:tasks.workunit.client.1.smithi192.stdout:7/678: read d2/f1a [4778636,7675] 0 2024-04-24T22:45:38.153 INFO:tasks.workunit.client.1.smithi192.stdout:5/486: mknod d0/d13/d19/d64/d4f/d6c/ca6 0 2024-04-24T22:45:38.154 INFO:tasks.workunit.client.1.smithi192.stdout:5/487: readlink d0/l98 0 2024-04-24T22:45:38.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/794: creat d6/d2f/dcd/db/d93/f109 x:0 0 0 2024-04-24T22:45:38.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/488: dread - d0/d13/d19/d71/d7f/d9a/fa5 zero size 2024-04-24T22:45:38.155 INFO:tasks.workunit.client.0.smithi063.stdout:6/833: creat db/d1c/dc3/f115 x:0 0 0 2024-04-24T22:45:38.156 INFO:tasks.workunit.client.1.smithi192.stdout:7/679: mkdir d2/db/d21/dd5 0 2024-04-24T22:45:38.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/795: truncate d6/d2f/dcd/db/d1c/ffe 23513 0 2024-04-24T22:45:38.157 INFO:tasks.workunit.client.1.smithi192.stdout:7/680: write d2/db/d49/f70 [3527734,80395] 0 2024-04-24T22:45:38.158 INFO:tasks.workunit.client.0.smithi063.stdout:9/796: dread - d6/d2f/dcd/d1f/d20/d2a/d4b/de0/ffd zero size 2024-04-24T22:45:38.158 INFO:tasks.workunit.client.0.smithi063.stdout:5/753: truncate d5/d11/d23/d2d/d62/d73/d95/ff5 907068 0 2024-04-24T22:45:38.159 INFO:tasks.workunit.client.0.smithi063.stdout:5/754: write d5/d11/d23/d2d/fd1 [504982,6240] 0 2024-04-24T22:45:38.161 INFO:tasks.workunit.client.0.smithi063.stdout:1/834: readlink d3/db5/d21/d9f/daa/db7/l51 0 2024-04-24T22:45:38.162 INFO:tasks.workunit.client.1.smithi192.stdout:7/681: symlink d2/db/d21/d42/d67/dae/ld6 0 2024-04-24T22:45:38.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/818: truncate d1/d11/d3c/f9e 624702 0 2024-04-24T22:45:38.163 INFO:tasks.workunit.client.0.smithi063.stdout:4/763: chown d2/d19/f2b 220505 1 2024-04-24T22:45:38.163 INFO:tasks.workunit.client.0.smithi063.stdout:6/834: rmdir db/d1d/d2d/d69 39 2024-04-24T22:45:38.164 INFO:tasks.workunit.client.0.smithi063.stdout:6/835: chown db/d85/db5/d10f/l10e 296 1 2024-04-24T22:45:38.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/836: fdatasync db/d1d/d2d/d48/f63 0 2024-04-24T22:45:38.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/682: unlink d2/db/d49/c5c 0 2024-04-24T22:45:38.168 INFO:tasks.workunit.client.0.smithi063.stdout:8/819: creat d1/d2/d3/f107 x:0 0 0 2024-04-24T22:45:38.169 INFO:tasks.workunit.client.0.smithi063.stdout:4/764: mkdir d2/d19/d33/dc1/d50/dfa 0 2024-04-24T22:45:38.170 INFO:tasks.workunit.client.0.smithi063.stdout:8/820: stat d1/d22/d33/da6/d54/d79/c90 0 2024-04-24T22:45:38.171 INFO:tasks.workunit.client.0.smithi063.stdout:6/837: mkdir db/d1c/dea/d116 0 2024-04-24T22:45:38.172 INFO:tasks.workunit.client.0.smithi063.stdout:4/765: chown d2/d8/d40/da0/fc2 720826 1 2024-04-24T22:45:38.173 INFO:tasks.workunit.client.0.smithi063.stdout:8/821: write d1/d11/f1b [4904270,114353] 0 2024-04-24T22:45:38.173 INFO:tasks.workunit.client.0.smithi063.stdout:1/835: dwrite d3/da/f37 [4194304,4194304] 0 2024-04-24T22:45:38.174 INFO:tasks.workunit.client.1.smithi192.stdout:7/683: dwrite d2/db/d21/d42/d67/d9b/fcd [0,4194304] 0 2024-04-24T22:45:38.174 INFO:tasks.workunit.client.1.smithi192.stdout:7/684: readlink d2/db/d2a/lac 0 2024-04-24T22:45:38.177 INFO:tasks.workunit.client.1.smithi192.stdout:7/685: creat d2/db/d21/fd7 x:0 0 0 2024-04-24T22:45:38.177 INFO:tasks.workunit.client.0.smithi063.stdout:2/875: sync 2024-04-24T22:45:38.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/876: readlink d0/d2/d2d/d31/d89/l8b 0 2024-04-24T22:45:38.181 INFO:tasks.workunit.client.0.smithi063.stdout:1/836: fsync d3/da/f6a 0 2024-04-24T22:45:38.182 INFO:tasks.workunit.client.1.smithi192.stdout:7/686: rename d2/db/d21/d42/d67/d9b/d6e/d77/c79 to d2/db/d21/d42/d67/d9b/d6e/db5/cd8 0 2024-04-24T22:45:38.184 INFO:tasks.workunit.client.0.smithi063.stdout:8/822: creat d1/d22/d33/f108 x:0 0 0 2024-04-24T22:45:38.185 INFO:tasks.workunit.client.0.smithi063.stdout:4/766: truncate d2/d19/d33/dc1/f6d 1493808 0 2024-04-24T22:45:38.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/687: creat d2/db/d2a/d57/fd9 x:0 0 0 2024-04-24T22:45:38.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/877: creat d0/d2/d2d/d31/d89/d7/d86/da0/deb/f114 x:0 0 0 2024-04-24T22:45:38.194 INFO:tasks.workunit.client.1.smithi192.stdout:0/641: write d3/d27/d28/d2e/d3f/fc8 [622088,25955] 0 2024-04-24T22:45:38.194 INFO:tasks.workunit.client.0.smithi063.stdout:3/793: dread da/f1d [0,4194304] 0 2024-04-24T22:45:38.194 INFO:tasks.workunit.client.0.smithi063.stdout:1/837: symlink d3/db5/d21/d9f/l10c 0 2024-04-24T22:45:38.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/489: dread d0/f18 [0,4194304] 0 2024-04-24T22:45:38.199 INFO:tasks.workunit.client.0.smithi063.stdout:1/838: creat d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f10d x:0 0 0 2024-04-24T22:45:38.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/642: mknod d3/d27/d28/dcc/d5c/d80/d83/dd9/cda 0 2024-04-24T22:45:38.205 INFO:tasks.workunit.client.1.smithi192.stdout:2/731: sync 2024-04-24T22:45:38.208 INFO:tasks.workunit.client.1.smithi192.stdout:5/490: creat d0/d1b/d62/fa7 x:0 0 0 2024-04-24T22:45:38.210 INFO:tasks.workunit.client.1.smithi192.stdout:0/643: mknod d3/d27/d28/d31/dbc/cdb 0 2024-04-24T22:45:38.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/721: write d0/d4/de/d6f/f53 [2248812,12334] 0 2024-04-24T22:45:38.212 INFO:tasks.workunit.client.1.smithi192.stdout:7/688: getdents d2/db/d21/d42/d67/d9b/dc1 0 2024-04-24T22:45:38.213 INFO:tasks.workunit.client.0.smithi063.stdout:8/823: dread d1/d2/f84 [0,4194304] 0 2024-04-24T22:45:38.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/824: fsync d1/d11/d32/dd3/fe8 0 2024-04-24T22:45:38.217 INFO:tasks.workunit.client.1.smithi192.stdout:2/732: rmdir d4/d9/d10/d6c 39 2024-04-24T22:45:38.217 INFO:tasks.workunit.client.0.smithi063.stdout:9/797: write d6/d2f/fa4 [786891,90596] 0 2024-04-24T22:45:38.218 INFO:tasks.workunit.client.1.smithi192.stdout:0/644: dwrite d3/d27/d6a/dc4/fd3 [0,4194304] 0 2024-04-24T22:45:38.218 INFO:tasks.workunit.client.0.smithi063.stdout:7/722: dwrite d0/d4/de/d6f/f53 [4194304,4194304] 0 2024-04-24T22:45:38.219 INFO:tasks.workunit.client.0.smithi063.stdout:7/723: readlink d0/d4/de/d6f/d2e/l4b 0 2024-04-24T22:45:38.220 INFO:tasks.workunit.client.1.smithi192.stdout:5/491: creat d0/d13/d19/d64/fa8 x:0 0 0 2024-04-24T22:45:38.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/689: rmdir d2/db/d2a 39 2024-04-24T22:45:38.227 INFO:tasks.workunit.client.0.smithi063.stdout:4/767: getdents d2/d19/dce 0 2024-04-24T22:45:38.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/492: dwrite d0/d1b/f8f [0,4194304] 0 2024-04-24T22:45:38.228 INFO:tasks.workunit.client.1.smithi192.stdout:2/733: chown d4/d9/d10/d6c/f8e 0 1 2024-04-24T22:45:38.231 INFO:tasks.workunit.client.1.smithi192.stdout:0/645: symlink d3/d27/d3c/d6c/ldc 0 2024-04-24T22:45:38.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/825: fsync d1/d22/d33/da6/d54/d79/fda 0 2024-04-24T22:45:38.233 INFO:tasks.workunit.client.0.smithi063.stdout:6/838: write db/d1d/d20/d3f/d7f/d68/f6d [892875,124368] 0 2024-04-24T22:45:38.234 INFO:tasks.workunit.client.0.smithi063.stdout:8/826: write d1/d2/d3/f14 [5343586,124141] 0 2024-04-24T22:45:38.238 INFO:tasks.workunit.client.0.smithi063.stdout:0/725: sync 2024-04-24T22:45:38.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/726: write d6/d64/da0/d33/fb8 [2589057,72200] 0 2024-04-24T22:45:38.243 INFO:tasks.workunit.client.0.smithi063.stdout:8/827: dwrite d1/d2/d3/db6/ffc [0,4194304] 0 2024-04-24T22:45:38.244 INFO:tasks.workunit.client.0.smithi063.stdout:4/768: truncate d2/f1c 1456235 0 2024-04-24T22:45:38.245 INFO:tasks.workunit.client.1.smithi192.stdout:0/646: fsync d3/d27/d28/dcc/d5c/f46 0 2024-04-24T22:45:38.246 INFO:tasks.workunit.client.0.smithi063.stdout:8/828: truncate d1/d22/d33/da6/d54/d7b/f7f 1162923 0 2024-04-24T22:45:38.246 INFO:tasks.workunit.client.1.smithi192.stdout:0/647: truncate d3/f48 2145549 0 2024-04-24T22:45:38.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/690: mkdir d2/db/d25/dda 0 2024-04-24T22:45:38.249 INFO:tasks.workunit.client.0.smithi063.stdout:2/878: write d0/dd/d72/d77/d87/f9c [728466,47896] 0 2024-04-24T22:45:38.252 INFO:tasks.workunit.client.0.smithi063.stdout:7/724: mkdir d0/d4/de/dee 0 2024-04-24T22:45:38.256 INFO:tasks.workunit.client.0.smithi063.stdout:3/794: dwrite da/d39/d94/db3/fd2 [0,4194304] 0 2024-04-24T22:45:38.257 INFO:tasks.workunit.client.0.smithi063.stdout:2/879: dwrite d0/d2/d27/f85 [0,4194304] 0 2024-04-24T22:45:38.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/880: readlink d0/d2/d2f/ld4 0 2024-04-24T22:45:38.260 INFO:tasks.workunit.client.0.smithi063.stdout:2/881: chown d0/d2/d35/l41 982 1 2024-04-24T22:45:38.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/648: truncate d3/d27/f71 169672 0 2024-04-24T22:45:38.261 INFO:tasks.workunit.client.0.smithi063.stdout:1/839: write d3/db5/d21/d9f/dd5/d106/d11/d5e/d6f/f90 [1588767,111467] 0 2024-04-24T22:45:38.266 INFO:tasks.workunit.client.1.smithi192.stdout:2/734: creat d4/d9/d10/d6c/d6f/ff0 x:0 0 0 2024-04-24T22:45:38.268 INFO:tasks.workunit.client.1.smithi192.stdout:0/649: dread d3/d27/d28/fc3 [0,4194304] 0 2024-04-24T22:45:38.269 INFO:tasks.workunit.client.0.smithi063.stdout:4/769: dread d2/d8/f23 [0,4194304] 0 2024-04-24T22:45:38.271 INFO:tasks.workunit.client.1.smithi192.stdout:5/493: write d0/d13/d3b/d2f/f84 [729134,54875] 0 2024-04-24T22:45:38.272 INFO:tasks.workunit.client.1.smithi192.stdout:5/494: stat d0/d13/d19/d64/d4f/d6c/f81 0 2024-04-24T22:45:38.272 INFO:tasks.workunit.client.1.smithi192.stdout:7/691: link d2/db/d21/d42/d5b/d63/d80/fcc d2/db/d25/dda/fdb 0 2024-04-24T22:45:38.274 INFO:tasks.workunit.client.1.smithi192.stdout:5/495: truncate d0/d1b/f89 5168714 0 2024-04-24T22:45:38.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/755: sync 2024-04-24T22:45:38.276 INFO:tasks.workunit.client.0.smithi063.stdout:6/839: creat db/d1d/d20/d29/d5d/df5/f117 x:0 0 0 2024-04-24T22:45:38.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/829: mknod d1/d2/d17/dd1/c109 0 2024-04-24T22:45:38.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/798: creat d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/f10a x:0 0 0 2024-04-24T22:45:38.278 INFO:tasks.workunit.client.0.smithi063.stdout:8/830: chown d1/f62 2 1 2024-04-24T22:45:38.278 INFO:tasks.workunit.client.0.smithi063.stdout:6/840: dread - db/d1d/d20/d3f/d7f/fac zero size 2024-04-24T22:45:38.279 INFO:tasks.workunit.client.1.smithi192.stdout:0/650: rmdir d3/d27/d28/dcc/d5c 39 2024-04-24T22:45:38.281 INFO:tasks.workunit.client.1.smithi192.stdout:7/692: dread - d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 zero size 2024-04-24T22:45:38.283 INFO:tasks.workunit.client.0.smithi063.stdout:7/725: mknod d0/d4/de/d17/d35/d69/da0/cef 0 2024-04-24T22:45:38.285 INFO:tasks.workunit.client.1.smithi192.stdout:7/693: dread d2/f2c [4194304,4194304] 0 2024-04-24T22:45:38.286 INFO:tasks.workunit.client.0.smithi063.stdout:6/841: dwrite db/d1c/dc2/fd3 [0,4194304] 0 2024-04-24T22:45:38.286 INFO:tasks.workunit.client.0.smithi063.stdout:7/726: write d0/d4/de/d17/d35/d69/db1/fc1 [4940118,70038] 0 2024-04-24T22:45:38.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/840: mkdir d3/db5/d21/d9f/dd5/d106/d39/d5a/d10e 0 2024-04-24T22:45:38.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/496: mkdir d0/d13/d19/da9 0 2024-04-24T22:45:38.298 INFO:tasks.workunit.client.1.smithi192.stdout:2/735: dread d4/f13 [0,4194304] 0 2024-04-24T22:45:38.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/736: dwrite d4/d29/fe9 [0,4194304] 0 2024-04-24T22:45:38.307 INFO:tasks.workunit.client.1.smithi192.stdout:2/737: truncate d4/fe1 177202 0 2024-04-24T22:45:38.315 INFO:tasks.workunit.client.1.smithi192.stdout:0/651: link d3/d27/d28/dcc/d5c/f61 d3/d27/d28/d31/fdd 0 2024-04-24T22:45:38.320 INFO:tasks.workunit.client.0.smithi063.stdout:3/795: dread da/f28 [0,4194304] 0 2024-04-24T22:45:38.321 INFO:tasks.workunit.client.1.smithi192.stdout:4/598: sync 2024-04-24T22:45:38.328 INFO:tasks.workunit.client.1.smithi192.stdout:0/652: link d3/d27/d28/d31/dbc/cdb d3/d27/d28/dcc/d5c/d80/d83/dd9/cde 0 2024-04-24T22:45:38.330 INFO:tasks.workunit.client.1.smithi192.stdout:5/497: dwrite d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:38.332 INFO:tasks.workunit.client.1.smithi192.stdout:7/694: dread d2/db/d2a/d57/fa3 [0,4194304] 0 2024-04-24T22:45:38.336 INFO:tasks.workunit.client.1.smithi192.stdout:2/738: dwrite d4/d9/fcc [0,4194304] 0 2024-04-24T22:45:38.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/739: stat d4/d9/f4c 0 2024-04-24T22:45:38.341 INFO:tasks.workunit.client.1.smithi192.stdout:4/599: dread d2/db/d5d/d77/d82/fa3 [0,4194304] 0 2024-04-24T22:45:38.347 INFO:tasks.workunit.client.1.smithi192.stdout:4/600: dread d2/d9/d6b/fac [0,4194304] 0 2024-04-24T22:45:38.353 INFO:tasks.workunit.client.1.smithi192.stdout:7/695: rename d2/db/d21/d42/d5b/l87 to d2/db/ldc 0 2024-04-24T22:45:38.359 INFO:tasks.workunit.client.1.smithi192.stdout:5/498: rename d0/c21 to d0/d13/d19/d64/d4f/d7c/caa 0 2024-04-24T22:45:38.360 INFO:tasks.workunit.client.1.smithi192.stdout:2/740: link d4/d9/d10/d6c/d8c/da5/ld2 d4/d29/d60/d9a/db3/lf1 0 2024-04-24T22:45:38.361 INFO:tasks.workunit.client.1.smithi192.stdout:4/601: mkdir d2/db/d75/da1/dc9 0 2024-04-24T22:45:38.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/499: fsync d0/d13/d19/d64/d4f/d52/f61 0 2024-04-24T22:45:38.364 INFO:tasks.workunit.client.0.smithi063.stdout:4/770: read d2/d8/f32 [1364347,96494] 0 2024-04-24T22:45:38.364 INFO:tasks.workunit.client.0.smithi063.stdout:5/756: rmdir d5/d11/d23/d2d/d62/d81 39 2024-04-24T22:45:38.365 INFO:tasks.workunit.client.1.smithi192.stdout:2/741: symlink d4/d9/d10/d6c/d6f/da8/lf2 0 2024-04-24T22:45:38.367 INFO:tasks.workunit.client.1.smithi192.stdout:4/602: chown d2/d9/f3f 13202 1 2024-04-24T22:45:38.371 INFO:tasks.workunit.client.0.smithi063.stdout:4/771: dwrite d2/d8/dd4/dda/dc6/ff4 [0,4194304] 0 2024-04-24T22:45:38.374 INFO:tasks.workunit.client.1.smithi192.stdout:2/742: mkdir d4/d9/d37/de5/df3 0 2024-04-24T22:45:38.376 INFO:tasks.workunit.client.0.smithi063.stdout:2/882: symlink d0/d2/l115 0 2024-04-24T22:45:38.376 INFO:tasks.workunit.client.0.smithi063.stdout:9/799: dread d6/d2f/dcd/db/d1c/f69 [0,4194304] 0 2024-04-24T22:45:38.377 INFO:tasks.workunit.client.0.smithi063.stdout:9/800: stat d6/d2f/dcd/db/d1c/l38 0 2024-04-24T22:45:38.378 INFO:tasks.workunit.client.1.smithi192.stdout:4/603: creat d2/db/d75/da1/d7d/fca x:0 0 0 2024-04-24T22:45:38.381 INFO:tasks.workunit.client.0.smithi063.stdout:7/727: chown d0/d2f/f5a 30863 1 2024-04-24T22:45:38.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/500: dread d0/d13/d19/d64/d4f/d52/f67 [0,4194304] 0 2024-04-24T22:45:38.383 INFO:tasks.workunit.client.0.smithi063.stdout:6/842: mkdir db/d1d/d8d/dce/d118 0 2024-04-24T22:45:38.383 INFO:tasks.workunit.client.0.smithi063.stdout:9/801: dwrite d6/d2f/dcd/db/d6b/f23 [0,4194304] 0 2024-04-24T22:45:38.385 INFO:tasks.workunit.client.0.smithi063.stdout:1/841: stat d3/da/d34/d6d/c8e 0 2024-04-24T22:45:38.387 INFO:tasks.workunit.client.1.smithi192.stdout:4/604: creat d2/da5/fcb x:0 0 0 2024-04-24T22:45:38.389 INFO:tasks.workunit.client.1.smithi192.stdout:4/605: dread - d2/db/d10/fc6 zero size 2024-04-24T22:45:38.390 INFO:tasks.workunit.client.1.smithi192.stdout:4/606: readlink d2/db/d10/l1a 0 2024-04-24T22:45:38.391 INFO:tasks.workunit.client.1.smithi192.stdout:2/743: dwrite d4/d29/d60/d9a/db3/fd6 [0,4194304] 0 2024-04-24T22:45:38.395 INFO:tasks.workunit.client.1.smithi192.stdout:2/744: dread - d4/d9/d10/d6c/d95/db7/dbf/fd9 zero size 2024-04-24T22:45:38.397 INFO:tasks.workunit.client.0.smithi063.stdout:5/757: read d5/d24/db6/fbd [4054020,107913] 0 2024-04-24T22:45:38.400 INFO:tasks.workunit.client.1.smithi192.stdout:4/607: write d2/db/d75/da1/d7d/dbd/fc3 [2726083,102807] 0 2024-04-24T22:45:38.401 INFO:tasks.workunit.client.0.smithi063.stdout:2/883: read - d0/dd/d72/d77/fce zero size 2024-04-24T22:45:38.401 INFO:tasks.workunit.client.1.smithi192.stdout:4/608: stat d2/db/d75/da1/d7d/c7f 0 2024-04-24T22:45:38.403 INFO:tasks.workunit.client.0.smithi063.stdout:8/831: dwrite d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:38.408 INFO:tasks.workunit.client.0.smithi063.stdout:7/728: rmdir d0/d4/de/d6f/d5d 39 2024-04-24T22:45:38.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/772: dread d2/d19/f2b [0,4194304] 0 2024-04-24T22:45:38.411 INFO:tasks.workunit.client.0.smithi063.stdout:9/802: fsync d6/f30 0 2024-04-24T22:45:38.413 INFO:tasks.workunit.client.0.smithi063.stdout:0/727: getdents d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:38.415 INFO:tasks.workunit.client.0.smithi063.stdout:3/796: symlink da/d18/d3a/da1/l112 0 2024-04-24T22:45:38.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/728: dread d6/d64/d21/d25/d31/da3/ff0 [0,4194304] 0 2024-04-24T22:45:38.419 INFO:tasks.workunit.client.0.smithi063.stdout:5/758: dread d5/d24/f91 [0,4194304] 0 2024-04-24T22:45:38.420 INFO:tasks.workunit.client.0.smithi063.stdout:5/759: fsync d5/d11/d23/d2d/ff9 0 2024-04-24T22:45:38.420 INFO:tasks.workunit.client.0.smithi063.stdout:2/884: rename d0/d2/d27/f7f to d0/d2/d2d/d31/d89/d7/d86/da0/deb/d10e/f116 0 2024-04-24T22:45:38.421 INFO:tasks.workunit.client.0.smithi063.stdout:2/885: dread - d0/d2/d2d/d31/d89/d1c/d3f/f110 zero size 2024-04-24T22:45:38.422 INFO:tasks.workunit.client.1.smithi192.stdout:4/609: getdents d2/db/d75/da1/d36/d70 0 2024-04-24T22:45:38.422 INFO:tasks.workunit.client.1.smithi192.stdout:5/501: dwrite d0/d13/f3f [0,4194304] 0 2024-04-24T22:45:38.424 INFO:tasks.workunit.client.0.smithi063.stdout:4/773: mknod d2/d80/d81/d90/cfb 0 2024-04-24T22:45:38.425 INFO:tasks.workunit.client.1.smithi192.stdout:2/745: dwrite d4/d9/d37/f62 [0,4194304] 0 2024-04-24T22:45:38.426 INFO:tasks.workunit.client.1.smithi192.stdout:4/610: dread - d2/db/d10/f96 zero size 2024-04-24T22:45:38.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/843: fsync db/d1d/d2d/d69/d82/f9a 0 2024-04-24T22:45:38.427 INFO:tasks.workunit.client.0.smithi063.stdout:1/842: mknod d3/db5/d21/d9f/daa/d108/c10f 0 2024-04-24T22:45:38.428 INFO:tasks.workunit.client.1.smithi192.stdout:5/502: dread - d0/d13/d19/d64/d5f/d2c/f51 zero size 2024-04-24T22:45:38.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/746: read d4/d9/d10/f1e [744666,104679] 0 2024-04-24T22:45:38.441 INFO:tasks.workunit.client.1.smithi192.stdout:4/611: link d2/db/d75/da1/d36/d5b/d28/f45 d2/db/d75/da1/d53/fcc 0 2024-04-24T22:45:38.441 INFO:tasks.workunit.client.1.smithi192.stdout:4/612: chown d2/lb0 583 1 2024-04-24T22:45:38.444 INFO:tasks.workunit.client.1.smithi192.stdout:2/747: rename d4/d9/d10/d6c/d95/db7/dbf/fc5 to d4/d9/d37/ff4 0 2024-04-24T22:45:38.444 INFO:tasks.workunit.client.1.smithi192.stdout:2/748: chown d4/f13 16212 1 2024-04-24T22:45:38.447 INFO:tasks.workunit.client.1.smithi192.stdout:5/503: creat d0/d13/d19/da9/fab x:0 0 0 2024-04-24T22:45:38.448 INFO:tasks.workunit.client.0.smithi063.stdout:8/832: creat d1/d22/d33/dbd/f10a x:0 0 0 2024-04-24T22:45:38.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/760: unlink f0 0 2024-04-24T22:45:38.448 INFO:tasks.workunit.client.0.smithi063.stdout:9/803: rename d6/d2f/dcd/d1f/d20/d2a/c92 to d6/d2f/dcd/db/d6b/c10b 0 2024-04-24T22:45:38.449 INFO:tasks.workunit.client.1.smithi192.stdout:5/504: dread d0/d13/d53/f94 [0,4194304] 0 2024-04-24T22:45:38.451 INFO:tasks.workunit.client.1.smithi192.stdout:2/749: dread d4/d9/f90 [0,4194304] 0 2024-04-24T22:45:38.451 INFO:tasks.workunit.client.0.smithi063.stdout:1/843: dread d3/f19 [0,4194304] 0 2024-04-24T22:45:38.451 INFO:tasks.workunit.client.0.smithi063.stdout:3/797: write da/d48/f5c [406514,73819] 0 2024-04-24T22:45:38.454 INFO:tasks.workunit.client.0.smithi063.stdout:4/774: unlink d2/d8/f10 0 2024-04-24T22:45:38.455 INFO:tasks.workunit.client.0.smithi063.stdout:9/804: dwrite d6/fd [0,4194304] 0 2024-04-24T22:45:38.460 INFO:tasks.workunit.client.1.smithi192.stdout:4/613: creat d2/d5c/dbb/fcd x:0 0 0 2024-04-24T22:45:38.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/614: dread - d2/da5/fcb zero size 2024-04-24T22:45:38.463 INFO:tasks.workunit.client.0.smithi063.stdout:7/729: write d0/d4/de/d17/d35/d6c/f9e [800749,129627] 0 2024-04-24T22:45:38.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/750: mkdir d4/d9/d10/da7/df5 0 2024-04-24T22:45:38.470 INFO:tasks.workunit.client.0.smithi063.stdout:5/761: rmdir d5/de5 39 2024-04-24T22:45:38.472 INFO:tasks.workunit.client.1.smithi192.stdout:4/615: mkdir d2/d5c/dce 0 2024-04-24T22:45:38.473 INFO:tasks.workunit.client.1.smithi192.stdout:4/616: chown d2/db/d5d/d77/d82/fa3 1971 1 2024-04-24T22:45:38.475 INFO:tasks.workunit.client.0.smithi063.stdout:5/762: dwrite d5/d24/db6/fcc [0,4194304] 0 2024-04-24T22:45:38.478 INFO:tasks.workunit.client.1.smithi192.stdout:4/617: mknod d2/db/d75/da1/d7d/dbd/ccf 0 2024-04-24T22:45:38.483 INFO:tasks.workunit.client.1.smithi192.stdout:5/505: link d0/d13/d19/d64/d5f/d2c/c56 d0/d13/d19/cac 0 2024-04-24T22:45:38.484 INFO:tasks.workunit.client.1.smithi192.stdout:5/506: fdatasync d0/d13/d19/da9/fab 0 2024-04-24T22:45:38.484 INFO:tasks.workunit.client.0.smithi063.stdout:4/775: creat d2/d19/d33/d9d/da8/dec/ffc x:0 0 0 2024-04-24T22:45:38.485 INFO:tasks.workunit.client.0.smithi063.stdout:9/805: fsync d6/d2f/dcd/db/d6b/d4f/d59/f9b 0 2024-04-24T22:45:38.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/844: mknod db/d1c/ddb/c119 0 2024-04-24T22:45:38.488 INFO:tasks.workunit.client.1.smithi192.stdout:4/618: creat d2/fd0 x:0 0 0 2024-04-24T22:45:38.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/730: creat d0/d4/de/d6f/d2e/ff0 x:0 0 0 2024-04-24T22:45:38.495 INFO:tasks.workunit.client.0.smithi063.stdout:7/731: truncate d0/d4/de/d6f/d5c/d9c/fd8 4728760 0 2024-04-24T22:45:38.497 INFO:tasks.workunit.client.1.smithi192.stdout:5/507: dwrite d0/d13/d19/d64/d4f/d6c/f9c [0,4194304] 0 2024-04-24T22:45:38.499 INFO:tasks.workunit.client.1.smithi192.stdout:2/751: dwrite d4/d9/d37/f3b [0,4194304] 0 2024-04-24T22:45:38.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/729: sync 2024-04-24T22:45:38.508 INFO:tasks.workunit.client.0.smithi063.stdout:0/730: truncate d6/d64/da0/d98/dd6/fe7 229900 0 2024-04-24T22:45:38.508 INFO:tasks.workunit.client.1.smithi192.stdout:5/508: fdatasync d0/d13/d53/f70 0 2024-04-24T22:45:38.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/844: fdatasync d3/f2c 0 2024-04-24T22:45:38.516 INFO:tasks.workunit.client.0.smithi063.stdout:3/798: creat da/d40/d111/f113 x:0 0 0 2024-04-24T22:45:38.517 INFO:tasks.workunit.client.1.smithi192.stdout:5/509: mkdir d0/d13/d19/d83/dad 0 2024-04-24T22:45:38.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/510: readlink d0/d13/d19/d64/d4f/la0 0 2024-04-24T22:45:38.519 INFO:tasks.workunit.client.1.smithi192.stdout:2/752: dwrite d4/d9/d10/da7/fc4 [4194304,4194304] 0 2024-04-24T22:45:38.520 INFO:tasks.workunit.client.0.smithi063.stdout:6/845: truncate db/d1d/d2d/d69/dd4/de4/fd5 118178 0 2024-04-24T22:45:38.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/511: stat d0/d13/d19/d64/d5f/d2c/d3c/d91 0 2024-04-24T22:45:38.526 INFO:tasks.workunit.client.1.smithi192.stdout:5/512: truncate d0/d1b/f6d 494110 0 2024-04-24T22:45:38.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/886: truncate d0/d2/d2d/d31/d89/d48/fee 170402 0 2024-04-24T22:45:38.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/513: creat d0/d13/d19/d64/d4f/d6c/fae x:0 0 0 2024-04-24T22:45:38.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/846: dread db/d1d/d20/d3f/d7f/d68/f6d [0,4194304] 0 2024-04-24T22:45:38.534 INFO:tasks.workunit.client.0.smithi063.stdout:9/806: write d6/d2f/d42/d81/fc2 [1094450,89321] 0 2024-04-24T22:45:38.535 INFO:tasks.workunit.client.0.smithi063.stdout:7/732: mknod d0/d4/de/d6f/cf1 0 2024-04-24T22:45:38.535 INFO:tasks.workunit.client.0.smithi063.stdout:9/807: read - d6/d2f/dcd/d1f/ddb/ff8 zero size 2024-04-24T22:45:38.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/733: rename d0 to d0/d4/de/ded/df2 22 2024-04-24T22:45:38.536 INFO:tasks.workunit.client.1.smithi192.stdout:5/514: truncate d0/d13/d19/d64/d5f/d2c/f3d 569149 0 2024-04-24T22:45:38.538 INFO:tasks.workunit.client.1.smithi192.stdout:2/753: write d4/d9/d10/d6c/d6f/da8/fac [175509,61055] 0 2024-04-24T22:45:38.539 INFO:tasks.workunit.client.0.smithi063.stdout:8/833: getdents d1/d2/d17 0 2024-04-24T22:45:38.540 INFO:tasks.workunit.client.1.smithi192.stdout:5/515: mkdir d0/d13/d19/d71/daf 0 2024-04-24T22:45:38.540 INFO:tasks.workunit.client.1.smithi192.stdout:6/550: sync 2024-04-24T22:45:38.540 INFO:tasks.workunit.client.1.smithi192.stdout:3/665: sync 2024-04-24T22:45:38.540 INFO:tasks.workunit.client.1.smithi192.stdout:8/616: sync 2024-04-24T22:45:38.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/535: sync 2024-04-24T22:45:38.541 INFO:tasks.workunit.client.1.smithi192.stdout:9/706: sync 2024-04-24T22:45:38.542 INFO:tasks.workunit.client.1.smithi192.stdout:2/754: read - d4/fbd zero size 2024-04-24T22:45:38.542 INFO:tasks.workunit.client.0.smithi063.stdout:0/731: mknod d6/d64/d21/d25/d62/d63/d89/cf8 0 2024-04-24T22:45:38.542 INFO:tasks.workunit.client.1.smithi192.stdout:8/617: write f5 [5209468,51518] 0 2024-04-24T22:45:38.543 INFO:tasks.workunit.client.1.smithi192.stdout:3/666: chown d2/d62/db2/l8e 1659087 1 2024-04-24T22:45:38.543 INFO:tasks.workunit.client.1.smithi192.stdout:5/516: stat d0/d1b/f37 0 2024-04-24T22:45:38.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/845: mknod d3/de0/d79/d96/df8/d10a/c110 0 2024-04-24T22:45:38.545 INFO:tasks.workunit.client.0.smithi063.stdout:1/846: chown d3/db5/d21/d9f/dd5/d106/d11/f41 31273 1 2024-04-24T22:45:38.547 INFO:tasks.workunit.client.0.smithi063.stdout:1/847: write d3/da/f37 [3373359,120310] 0 2024-04-24T22:45:38.552 INFO:tasks.workunit.client.1.smithi192.stdout:6/551: rmdir d0/d8 39 2024-04-24T22:45:38.554 INFO:tasks.workunit.client.0.smithi063.stdout:2/887: read d0/d2/d27/f29 [3358363,99174] 0 2024-04-24T22:45:38.558 INFO:tasks.workunit.client.1.smithi192.stdout:8/618: stat ca 0 2024-04-24T22:45:38.561 INFO:tasks.workunit.client.1.smithi192.stdout:2/755: mknod d4/d29/d60/cf6 0 2024-04-24T22:45:38.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/536: write d9/d15/f1b [128074,55868] 0 2024-04-24T22:45:38.563 INFO:tasks.workunit.client.0.smithi063.stdout:7/734: creat d0/d4/de/d6f/d2e/d76/ff3 x:0 0 0 2024-04-24T22:45:38.563 INFO:tasks.workunit.client.1.smithi192.stdout:9/707: write d3/d24/f65 [329046,91184] 0 2024-04-24T22:45:38.565 INFO:tasks.workunit.client.1.smithi192.stdout:5/517: creat d0/d13/d19/d64/d5f/d2c/fb0 x:0 0 0 2024-04-24T22:45:38.565 INFO:tasks.workunit.client.0.smithi063.stdout:2/888: dwrite d0/d2/d2d/ff7 [0,4194304] 0 2024-04-24T22:45:38.567 INFO:tasks.workunit.client.0.smithi063.stdout:0/732: creat d6/d64/d21/d25/d62/d63/d89/d95/da4/ff9 x:0 0 0 2024-04-24T22:45:38.568 INFO:tasks.workunit.client.0.smithi063.stdout:4/776: dwrite d2/f7c [0,4194304] 0 2024-04-24T22:45:38.569 INFO:tasks.workunit.client.1.smithi192.stdout:8/619: mknod db/dc/d22/d31/d33/cbf 0 2024-04-24T22:45:38.570 INFO:tasks.workunit.client.0.smithi063.stdout:5/763: dwrite d5/d11/f6a [0,4194304] 0 2024-04-24T22:45:38.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/847: dwrite db/d1d/d20/d84/f9e [0,4194304] 0 2024-04-24T22:45:38.662 INFO:tasks.workunit.client.1.smithi192.stdout:1/537: mkdir d9/d15/d2d/d48/d93/db4 0 2024-04-24T22:45:38.664 INFO:tasks.workunit.client.1.smithi192.stdout:9/708: creat d3/dd/de/da7/dae/ffc x:0 0 0 2024-04-24T22:45:38.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/518: creat d0/d1b/d50/fb1 x:0 0 0 2024-04-24T22:45:38.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/834: symlink d1/d2/l10b 0 2024-04-24T22:45:38.671 INFO:tasks.workunit.client.0.smithi063.stdout:3/799: getdents da/d18/dfb 0 2024-04-24T22:45:38.673 INFO:tasks.workunit.client.1.smithi192.stdout:8/620: creat db/dc/d29/d9f/fc0 x:0 0 0 2024-04-24T22:45:38.675 INFO:tasks.workunit.client.0.smithi063.stdout:7/735: creat d0/d4/de/d17/d28/d31/d99/ff4 x:0 0 0 2024-04-24T22:45:38.676 INFO:tasks.workunit.client.1.smithi192.stdout:2/756: truncate d4/d9/d10/f5d 2379834 0 2024-04-24T22:45:38.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/800: dwrite da/d39/d94/db3/fdf [4194304,4194304] 0 2024-04-24T22:45:38.679 INFO:tasks.workunit.client.1.smithi192.stdout:3/667: creat d2/d4/fd2 x:0 0 0 2024-04-24T22:45:38.680 INFO:tasks.workunit.client.0.smithi063.stdout:4/777: fsync d2/f2e 0 2024-04-24T22:45:38.683 INFO:tasks.workunit.client.0.smithi063.stdout:2/889: symlink d0/d2/d35/dc6/de2/de4/l117 0 2024-04-24T22:45:38.685 INFO:tasks.workunit.client.0.smithi063.stdout:0/733: creat d6/da/d2b/d60/ffa x:0 0 0 2024-04-24T22:45:38.688 INFO:tasks.workunit.client.1.smithi192.stdout:8/621: dread db/dc/f6e [0,4194304] 0 2024-04-24T22:45:38.688 INFO:tasks.workunit.client.0.smithi063.stdout:5/764: fsync d5/d24/d44/f46 0 2024-04-24T22:45:38.689 INFO:tasks.workunit.client.0.smithi063.stdout:5/765: write d5/d24/d49/f97 [839669,14304] 0 2024-04-24T22:45:38.691 INFO:tasks.workunit.client.1.smithi192.stdout:6/552: rename d0/d8/d29/d4c/d4d/d6a/la6 to d0/d8/d18/lc0 0 2024-04-24T22:45:38.693 INFO:tasks.workunit.client.0.smithi063.stdout:6/848: creat db/d1c/dea/f11a x:0 0 0 2024-04-24T22:45:38.698 INFO:tasks.workunit.client.1.smithi192.stdout:6/553: write d0/d8/d3d/d3e/d40/d44/fbd [1886,73654] 0 2024-04-24T22:45:38.699 INFO:tasks.workunit.client.1.smithi192.stdout:2/757: unlink d4/d9/d10/d6c/d8c/da5/le8 0 2024-04-24T22:45:38.703 INFO:tasks.workunit.client.1.smithi192.stdout:2/758: dread d4/d9/d10/d6c/d8c/f8f [0,4194304] 0 2024-04-24T22:45:38.704 INFO:tasks.workunit.client.1.smithi192.stdout:1/538: dwrite d9/f6a [0,4194304] 0 2024-04-24T22:45:38.707 INFO:tasks.workunit.client.1.smithi192.stdout:9/709: creat d3/dd/d53/ddd/ffd x:0 0 0 2024-04-24T22:45:38.710 INFO:tasks.workunit.client.1.smithi192.stdout:5/519: mknod d0/d13/d19/d71/daf/cb2 0 2024-04-24T22:45:38.712 INFO:tasks.workunit.client.0.smithi063.stdout:1/848: creat d3/db5/d21/d9f/dd5/d106/d11/f111 x:0 0 0 2024-04-24T22:45:38.712 INFO:tasks.workunit.client.1.smithi192.stdout:9/710: write d3/dd/de/da7/fd3 [351148,59161] 0 2024-04-24T22:45:38.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/520: fsync d0/d13/d19/d64/d5f/f20 0 2024-04-24T22:45:38.716 INFO:tasks.workunit.client.1.smithi192.stdout:8/622: dread db/f34 [0,4194304] 0 2024-04-24T22:45:38.716 INFO:tasks.workunit.client.0.smithi063.stdout:2/890: mkdir d0/d2/d2d/d31/d89/d1c/d3f/dca/d118 0 2024-04-24T22:45:38.716 INFO:tasks.workunit.client.1.smithi192.stdout:9/711: chown d3/dd/de/d17/d89/d8b 79 1 2024-04-24T22:45:38.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/539: dwrite d9/d51/f54 [0,4194304] 0 2024-04-24T22:45:38.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/734: rename d6/d64/d21/d25/d62/d63/c6c to d6/d64/d24/dd1/cfb 0 2024-04-24T22:45:38.719 INFO:tasks.workunit.client.1.smithi192.stdout:1/540: chown d9/d15/d5e/l80 19755 1 2024-04-24T22:45:38.721 INFO:tasks.workunit.client.1.smithi192.stdout:6/554: write d0/d8/d29/d5a/f60 [3065912,48232] 0 2024-04-24T22:45:38.726 INFO:tasks.workunit.client.0.smithi063.stdout:2/891: dwrite d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa [0,4194304] 0 2024-04-24T22:45:38.729 INFO:tasks.workunit.client.0.smithi063.stdout:0/735: dread d6/d64/da0/d33/f99 [0,4194304] 0 2024-04-24T22:45:38.729 INFO:tasks.workunit.client.0.smithi063.stdout:9/808: getdents d6/d2f/dcd/d1f/ddb 0 2024-04-24T22:45:38.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/668: truncate d2/d27/dc6/f2e 3322669 0 2024-04-24T22:45:38.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/801: mknod da/d18/d3a/c114 0 2024-04-24T22:45:38.739 INFO:tasks.workunit.client.0.smithi063.stdout:4/778: symlink d2/d19/d33/d9d/da8/dec/lfd 0 2024-04-24T22:45:38.742 INFO:tasks.workunit.client.1.smithi192.stdout:2/759: dread d4/d9/d10/d6c/d8c/fc6 [0,4194304] 0 2024-04-24T22:45:38.742 INFO:tasks.workunit.client.1.smithi192.stdout:9/712: unlink d3/dd/de/da7/dae/ffc 0 2024-04-24T22:45:38.744 INFO:tasks.workunit.client.1.smithi192.stdout:9/713: write d3/dd/de/da7/dae/fd6 [556986,66252] 0 2024-04-24T22:45:38.744 INFO:tasks.workunit.client.0.smithi063.stdout:6/849: write db/d1d/d2d/d69/d82/f9a [2565524,14791] 0 2024-04-24T22:45:38.748 INFO:tasks.workunit.client.1.smithi192.stdout:6/555: creat d0/d8/d29/d4c/fc1 x:0 0 0 2024-04-24T22:45:38.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/849: rename d3/da/d34/d6d to d3/de0/d79/d96/d112 0 2024-04-24T22:45:38.757 INFO:tasks.workunit.client.1.smithi192.stdout:3/669: truncate d2/d4/d12/f1c 3057020 0 2024-04-24T22:45:38.758 INFO:tasks.workunit.client.1.smithi192.stdout:5/521: write d0/d1b/f25 [164958,130598] 0 2024-04-24T22:45:38.759 INFO:tasks.workunit.client.1.smithi192.stdout:3/670: truncate d2/d4/d12/d39/d97/f9e 1069797 0 2024-04-24T22:45:38.763 INFO:tasks.workunit.client.0.smithi063.stdout:5/766: dread d5/d11/d23/d2d/d62/d81/da4/da6/fc6 [0,4194304] 0 2024-04-24T22:45:38.763 INFO:tasks.workunit.client.0.smithi063.stdout:8/835: creat d1/d2/f10c x:0 0 0 2024-04-24T22:45:38.765 INFO:tasks.workunit.client.1.smithi192.stdout:2/760: symlink d4/d9/d10/d6c/d95/lf7 0 2024-04-24T22:45:38.771 INFO:tasks.workunit.client.0.smithi063.stdout:0/736: truncate d6/d74/f7e 290528 0 2024-04-24T22:45:38.771 INFO:tasks.workunit.client.0.smithi063.stdout:9/809: creat d6/d2f/dcd/db/d6b/d4f/dad/f10c x:0 0 0 2024-04-24T22:45:38.773 INFO:tasks.workunit.client.1.smithi192.stdout:3/671: dwrite d2/d27/d28/dab/fc2 [0,4194304] 0 2024-04-24T22:45:38.773 INFO:tasks.workunit.client.1.smithi192.stdout:9/714: read - d3/d59/d7d/f8a zero size 2024-04-24T22:45:38.773 INFO:tasks.workunit.client.1.smithi192.stdout:6/556: truncate d0/d8/d3d/f62 63132 0 2024-04-24T22:45:38.773 INFO:tasks.workunit.client.1.smithi192.stdout:6/557: dread - d0/d8/d29/d58/f9f zero size 2024-04-24T22:45:38.774 INFO:tasks.workunit.client.0.smithi063.stdout:0/737: dread d6/da/f10 [0,4194304] 0 2024-04-24T22:45:38.776 INFO:tasks.workunit.client.0.smithi063.stdout:6/850: rmdir db/d1c/dea 39 2024-04-24T22:45:38.777 INFO:tasks.workunit.client.0.smithi063.stdout:4/779: dwrite d2/ff [0,4194304] 0 2024-04-24T22:45:38.779 INFO:tasks.workunit.client.1.smithi192.stdout:1/541: truncate d9/d15/d2d/d31/f99 3343420 0 2024-04-24T22:45:38.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/850: dread - d3/db5/d21/d9f/dd5/d106/df5/f103 zero size 2024-04-24T22:45:38.780 INFO:tasks.workunit.client.1.smithi192.stdout:1/542: chown d9/dd/d74/caa 1 1 2024-04-24T22:45:38.781 INFO:tasks.workunit.client.1.smithi192.stdout:2/761: dread d4/d29/d60/d9a/f7b [0,4194304] 0 2024-04-24T22:45:38.787 INFO:tasks.workunit.client.1.smithi192.stdout:3/672: write d2/d4/d12/d1a/d1d/d55/d5f/f80 [4055136,92815] 0 2024-04-24T22:45:38.788 INFO:tasks.workunit.client.1.smithi192.stdout:2/762: dread d4/d9/d37/f62 [0,4194304] 0 2024-04-24T22:45:38.790 INFO:tasks.workunit.client.1.smithi192.stdout:9/715: unlink d3/dd/de/d17/f1d 0 2024-04-24T22:45:38.792 INFO:tasks.workunit.client.1.smithi192.stdout:5/522: dwrite d0/d1b/f45 [0,4194304] 0 2024-04-24T22:45:38.793 INFO:tasks.workunit.client.0.smithi063.stdout:7/736: getdents d0/d4/de/d6f/d2e/d76 0 2024-04-24T22:45:38.795 INFO:tasks.workunit.client.0.smithi063.stdout:7/737: write d0/d4/fe1 [266655,97715] 0 2024-04-24T22:45:38.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/892: sync 2024-04-24T22:45:38.798 INFO:tasks.workunit.client.0.smithi063.stdout:0/738: fdatasync d6/d3a/dba/fe0 0 2024-04-24T22:45:38.805 INFO:tasks.workunit.client.1.smithi192.stdout:3/673: symlink d2/d62/db2/ld3 0 2024-04-24T22:45:38.812 INFO:tasks.workunit.client.0.smithi063.stdout:9/810: dwrite d6/d2f/dcd/db/d6b/d4f/dad/f34 [0,4194304] 0 2024-04-24T22:45:38.812 INFO:tasks.workunit.client.1.smithi192.stdout:2/763: rmdir d4/d9/d37 39 2024-04-24T22:45:38.812 INFO:tasks.workunit.client.1.smithi192.stdout:9/716: stat d3/dd/de/l1b 0 2024-04-24T22:45:38.813 INFO:tasks.workunit.client.0.smithi063.stdout:7/738: mknod d0/d4/de/d6f/d2e/d55/ddc/cf5 0 2024-04-24T22:45:38.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/851: dread db/d1d/d2d/d69/d82/f9a [0,4194304] 0 2024-04-24T22:45:38.814 INFO:tasks.workunit.client.0.smithi063.stdout:5/767: dwrite d5/d11/d76/d92/dea/ff0 [0,4194304] 0 2024-04-24T22:45:38.815 INFO:tasks.workunit.client.0.smithi063.stdout:7/739: fdatasync d0/d4/de/d17/d28/fb8 0 2024-04-24T22:45:38.816 INFO:tasks.workunit.client.0.smithi063.stdout:2/893: truncate d0/d2/d35/dc6/de2/f6b 525568 0 2024-04-24T22:45:38.817 INFO:tasks.workunit.client.0.smithi063.stdout:2/894: dread - d0/dd/d72/d77/d87/fc8 zero size 2024-04-24T22:45:38.817 INFO:tasks.workunit.client.0.smithi063.stdout:8/836: truncate d1/f58 244519 0 2024-04-24T22:45:38.818 INFO:tasks.workunit.client.0.smithi063.stdout:7/740: truncate d0/d4/de/d6f/d2e/ff0 77221 0 2024-04-24T22:45:38.822 INFO:tasks.workunit.client.0.smithi063.stdout:3/802: rename da/d18/d47/d4d/d62/d109 to da/d18/d115 0 2024-04-24T22:45:38.824 INFO:tasks.workunit.client.1.smithi192.stdout:2/764: write d4/d9/d10/d6c/d8c/fc6 [1711697,98888] 0 2024-04-24T22:45:38.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/558: creat d0/fc2 x:0 0 0 2024-04-24T22:45:38.827 INFO:tasks.workunit.client.0.smithi063.stdout:4/780: write d2/f99 [3341098,15450] 0 2024-04-24T22:45:38.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/811: write d6/d2f/dcd/d53/dd5/fe1 [660234,91212] 0 2024-04-24T22:45:38.829 INFO:tasks.workunit.client.1.smithi192.stdout:9/717: fsync d3/dd/de/d17/f68 0 2024-04-24T22:45:38.829 INFO:tasks.workunit.client.1.smithi192.stdout:1/543: dwrite d9/d51/d70/f94 [0,4194304] 0 2024-04-24T22:45:38.833 INFO:tasks.workunit.client.0.smithi063.stdout:5/768: chown d5/d24/d36/d65/cc2 896 1 2024-04-24T22:45:38.834 INFO:tasks.workunit.client.1.smithi192.stdout:5/523: link d0/d13/d3b/f55 d0/d13/d19/d64/d5f/d2c/d3c/d91/fb3 0 2024-04-24T22:45:38.835 INFO:tasks.workunit.client.1.smithi192.stdout:3/674: mknod d2/d4/d12/d1a/cd4 0 2024-04-24T22:45:38.836 INFO:tasks.workunit.client.0.smithi063.stdout:8/837: truncate d1/d22/d33/fc4 370749 0 2024-04-24T22:45:38.838 INFO:tasks.workunit.client.0.smithi063.stdout:7/741: mknod d0/d4/de/d95/cf6 0 2024-04-24T22:45:38.839 INFO:tasks.workunit.client.0.smithi063.stdout:7/742: write d0/d4/de/d17/d35/f54 [4870358,73039] 0 2024-04-24T22:45:38.843 INFO:tasks.workunit.client.1.smithi192.stdout:9/718: creat d3/dd/d53/ddd/ffe x:0 0 0 2024-04-24T22:45:38.844 INFO:tasks.workunit.client.0.smithi063.stdout:1/851: rename d3/db5/d21/d49/f9e to d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f113 0 2024-04-24T22:45:38.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/852: dread db/d1d/d20/f40 [0,4194304] 0 2024-04-24T22:45:38.849 INFO:tasks.workunit.client.1.smithi192.stdout:2/765: rename d4/d9/d37/c6a to d4/d29/d60/d9a/cf8 0 2024-04-24T22:45:38.850 INFO:tasks.workunit.client.0.smithi063.stdout:5/769: dread d5/d24/d49/f97 [0,4194304] 0 2024-04-24T22:45:38.851 INFO:tasks.workunit.client.1.smithi192.stdout:0/653: sync 2024-04-24T22:45:38.852 INFO:tasks.workunit.client.0.smithi063.stdout:8/838: dread d1/d2/d3/f59 [0,4194304] 0 2024-04-24T22:45:38.853 INFO:tasks.workunit.client.1.smithi192.stdout:0/654: write d3/d27/d28/d2e/dc7/fd0 [243623,57492] 0 2024-04-24T22:45:38.861 INFO:tasks.workunit.client.1.smithi192.stdout:1/544: mkdir d9/d15/d2d/d48/d93/da4/db5 0 2024-04-24T22:45:38.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/545: dwrite d9/d51/f54 [0,4194304] 0 2024-04-24T22:45:38.872 INFO:tasks.workunit.client.0.smithi063.stdout:0/739: rename d6/da/d2b/d60/f76 to d6/d64/da0/d98/dd6/ffc 0 2024-04-24T22:45:38.873 INFO:tasks.workunit.client.1.smithi192.stdout:6/559: creat d0/d8/d3d/fc3 x:0 0 0 2024-04-24T22:45:38.877 INFO:tasks.workunit.client.0.smithi063.stdout:9/812: mknod d6/d2f/dcd/d1f/d20/d22/c10d 0 2024-04-24T22:45:38.880 INFO:tasks.workunit.client.1.smithi192.stdout:5/524: dwrite d0/f73 [0,4194304] 0 2024-04-24T22:45:38.881 INFO:tasks.workunit.client.1.smithi192.stdout:5/525: chown d0/d1b/c2e 3500 1 2024-04-24T22:45:38.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/675: creat d2/d4/d12/fd5 x:0 0 0 2024-04-24T22:45:38.882 INFO:tasks.workunit.client.1.smithi192.stdout:9/719: write d3/dd/de/dba/f66 [797752,116184] 0 2024-04-24T22:45:38.883 INFO:tasks.workunit.client.1.smithi192.stdout:1/546: truncate d9/f4f 267594 0 2024-04-24T22:45:38.883 INFO:tasks.workunit.client.0.smithi063.stdout:2/895: dwrite d0/dd/d72/d77/dda/d7c/fb9 [0,4194304] 0 2024-04-24T22:45:38.884 INFO:tasks.workunit.client.0.smithi063.stdout:5/770: stat d5/d11/f15 0 2024-04-24T22:45:38.889 INFO:tasks.workunit.client.1.smithi192.stdout:2/766: write d4/d9/d10/d6c/f8e [4143667,84424] 0 2024-04-24T22:45:38.890 INFO:tasks.workunit.client.1.smithi192.stdout:3/676: read d2/d27/dc6/f60 [2497097,88314] 0 2024-04-24T22:45:38.894 INFO:tasks.workunit.client.0.smithi063.stdout:3/803: dwrite da/f1a [4194304,4194304] 0 2024-04-24T22:45:38.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/804: fdatasync da/d39/d94/db3/fd2 0 2024-04-24T22:45:38.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/853: write db/d1d/d20/f38 [3734398,44729] 0 2024-04-24T22:45:38.897 INFO:tasks.workunit.client.0.smithi063.stdout:3/805: fdatasync da/d18/f30 0 2024-04-24T22:45:38.899 INFO:tasks.workunit.client.1.smithi192.stdout:5/526: creat d0/d1b/d62/fb4 x:0 0 0 2024-04-24T22:45:38.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/740: mknod d6/d74/cfd 0 2024-04-24T22:45:38.912 INFO:tasks.workunit.client.1.smithi192.stdout:6/560: mknod d0/d8/d29/cc4 0 2024-04-24T22:45:38.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/720: creat d3/dd/de/dba/dbc/fff x:0 0 0 2024-04-24T22:45:38.913 INFO:tasks.workunit.client.1.smithi192.stdout:2/767: rename d4/d9/d37/f62 to d4/d9/d10/da7/df5/ff9 0 2024-04-24T22:45:38.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/561: dread - d0/d8/d3d/d3e/d7e/fa1 zero size 2024-04-24T22:45:38.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/562: dread - d0/fc2 zero size 2024-04-24T22:45:38.918 INFO:tasks.workunit.client.1.smithi192.stdout:7/696: sync 2024-04-24T22:45:38.918 INFO:tasks.workunit.client.1.smithi192.stdout:4/619: sync 2024-04-24T22:45:38.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/623: sync 2024-04-24T22:45:38.918 INFO:tasks.workunit.client.0.smithi063.stdout:8/839: fsync d1/d11/f29 0 2024-04-24T22:45:38.919 INFO:tasks.workunit.client.0.smithi063.stdout:8/840: stat d1/d21/de6 0 2024-04-24T22:45:38.919 INFO:tasks.workunit.client.0.smithi063.stdout:8/841: dread - d1/dc0/fe0 zero size 2024-04-24T22:45:38.920 INFO:tasks.workunit.client.0.smithi063.stdout:8/842: readlink d1/d22/l30 0 2024-04-24T22:45:38.920 INFO:tasks.workunit.client.1.smithi192.stdout:1/547: dread d9/f14 [0,4194304] 0 2024-04-24T22:45:38.921 INFO:tasks.workunit.client.0.smithi063.stdout:8/843: fsync d1/d22/d33/da6/fdb 0 2024-04-24T22:45:38.921 INFO:tasks.workunit.client.1.smithi192.stdout:8/624: chown db/l60 16 1 2024-04-24T22:45:38.923 INFO:tasks.workunit.client.1.smithi192.stdout:7/697: dread - d2/db/d21/fc6 zero size 2024-04-24T22:45:38.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/721: dwrite d3/dd/de/d17/d89/de1/fe7 [0,4194304] 0 2024-04-24T22:45:38.924 INFO:tasks.workunit.client.0.smithi063.stdout:6/854: symlink db/d1d/d8d/dfc/l11b 0 2024-04-24T22:45:38.925 INFO:tasks.workunit.client.1.smithi192.stdout:0/655: truncate d3/d27/d28/f8e 2971876 0 2024-04-24T22:45:38.926 INFO:tasks.workunit.client.0.smithi063.stdout:6/855: write db/d1d/d20/d3f/d7f/d68/f7e [320066,63909] 0 2024-04-24T22:45:38.926 INFO:tasks.workunit.client.1.smithi192.stdout:9/722: chown d3/dd/de/d17/l80 1 1 2024-04-24T22:45:38.930 INFO:tasks.workunit.client.0.smithi063.stdout:7/743: write d0/d4/de/d6f/d5c/d8b/fae [3903033,20096] 0 2024-04-24T22:45:38.935 INFO:tasks.workunit.client.0.smithi063.stdout:4/781: creat d2/d8/dd4/dda/ffe x:0 0 0 2024-04-24T22:45:38.938 INFO:tasks.workunit.client.0.smithi063.stdout:1/852: sync 2024-04-24T22:45:38.939 INFO:tasks.workunit.client.0.smithi063.stdout:7/744: dread d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:38.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/856: mkdir db/d1d/d8d/dfc/d11c 0 2024-04-24T22:45:38.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/857: write db/d1d/d20/fa1 [505095,71989] 0 2024-04-24T22:45:38.950 INFO:tasks.workunit.client.0.smithi063.stdout:2/896: rmdir d0/d2/d2d/d31/d89/d1c/d3f/dca/d118 0 2024-04-24T22:45:38.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/782: dread d2/d19/d33/dc1/d50/f5f [0,4194304] 0 2024-04-24T22:45:38.952 INFO:tasks.workunit.client.0.smithi063.stdout:5/771: truncate d5/d24/f91 2122805 0 2024-04-24T22:45:38.953 INFO:tasks.workunit.client.0.smithi063.stdout:2/897: fsync d0/d2/d2d/d31/d89/d1c/f2a 0 2024-04-24T22:45:38.953 INFO:tasks.workunit.client.0.smithi063.stdout:4/783: chown d2/d8/dd4/dda 4 1 2024-04-24T22:45:38.957 INFO:tasks.workunit.client.0.smithi063.stdout:8/844: symlink d1/l10d 0 2024-04-24T22:45:38.958 INFO:tasks.workunit.client.0.smithi063.stdout:2/898: dwrite d0/ff8 [0,4194304] 0 2024-04-24T22:45:38.961 INFO:tasks.workunit.client.0.smithi063.stdout:6/858: symlink db/d1c/dc3/ddd/l11d 0 2024-04-24T22:45:38.963 INFO:tasks.workunit.client.0.smithi063.stdout:3/806: rmdir da/d39/d94/db3/dbe/df7 0 2024-04-24T22:45:38.967 INFO:tasks.workunit.client.0.smithi063.stdout:9/813: link d6/d2f/d42/f6f d6/d2f/dcd/db/d6b/d4f/dad/d36/f10e 0 2024-04-24T22:45:38.967 INFO:tasks.workunit.client.0.smithi063.stdout:2/899: dwrite d0/d2/d2d/d31/d89/d7/f64 [0,4194304] 0 2024-04-24T22:45:38.967 INFO:tasks.workunit.client.0.smithi063.stdout:1/853: dwrite d3/db5/d21/f69 [0,4194304] 0 2024-04-24T22:45:38.983 INFO:tasks.workunit.client.0.smithi063.stdout:6/859: chown db/d1d/d2d/d69/dd4/cf9 2393 1 2024-04-24T22:45:38.984 INFO:tasks.workunit.client.1.smithi192.stdout:2/768: creat d4/d9/d10/d6c/d95/db7/dbf/ffa x:0 0 0 2024-04-24T22:45:38.984 INFO:tasks.workunit.client.0.smithi063.stdout:6/860: truncate db/d1d/d8d/dfc/f10c 683548 0 2024-04-24T22:45:38.985 INFO:tasks.workunit.client.0.smithi063.stdout:0/741: getdents d6/d64/d21/d25/d62/d63/d7c 0 2024-04-24T22:45:38.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/563: unlink d0/f94 0 2024-04-24T22:45:38.987 INFO:tasks.workunit.client.0.smithi063.stdout:3/807: symlink da/d18/d47/d4d/d9b/dde/df9/l116 0 2024-04-24T22:45:38.990 INFO:tasks.workunit.client.0.smithi063.stdout:9/814: rename d6/d2f/dcd/d1f to d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e/d10f 22 2024-04-24T22:45:38.991 INFO:tasks.workunit.client.1.smithi192.stdout:7/698: mknod d2/db/d21/d42/d67/dae/dbb/cdd 0 2024-04-24T22:45:38.992 INFO:tasks.workunit.client.0.smithi063.stdout:1/854: dread d3/de0/d79/f2e [0,4194304] 0 2024-04-24T22:45:38.993 INFO:tasks.workunit.client.0.smithi063.stdout:1/855: chown d3/db5/d21/d49/fd2 1765350 1 2024-04-24T22:45:38.993 INFO:tasks.workunit.client.0.smithi063.stdout:5/772: mknod d5/d11/d23/daf/c106 0 2024-04-24T22:45:38.994 INFO:tasks.workunit.client.0.smithi063.stdout:1/856: stat d3/de0/d79/d96/d112/cf6 0 2024-04-24T22:45:38.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/857: fsync d3/da/d34/d99/ff9 0 2024-04-24T22:45:38.998 INFO:tasks.workunit.client.1.smithi192.stdout:9/723: dread d3/f45 [0,4194304] 0 2024-04-24T22:45:38.999 INFO:tasks.workunit.client.0.smithi063.stdout:8/845: dread d1/d21/f6f [0,4194304] 0 2024-04-24T22:45:39.000 INFO:tasks.workunit.client.1.smithi192.stdout:5/527: link d0/f4a d0/d13/d19/d64/d4f/d6c/fb5 0 2024-04-24T22:45:39.000 INFO:tasks.workunit.client.1.smithi192.stdout:4/620: write d2/db/d75/da1/d36/d5b/f24 [3898459,14043] 0 2024-04-24T22:45:39.001 INFO:tasks.workunit.client.1.smithi192.stdout:4/621: readlink d2/db/d10/l1a 0 2024-04-24T22:45:39.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/677: creat d2/d4/d12/d39/fd6 x:0 0 0 2024-04-24T22:45:39.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/564: creat d0/d8/d29/d4c/fc5 x:0 0 0 2024-04-24T22:45:39.011 INFO:tasks.workunit.client.0.smithi063.stdout:0/742: dwrite d6/d64/d21/d25/d31/f35 [0,4194304] 0 2024-04-24T22:45:39.013 INFO:tasks.workunit.client.1.smithi192.stdout:7/699: mkdir d2/db/d36/dde 0 2024-04-24T22:45:39.014 INFO:tasks.workunit.client.0.smithi063.stdout:7/745: sync 2024-04-24T22:45:39.015 INFO:tasks.workunit.client.1.smithi192.stdout:1/548: rename d9/d15/d2d/d63/d68/ca0 to d9/d15/d2d/d31/d2f/dae/cb6 0 2024-04-24T22:45:39.016 INFO:tasks.workunit.client.1.smithi192.stdout:6/565: dread d0/d8/fe [0,4194304] 0 2024-04-24T22:45:39.017 INFO:tasks.workunit.client.1.smithi192.stdout:2/769: dwrite d4/d29/f26 [4194304,4194304] 0 2024-04-24T22:45:39.018 INFO:tasks.workunit.client.0.smithi063.stdout:7/746: write d0/d4/de/d17/d35/d69/f6b [730789,63975] 0 2024-04-24T22:45:39.018 INFO:tasks.workunit.client.0.smithi063.stdout:6/861: mkdir db/d1d/d20/d11e 0 2024-04-24T22:45:39.020 INFO:tasks.workunit.client.0.smithi063.stdout:8/846: creat d1/d22/d33/dbd/f10e x:0 0 0 2024-04-24T22:45:39.022 INFO:tasks.workunit.client.0.smithi063.stdout:3/808: truncate da/f29 1239447 0 2024-04-24T22:45:39.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/809: fsync da/d18/d9e/fed 0 2024-04-24T22:45:39.023 INFO:tasks.workunit.client.1.smithi192.stdout:4/622: mknod d2/db/d75/da1/d36/cd1 0 2024-04-24T22:45:39.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/528: chown d0/d13/d19/cac 976205 1 2024-04-24T22:45:39.026 INFO:tasks.workunit.client.1.smithi192.stdout:5/529: write d0/d13/d19/f1a [3449536,22902] 0 2024-04-24T22:45:39.028 INFO:tasks.workunit.client.0.smithi063.stdout:3/810: dread da/f28 [0,4194304] 0 2024-04-24T22:45:39.028 INFO:tasks.workunit.client.1.smithi192.stdout:8/625: creat db/dc/d22/d31/fc1 x:0 0 0 2024-04-24T22:45:39.029 INFO:tasks.workunit.client.1.smithi192.stdout:6/566: dwrite d0/d8/d3d/d9d/fb2 [0,4194304] 0 2024-04-24T22:45:39.030 INFO:tasks.workunit.client.0.smithi063.stdout:5/773: symlink d5/d24/l107 0 2024-04-24T22:45:39.030 INFO:tasks.workunit.client.0.smithi063.stdout:0/743: mknod d6/d64/d21/d25/d62/cfe 0 2024-04-24T22:45:39.030 INFO:tasks.workunit.client.0.smithi063.stdout:0/744: fdatasync d6/d64/d21/d9c/fbd 0 2024-04-24T22:45:39.031 INFO:tasks.workunit.client.1.smithi192.stdout:7/700: fsync d2/db/d21/d42/d67/d9b/d6e/f9e 0 2024-04-24T22:45:39.032 INFO:tasks.workunit.client.0.smithi063.stdout:0/745: chown d6/d64/d21/d25/d62/d63/d89/d95/da4/fc2 15031540 1 2024-04-24T22:45:39.033 INFO:tasks.workunit.client.1.smithi192.stdout:7/701: chown d2/db/d21/d42/d67/ca1 1239691590 1 2024-04-24T22:45:39.035 INFO:tasks.workunit.client.0.smithi063.stdout:4/784: link d2/d8/d5c/l98 d2/d80/d81/d90/lff 0 2024-04-24T22:45:39.036 INFO:tasks.workunit.client.1.smithi192.stdout:1/549: creat d9/dd/d74/d83/fb7 x:0 0 0 2024-04-24T22:45:39.037 INFO:tasks.workunit.client.0.smithi063.stdout:7/747: rename d0/d4/de/d6f/f53 to d0/d4/de/d17/d28/d31/dad/ff7 0 2024-04-24T22:45:39.041 INFO:tasks.workunit.client.1.smithi192.stdout:3/678: mknod d2/d4/d12/d1a/d1d/d55/d5f/d9b/cd7 0 2024-04-24T22:45:39.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/847: symlink d1/dc0/l10f 0 2024-04-24T22:45:39.043 INFO:tasks.workunit.client.0.smithi063.stdout:2/900: link d0/d2/d2d/d31/d89/l8b d0/d2/d2d/l119 0 2024-04-24T22:45:39.045 INFO:tasks.workunit.client.1.smithi192.stdout:5/530: symlink d0/d13/d19/d64/d4f/lb6 0 2024-04-24T22:45:39.048 INFO:tasks.workunit.client.1.smithi192.stdout:9/724: dread d3/d24/f4e [0,4194304] 0 2024-04-24T22:45:39.049 INFO:tasks.workunit.client.1.smithi192.stdout:8/626: mknod db/d43/d76/d92/cc2 0 2024-04-24T22:45:39.051 INFO:tasks.workunit.client.0.smithi063.stdout:4/785: mknod d2/d19/d82/c100 0 2024-04-24T22:45:39.056 INFO:tasks.workunit.client.0.smithi063.stdout:8/848: unlink d1/la8 0 2024-04-24T22:45:39.057 INFO:tasks.workunit.client.1.smithi192.stdout:3/679: dread d2/d4/d12/fb4 [0,4194304] 0 2024-04-24T22:45:39.062 INFO:tasks.workunit.client.0.smithi063.stdout:2/901: rename d0/d2/d35/dc6 to d0/d2/d39/d11a 0 2024-04-24T22:45:39.063 INFO:tasks.workunit.client.1.smithi192.stdout:0/656: getdents d3/d27/d6a 0 2024-04-24T22:45:39.064 INFO:tasks.workunit.client.0.smithi063.stdout:9/815: truncate d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e/f98 1077104 0 2024-04-24T22:45:39.065 INFO:tasks.workunit.client.1.smithi192.stdout:3/680: dwrite d2/d4/d12/d1a/d1d/d7b/fb6 [0,4194304] 0 2024-04-24T22:45:39.067 INFO:tasks.workunit.client.1.smithi192.stdout:4/623: creat d2/d5c/dce/fd2 x:0 0 0 2024-04-24T22:45:39.068 INFO:tasks.workunit.client.1.smithi192.stdout:4/624: truncate d2/db/d75/da1/d7d/fca 193862 0 2024-04-24T22:45:39.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/531: symlink d0/d13/d19/d64/d4f/d7c/lb7 0 2024-04-24T22:45:39.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/858: link d3/db5/d21/d9f/dd5/d106/c13 d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/c114 0 2024-04-24T22:45:39.074 INFO:tasks.workunit.client.1.smithi192.stdout:9/725: unlink d3/dd/c29 0 2024-04-24T22:45:39.078 INFO:tasks.workunit.client.0.smithi063.stdout:8/849: chown d1/d22/f31 13109 1 2024-04-24T22:45:39.078 INFO:tasks.workunit.client.0.smithi063.stdout:8/850: readlink d1/d22/d33/da6/d54/d7b/la4 0 2024-04-24T22:45:39.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/567: write d0/d8/d3d/f49 [869476,4141] 0 2024-04-24T22:45:39.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/851: truncate d1/d11/d32/f100 962990 0 2024-04-24T22:45:39.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/786: rename d2/d8/dd4/dda/dc6/fe5 to d2/d19/d33/dc1/d50/dc3/f101 0 2024-04-24T22:45:39.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/787: write d2/d80/fb0 [112538,12308] 0 2024-04-24T22:45:39.083 INFO:tasks.workunit.client.1.smithi192.stdout:2/770: creat d4/d9/d10/d6c/ffb x:0 0 0 2024-04-24T22:45:39.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/788: fdatasync d2/f7c 0 2024-04-24T22:45:39.084 INFO:tasks.workunit.client.0.smithi063.stdout:2/902: mkdir d0/d2/d2f/dba/d11b 0 2024-04-24T22:45:39.087 INFO:tasks.workunit.client.0.smithi063.stdout:3/811: creat da/d18/d3a/f117 x:0 0 0 2024-04-24T22:45:39.088 INFO:tasks.workunit.client.0.smithi063.stdout:3/812: read da/d18/f2a [131550,61848] 0 2024-04-24T22:45:39.089 INFO:tasks.workunit.client.1.smithi192.stdout:1/550: mkdir d9/db8 0 2024-04-24T22:45:39.089 INFO:tasks.workunit.client.1.smithi192.stdout:1/551: write d9/f3c [4645655,12474] 0 2024-04-24T22:45:39.098 INFO:tasks.workunit.client.0.smithi063.stdout:9/816: unlink d6/d2f/dcd/db/d1c/d78/d9a/daa/fd4 0 2024-04-24T22:45:39.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/552: dwrite d9/d15/d5e/d67/fa2 [0,4194304] 0 2024-04-24T22:45:39.099 INFO:tasks.workunit.client.1.smithi192.stdout:1/553: stat d9/d15/d2d/d63/d68/c89 0 2024-04-24T22:45:39.100 INFO:tasks.workunit.client.1.smithi192.stdout:1/554: chown d9/d15/d2d 879792003 1 2024-04-24T22:45:39.105 INFO:tasks.workunit.client.1.smithi192.stdout:3/681: read d2/d4/f14 [199181,129429] 0 2024-04-24T22:45:39.106 INFO:tasks.workunit.client.0.smithi063.stdout:7/748: link d0/d4/de/d95/cf6 d0/d4/de/d17/d28/d31/cf8 0 2024-04-24T22:45:39.107 INFO:tasks.workunit.client.1.smithi192.stdout:4/625: mkdir d2/db/d75/dd3 0 2024-04-24T22:45:39.108 INFO:tasks.workunit.client.1.smithi192.stdout:3/682: truncate d2/d27/dc6/da0/fce 140279 0 2024-04-24T22:45:39.108 INFO:tasks.workunit.client.0.smithi063.stdout:0/746: write d6/da/dc/f1e [486178,41398] 0 2024-04-24T22:45:39.109 INFO:tasks.workunit.client.1.smithi192.stdout:5/532: mknod d0/d13/d19/d64/d5f/d2c/cb8 0 2024-04-24T22:45:39.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/747: fsync d6/d64/d21/d25/d62/d63/d89/d95/fe8 0 2024-04-24T22:45:39.111 INFO:tasks.workunit.client.0.smithi063.stdout:8/852: mkdir d1/d22/d33/da6/d54/d7b/d110 0 2024-04-24T22:45:39.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/726: mkdir d3/dd/d53/d100 0 2024-04-24T22:45:39.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/727: dread - d3/dd/de/dba/dbc/fff zero size 2024-04-24T22:45:39.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/626: dwrite d2/db/d75/da1/d7d/dbd/fc3 [0,4194304] 0 2024-04-24T22:45:39.116 INFO:tasks.workunit.client.0.smithi063.stdout:0/748: dwrite d6/da/d2b/d60/faf [0,4194304] 0 2024-04-24T22:45:39.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/728: stat d3/dd/f3a 0 2024-04-24T22:45:39.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/789: symlink d2/d8/d40/d51/dd5/df6/l102 0 2024-04-24T22:45:39.117 INFO:tasks.workunit.client.1.smithi192.stdout:9/729: dread - d3/dd/de/dba/dbc/fff zero size 2024-04-24T22:45:39.119 INFO:tasks.workunit.client.0.smithi063.stdout:2/903: dread - d0/d2/d2d/d31/f92 zero size 2024-04-24T22:45:39.120 INFO:tasks.workunit.client.0.smithi063.stdout:2/904: chown d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa 31233 1 2024-04-24T22:45:39.120 INFO:tasks.workunit.client.1.smithi192.stdout:9/730: truncate d3/dd/de/d17/d89/d8b/f99 1132798 0 2024-04-24T22:45:39.122 INFO:tasks.workunit.client.1.smithi192.stdout:9/731: read - d3/dd/de/dba/dbc/fff zero size 2024-04-24T22:45:39.127 INFO:tasks.workunit.client.0.smithi063.stdout:3/813: creat da/d18/d47/d4d/d9b/f118 x:0 0 0 2024-04-24T22:45:39.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/814: stat da/d18/d3a/d63/c83 0 2024-04-24T22:45:39.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/817: unlink d6/d2f/dcd/l41 0 2024-04-24T22:45:39.133 INFO:tasks.workunit.client.1.smithi192.stdout:7/702: dwrite d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:39.133 INFO:tasks.workunit.client.0.smithi063.stdout:1/859: unlink d3/db5/d21/l84 0 2024-04-24T22:45:39.137 INFO:tasks.workunit.client.0.smithi063.stdout:7/749: rmdir d0/d4/de/d6f/d2e 39 2024-04-24T22:45:39.138 INFO:tasks.workunit.client.0.smithi063.stdout:7/750: stat d0/d4/de/d6f/fd6 0 2024-04-24T22:45:39.138 INFO:tasks.workunit.client.1.smithi192.stdout:0/657: fdatasync d3/d27/d28/dcc/d5c/f61 0 2024-04-24T22:45:39.139 INFO:tasks.workunit.client.1.smithi192.stdout:0/658: chown d3/fc0 1 1 2024-04-24T22:45:39.141 INFO:tasks.workunit.client.0.smithi063.stdout:8/853: stat d1/d11/d32/d38/ccc 0 2024-04-24T22:45:39.142 INFO:tasks.workunit.client.0.smithi063.stdout:4/790: creat d2/d80/f103 x:0 0 0 2024-04-24T22:45:39.143 INFO:tasks.workunit.client.0.smithi063.stdout:4/791: write d2/d19/d33/dc1/d50/f9e [2381185,5337] 0 2024-04-24T22:45:39.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/749: mkdir d6/d64/d21/d25/d62/d63/d89/d95/da4/dff 0 2024-04-24T22:45:39.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/862: sync 2024-04-24T22:45:39.147 INFO:tasks.workunit.client.0.smithi063.stdout:4/792: dread d2/f2e [0,4194304] 0 2024-04-24T22:45:39.148 INFO:tasks.workunit.client.1.smithi192.stdout:5/533: creat d0/d13/d19/d71/d88/fb9 x:0 0 0 2024-04-24T22:45:39.153 INFO:tasks.workunit.client.0.smithi063.stdout:3/815: write da/f1d [2029015,26596] 0 2024-04-24T22:45:39.153 INFO:tasks.workunit.client.1.smithi192.stdout:9/732: truncate d3/dd/fa4 770578 0 2024-04-24T22:45:39.155 INFO:tasks.workunit.client.0.smithi063.stdout:9/818: mkdir d6/d2f/dcd/db/d6b/d4f/dad/d110 0 2024-04-24T22:45:39.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/819: stat d6/d2f/dcd/db/d93/da0/fdf 0 2024-04-24T22:45:39.157 INFO:tasks.workunit.client.1.smithi192.stdout:6/568: mkdir d0/d8/d3d/dc6 0 2024-04-24T22:45:39.159 INFO:tasks.workunit.client.0.smithi063.stdout:0/750: dread d6/d64/d21/d25/d62/d63/d89/d95/fdc [0,4194304] 0 2024-04-24T22:45:39.161 INFO:tasks.workunit.client.0.smithi063.stdout:1/860: mkdir d3/db5/d21/d9f/dd5/d106/d39/d115 0 2024-04-24T22:45:39.163 INFO:tasks.workunit.client.1.smithi192.stdout:7/703: creat d2/db/d21/d42/d67/d9b/d6e/db5/fdf x:0 0 0 2024-04-24T22:45:39.172 INFO:tasks.workunit.client.0.smithi063.stdout:8/854: readlink d1/d22/l64 0 2024-04-24T22:45:39.177 INFO:tasks.workunit.client.1.smithi192.stdout:7/704: chown d2/db/d21/d42/d5b 89925 1 2024-04-24T22:45:39.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/683: dread d2/d27/f4a [0,4194304] 0 2024-04-24T22:45:39.177 INFO:tasks.workunit.client.1.smithi192.stdout:1/555: write d9/f42 [4875617,39330] 0 2024-04-24T22:45:39.178 INFO:tasks.workunit.client.1.smithi192.stdout:0/659: rmdir d3/d27/d3c/d6c 39 2024-04-24T22:45:39.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:38 smithi192 ceph-mon[33991]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 285 MiB/s wr, 556 op/s 2024-04-24T22:45:39.183 INFO:tasks.workunit.client.1.smithi192.stdout:8/627: getdents db/dc/d22/d31/d33/d81 0 2024-04-24T22:45:39.184 INFO:tasks.workunit.client.0.smithi063.stdout:7/751: dwrite d0/d4/de/d6f/d2e/d76/ff3 [0,4194304] 0 2024-04-24T22:45:39.184 INFO:tasks.workunit.client.0.smithi063.stdout:5/774: sync 2024-04-24T22:45:39.184 INFO:tasks.workunit.client.0.smithi063.stdout:4/793: rmdir d2/d19 39 2024-04-24T22:45:39.184 INFO:tasks.workunit.client.0.smithi063.stdout:3/816: dread da/d39/d94/db3/fc8 [0,4194304] 0 2024-04-24T22:45:39.186 INFO:tasks.workunit.client.1.smithi192.stdout:9/733: creat d3/dd/de/dba/f101 x:0 0 0 2024-04-24T22:45:39.187 INFO:tasks.workunit.client.1.smithi192.stdout:9/734: chown d3/dd/de/d17/f8f 170863315 1 2024-04-24T22:45:39.189 INFO:tasks.workunit.client.0.smithi063.stdout:1/861: dread d3/da/f82 [0,4194304] 0 2024-04-24T22:45:39.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/905: dwrite d0/d2/d2d/f4d [0,4194304] 0 2024-04-24T22:45:39.190 INFO:tasks.workunit.client.0.smithi063.stdout:1/862: read - d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/fd3 zero size 2024-04-24T22:45:39.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/628: dwrite db/d43/f4e [0,4194304] 0 2024-04-24T22:45:39.191 INFO:tasks.workunit.client.1.smithi192.stdout:6/569: mkdir d0/d8/d3d/d3e/d40/d44/dc7 0 2024-04-24T22:45:39.191 INFO:tasks.workunit.client.1.smithi192.stdout:6/570: stat d0/d8/d18/ca3 0 2024-04-24T22:45:39.191 INFO:tasks.workunit.client.1.smithi192.stdout:6/571: readlink d0/d8/d3d/d3e/d40/l65 0 2024-04-24T22:45:39.191 INFO:tasks.workunit.client.1.smithi192.stdout:6/572: truncate d0/d8/f33 2141244 0 2024-04-24T22:45:39.192 INFO:tasks.workunit.client.1.smithi192.stdout:7/705: read - d2/db/d25/f7d zero size 2024-04-24T22:45:39.194 INFO:tasks.workunit.client.1.smithi192.stdout:2/771: truncate d4/d9/d10/d6c/f85 2685867 0 2024-04-24T22:45:39.195 INFO:tasks.workunit.client.1.smithi192.stdout:3/684: chown d2/d27/dc6/f2e 354321 1 2024-04-24T22:45:39.196 INFO:tasks.workunit.client.1.smithi192.stdout:2/772: write d4/d9/d9f/fd7 [531698,115797] 0 2024-04-24T22:45:39.198 INFO:tasks.workunit.client.0.smithi063.stdout:0/751: rename d6/d64/da0/f5f to d6/d64/d21/d25/d31/da3/f100 0 2024-04-24T22:45:39.207 INFO:tasks.workunit.client.0.smithi063.stdout:6/863: symlink db/d1d/d20/d11e/l11f 0 2024-04-24T22:45:39.208 INFO:tasks.workunit.client.1.smithi192.stdout:8/629: mknod db/d43/d76/cc3 0 2024-04-24T22:45:39.216 INFO:tasks.workunit.client.0.smithi063.stdout:9/820: symlink d6/d2f/dcd/db/d93/l111 0 2024-04-24T22:45:39.217 INFO:tasks.workunit.client.1.smithi192.stdout:3/685: rename d2/l4c to d2/d4/d12/d1a/d1d/d55/d5f/ld8 0 2024-04-24T22:45:39.218 INFO:tasks.workunit.client.0.smithi063.stdout:1/863: dread d3/de0/d79/ff0 [0,4194304] 0 2024-04-24T22:45:39.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/686: write d2/d27/dc6/d4f/f52 [1409846,25474] 0 2024-04-24T22:45:39.220 INFO:tasks.workunit.client.0.smithi063.stdout:5/775: rename d5/d24/lb9 to d5/d1d/d7d/d67/dbe/l108 0 2024-04-24T22:45:39.222 INFO:tasks.workunit.client.0.smithi063.stdout:6/864: dread db/d1d/d20/d3e/f4f [0,4194304] 0 2024-04-24T22:45:39.223 INFO:tasks.workunit.client.0.smithi063.stdout:0/752: mknod d6/d64/d24/dd1/c101 0 2024-04-24T22:45:39.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/660: dwrite d3/d27/d28/dcc/d5c/f46 [0,4194304] 0 2024-04-24T22:45:39.229 INFO:tasks.workunit.client.0.smithi063.stdout:7/752: mkdir d0/df9 0 2024-04-24T22:45:39.230 INFO:tasks.workunit.client.1.smithi192.stdout:0/661: stat d3/d27/d28/d2e/f4c 0 2024-04-24T22:45:39.231 INFO:tasks.workunit.client.0.smithi063.stdout:4/794: symlink d2/d19/d82/d96/l104 0 2024-04-24T22:45:39.232 INFO:tasks.workunit.client.1.smithi192.stdout:9/735: dwrite d3/f45 [0,4194304] 0 2024-04-24T22:45:39.233 INFO:tasks.workunit.client.1.smithi192.stdout:0/662: chown d3/d27/d28/dcc/d5c/d80/cb0 4162 1 2024-04-24T22:45:39.234 INFO:tasks.workunit.client.0.smithi063.stdout:2/906: dread d0/d2/d27/f37 [0,4194304] 0 2024-04-24T22:45:39.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/630: truncate db/d43/d7c/d85/fa7 185966 0 2024-04-24T22:45:39.235 INFO:tasks.workunit.client.1.smithi192.stdout:0/663: write d3/f39 [1641346,7224] 0 2024-04-24T22:45:39.236 INFO:tasks.workunit.client.0.smithi063.stdout:2/907: write d0/d2/d2f/dba/f104 [861044,120829] 0 2024-04-24T22:45:39.240 INFO:tasks.workunit.client.1.smithi192.stdout:1/556: write d9/f4f [235361,50519] 0 2024-04-24T22:45:39.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/773: dwrite d4/d9/d37/f64 [4194304,4194304] 0 2024-04-24T22:45:39.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/855: creat d1/f111 x:0 0 0 2024-04-24T22:45:39.246 INFO:tasks.workunit.client.0.smithi063.stdout:4/795: creat d2/d8/d40/f105 x:0 0 0 2024-04-24T22:45:39.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/534: getdents d0/d13/d19/d64/d5f/d66 0 2024-04-24T22:45:39.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/908: symlink d0/d2/d2d/d31/d89/d7/d86/da0/l11c 0 2024-04-24T22:45:39.253 INFO:tasks.workunit.client.0.smithi063.stdout:5/776: dwrite d5/d24/d49/f97 [0,4194304] 0 2024-04-24T22:45:39.253 INFO:tasks.workunit.client.0.smithi063.stdout:2/909: chown d0/d2/d2d/d31/d89/d7/d86/l8d 6693985 1 2024-04-24T22:45:39.254 INFO:tasks.workunit.client.1.smithi192.stdout:5/535: dread d0/d13/d19/d64/d5f/d2c/d3c/d91/fb3 [0,4194304] 0 2024-04-24T22:45:39.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/631: creat db/d75/fc4 x:0 0 0 2024-04-24T22:45:39.255 INFO:tasks.workunit.client.0.smithi063.stdout:0/753: mknod d6/d64/d21/d80/dea/c102 0 2024-04-24T22:45:39.256 INFO:tasks.workunit.client.0.smithi063.stdout:7/753: getdents d0/d4/de/d17/d35/d69/da0/dda 0 2024-04-24T22:45:39.257 INFO:tasks.workunit.client.0.smithi063.stdout:8/856: unlink d1/d11/cca 0 2024-04-24T22:45:39.259 INFO:tasks.workunit.client.0.smithi063.stdout:3/817: rename da/d39/f41 to da/d39/f119 0 2024-04-24T22:45:39.260 INFO:tasks.workunit.client.1.smithi192.stdout:7/706: rmdir d2/db/d21/d42/d67/d9b/d6e/d77/dd2 0 2024-04-24T22:45:39.263 INFO:tasks.workunit.client.0.smithi063.stdout:1/864: creat d3/f116 x:0 0 0 2024-04-24T22:45:39.268 INFO:tasks.workunit.client.0.smithi063.stdout:6/865: creat db/d1c/f120 x:0 0 0 2024-04-24T22:45:39.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/821: dread d6/d2f/dcd/d1f/d20/d22/f6d [4194304,4194304] 0 2024-04-24T22:45:39.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/857: dwrite d1/d11/f47 [0,4194304] 0 2024-04-24T22:45:39.270 INFO:tasks.workunit.client.0.smithi063.stdout:5/777: dread d5/f1c [0,4194304] 0 2024-04-24T22:45:39.271 INFO:tasks.workunit.client.1.smithi192.stdout:5/536: creat d0/d13/d19/da9/fba x:0 0 0 2024-04-24T22:45:39.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/537: write d0/f73 [2895775,126956] 0 2024-04-24T22:45:39.274 INFO:tasks.workunit.client.0.smithi063.stdout:0/754: rmdir d6/d74 39 2024-04-24T22:45:39.276 INFO:tasks.workunit.client.1.smithi192.stdout:0/664: creat d3/d27/d28/d2e/dc7/dd4/fdf x:0 0 0 2024-04-24T22:45:39.282 INFO:tasks.workunit.client.1.smithi192.stdout:8/632: mknod db/d5d/d74/d96/cc5 0 2024-04-24T22:45:39.282 INFO:tasks.workunit.client.1.smithi192.stdout:1/557: dread d9/d15/d2d/d31/d2f/f85 [0,4194304] 0 2024-04-24T22:45:39.283 INFO:tasks.workunit.client.0.smithi063.stdout:4/796: truncate d2/d19/d33/dc1/d50/f5d 615148 0 2024-04-24T22:45:39.287 INFO:tasks.workunit.client.0.smithi063.stdout:8/858: fsync d1/f111 0 2024-04-24T22:45:39.287 INFO:tasks.workunit.client.1.smithi192.stdout:6/573: getdents d0/d8/d29/d58 0 2024-04-24T22:45:39.287 INFO:tasks.workunit.client.1.smithi192.stdout:6/574: fdatasync d0/d8/d29/d4c/fc5 0 2024-04-24T22:45:39.288 INFO:tasks.workunit.client.0.smithi063.stdout:8/859: write d1/d2/d3/f82 [4647006,87868] 0 2024-04-24T22:45:39.289 INFO:tasks.workunit.client.0.smithi063.stdout:6/866: unlink db/d1d/d20/d3e/f74 0 2024-04-24T22:45:39.289 INFO:tasks.workunit.client.0.smithi063.stdout:2/910: dread d0/d2/d2d/f81 [0,4194304] 0 2024-04-24T22:45:39.291 INFO:tasks.workunit.client.1.smithi192.stdout:3/687: creat d2/d4/d12/d1a/d1d/fd9 x:0 0 0 2024-04-24T22:45:39.294 INFO:tasks.workunit.client.1.smithi192.stdout:9/736: dwrite d3/dd/de/d17/f68 [0,4194304] 0 2024-04-24T22:45:39.294 INFO:tasks.workunit.client.0.smithi063.stdout:2/911: dwrite d0/d2/f6a [0,4194304] 0 2024-04-24T22:45:39.301 INFO:tasks.workunit.client.0.smithi063.stdout:9/822: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/db9/d112 0 2024-04-24T22:45:39.303 INFO:tasks.workunit.client.1.smithi192.stdout:8/633: creat db/d43/d76/d92/fc6 x:0 0 0 2024-04-24T22:45:39.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:38 smithi063 ceph-mon[26912]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 109 MiB/s rd, 285 MiB/s wr, 556 op/s 2024-04-24T22:45:39.304 INFO:tasks.workunit.client.1.smithi192.stdout:5/538: write d0/d13/d19/d64/d5f/d2c/d3c/d91/fb3 [79493,119902] 0 2024-04-24T22:45:39.306 INFO:tasks.workunit.client.1.smithi192.stdout:5/539: stat d0/d13/d19/d64/d4f/d7c 0 2024-04-24T22:45:39.309 INFO:tasks.workunit.client.0.smithi063.stdout:7/754: dwrite d0/d4/de/d6f/d2e/f85 [0,4194304] 0 2024-04-24T22:45:39.316 INFO:tasks.workunit.client.1.smithi192.stdout:3/688: truncate d2/d27/dc6/f2e 1968589 0 2024-04-24T22:45:39.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/737: write d3/f8d [1161756,68552] 0 2024-04-24T22:45:39.330 INFO:tasks.workunit.client.1.smithi192.stdout:9/738: stat d3/dd/f3a 0 2024-04-24T22:45:39.332 INFO:tasks.workunit.client.1.smithi192.stdout:6/575: mkdir d0/d8/d3d/d9d/da5/dc8 0 2024-04-24T22:45:39.336 INFO:tasks.workunit.client.1.smithi192.stdout:7/707: creat d2/fe0 x:0 0 0 2024-04-24T22:45:39.338 INFO:tasks.workunit.client.1.smithi192.stdout:1/558: truncate d9/d15/f2e 671342 0 2024-04-24T22:45:39.338 INFO:tasks.workunit.client.1.smithi192.stdout:1/559: fdatasync d9/f42 0 2024-04-24T22:45:39.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/560: fsync d9/d15/d2d/d4a/f8b 0 2024-04-24T22:45:39.339 INFO:tasks.workunit.client.1.smithi192.stdout:3/689: symlink d2/d4/d12/d1a/d1d/d55/d5f/d9b/lda 0 2024-04-24T22:45:39.340 INFO:tasks.workunit.client.1.smithi192.stdout:1/561: chown d9/d15/d2d/d48/d93/db4 1 1 2024-04-24T22:45:39.344 INFO:tasks.workunit.client.1.smithi192.stdout:5/540: link d0/d13/d19/d64/d5f/d2c/d3c/d91/fb3 d0/d13/d19/d64/d4f/d52/fbb 0 2024-04-24T22:45:39.345 INFO:tasks.workunit.client.1.smithi192.stdout:9/739: symlink d3/d59/d7d/d9b/dd1/l102 0 2024-04-24T22:45:39.346 INFO:tasks.workunit.client.1.smithi192.stdout:6/576: fsync d0/d8/f31 0 2024-04-24T22:45:39.351 INFO:tasks.workunit.client.0.smithi063.stdout:7/755: dread d0/d4/de/d17/d35/d6c/d8e/fd9 [0,4194304] 0 2024-04-24T22:45:39.356 INFO:tasks.workunit.client.1.smithi192.stdout:0/665: rename d3/d27/d3c/d6c/db3/f86 to d3/d27/d28/dcc/d5c/d80/fe0 0 2024-04-24T22:45:39.359 INFO:tasks.workunit.client.1.smithi192.stdout:8/634: link f9 db/dc/d22/d31/d33/db7/fc7 0 2024-04-24T22:45:39.365 INFO:tasks.workunit.client.1.smithi192.stdout:9/740: creat d3/d5d/da1/d5e/d98/f103 x:0 0 0 2024-04-24T22:45:39.366 INFO:tasks.workunit.client.1.smithi192.stdout:9/741: chown d3/d5d/fd7 39283 1 2024-04-24T22:45:39.372 INFO:tasks.workunit.client.1.smithi192.stdout:3/690: fdatasync d2/d4/d12/f1c 0 2024-04-24T22:45:39.373 INFO:tasks.workunit.client.1.smithi192.stdout:3/691: chown d2/d4/d12/d1a/d1d/d55/cb 196949799 1 2024-04-24T22:45:39.375 INFO:tasks.workunit.client.1.smithi192.stdout:3/692: read d2/d4/d12/f1b [1608638,111249] 0 2024-04-24T22:45:39.376 INFO:tasks.workunit.client.1.smithi192.stdout:3/693: write d2/d27/dc6/d4b/d54/fb3 [779496,76917] 0 2024-04-24T22:45:39.377 INFO:tasks.workunit.client.1.smithi192.stdout:8/635: symlink db/dc/d9a/lc8 0 2024-04-24T22:45:39.382 INFO:tasks.workunit.client.1.smithi192.stdout:9/742: unlink d3/dd/de/da7/dae/fd6 0 2024-04-24T22:45:39.383 INFO:tasks.workunit.client.1.smithi192.stdout:5/541: write d0/d13/d19/d64/d5f/d2c/f44 [675938,129513] 0 2024-04-24T22:45:39.383 INFO:tasks.workunit.client.0.smithi063.stdout:8/860: symlink d1/d2/d17/l112 0 2024-04-24T22:45:39.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/708: link d2/db/d21/l35 d2/db/d21/le1 0 2024-04-24T22:45:39.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/709: chown d2/fcb 5330419 1 2024-04-24T22:45:39.385 INFO:tasks.workunit.client.1.smithi192.stdout:7/710: stat d2/db/d21/d42/d5b/d63 0 2024-04-24T22:45:39.390 INFO:tasks.workunit.client.1.smithi192.stdout:8/636: fdatasync db/dc/d22/d31/d33/d81/d8a/f8b 0 2024-04-24T22:45:39.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/756: unlink d0/d4/de/f61 0 2024-04-24T22:45:39.398 INFO:tasks.workunit.client.0.smithi063.stdout:1/865: creat d3/da/da2/f117 x:0 0 0 2024-04-24T22:45:39.399 INFO:tasks.workunit.client.0.smithi063.stdout:1/866: write d3/de0/d79/fc1 [4466745,102813] 0 2024-04-24T22:45:39.399 INFO:tasks.workunit.client.1.smithi192.stdout:5/542: rename d0/d13/d19/d64/d4f/l58 to d0/d1b/d62/lbc 0 2024-04-24T22:45:39.402 INFO:tasks.workunit.client.0.smithi063.stdout:8/861: creat d1/d2/d17/dd1/f113 x:0 0 0 2024-04-24T22:45:39.403 INFO:tasks.workunit.client.0.smithi063.stdout:8/862: chown d1/d22/d33/dbd/fbe 2 1 2024-04-24T22:45:39.404 INFO:tasks.workunit.client.0.smithi063.stdout:6/867: creat db/d1d/d20/d3f/d94/f121 x:0 0 0 2024-04-24T22:45:39.408 INFO:tasks.workunit.client.1.smithi192.stdout:9/743: write d3/f21 [2551047,120445] 0 2024-04-24T22:45:39.411 INFO:tasks.workunit.client.1.smithi192.stdout:6/577: getdents d0/d8/d29 0 2024-04-24T22:45:39.412 INFO:tasks.workunit.client.0.smithi063.stdout:7/757: creat d0/d4/de/d6f/d5c/d9c/ffa x:0 0 0 2024-04-24T22:45:39.414 INFO:tasks.workunit.client.0.smithi063.stdout:7/758: write d0/d4/de/d6f/d5c/d9c/fd8 [4939750,70817] 0 2024-04-24T22:45:39.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/755: sync 2024-04-24T22:45:39.416 INFO:tasks.workunit.client.0.smithi063.stdout:3/818: sync 2024-04-24T22:45:39.417 INFO:tasks.workunit.client.0.smithi063.stdout:0/756: write d6/d3a/fee [248521,111676] 0 2024-04-24T22:45:39.417 INFO:tasks.workunit.client.1.smithi192.stdout:7/711: symlink d2/db/d21/d42/d5b/le2 0 2024-04-24T22:45:39.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/759: dread d0/f48 [0,4194304] 0 2024-04-24T22:45:39.422 INFO:tasks.workunit.client.0.smithi063.stdout:0/757: truncate d6/d64/d21/d80/fef 770309 0 2024-04-24T22:45:39.422 INFO:tasks.workunit.client.0.smithi063.stdout:7/760: chown d0/d4/de/d17/cb3 50664318 1 2024-04-24T22:45:39.422 INFO:tasks.workunit.client.0.smithi063.stdout:1/867: creat d3/db5/d21/d9f/f118 x:0 0 0 2024-04-24T22:45:39.423 INFO:tasks.workunit.client.1.smithi192.stdout:1/562: link d9/d15/d2d/d48/d93/ca5 d9/dd/cb9 0 2024-04-24T22:45:39.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/863: unlink d1/d21/f5d 0 2024-04-24T22:45:39.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/864: readlink d1/d2/d3/l4 0 2024-04-24T22:45:39.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/865: fsync d1/d11/f63 0 2024-04-24T22:45:39.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/868: symlink db/d1d/d20/d29/dab/dd0/l122 0 2024-04-24T22:45:39.424 INFO:tasks.workunit.client.0.smithi063.stdout:8/866: rename d1/d22 to d1/d22/d33/da6/d54/d7b/d114 22 2024-04-24T22:45:39.426 INFO:tasks.workunit.client.0.smithi063.stdout:5/778: link d5/d11/d23/d2d/d62/la1 d5/l109 0 2024-04-24T22:45:39.428 INFO:tasks.workunit.client.1.smithi192.stdout:8/637: dwrite db/f34 [8388608,4194304] 0 2024-04-24T22:45:39.434 INFO:tasks.workunit.client.1.smithi192.stdout:5/543: dwrite d0/d13/d19/d64/d5f/d2c/f5c [0,4194304] 0 2024-04-24T22:45:39.439 INFO:tasks.workunit.client.1.smithi192.stdout:6/578: write d0/d8/d29/f30 [26624,101475] 0 2024-04-24T22:45:39.440 INFO:tasks.workunit.client.1.smithi192.stdout:6/579: stat d0/d8/d3d/d3e/d7e/d9b 0 2024-04-24T22:45:39.441 INFO:tasks.workunit.client.0.smithi063.stdout:3/819: chown da/d18/d47/d4d/d62/l93 184020240 1 2024-04-24T22:45:39.442 INFO:tasks.workunit.client.0.smithi063.stdout:4/797: getdents d2/d19/d33 0 2024-04-24T22:45:39.444 INFO:tasks.workunit.client.1.smithi192.stdout:9/744: dwrite d3/dd/de/f12 [0,4194304] 0 2024-04-24T22:45:39.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/798: dread - d2/d19/d33/dc1/d50/dc3/fdd zero size 2024-04-24T22:45:39.446 INFO:tasks.workunit.client.0.smithi063.stdout:9/823: dwrite d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fd1 [0,4194304] 0 2024-04-24T22:45:39.448 INFO:tasks.workunit.client.0.smithi063.stdout:8/867: stat d1/d22/d33/da6/d54/d79/lf9 0 2024-04-24T22:45:39.449 INFO:tasks.workunit.client.1.smithi192.stdout:8/638: read db/dc/d29/f89 [2806102,126099] 0 2024-04-24T22:45:39.449 INFO:tasks.workunit.client.0.smithi063.stdout:8/868: write d1/d2/d3/d86/ff2 [724546,25576] 0 2024-04-24T22:45:39.450 INFO:tasks.workunit.client.0.smithi063.stdout:8/869: dread - d1/d2/f10c zero size 2024-04-24T22:45:39.450 INFO:tasks.workunit.client.1.smithi192.stdout:9/745: dread - d3/f58 zero size 2024-04-24T22:45:39.451 INFO:tasks.workunit.client.0.smithi063.stdout:5/779: mkdir d5/d11/d23/d2d/d62/d73/d95/d10a 0 2024-04-24T22:45:39.452 INFO:tasks.workunit.client.1.smithi192.stdout:7/712: dwrite d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 [0,4194304] 0 2024-04-24T22:45:39.454 INFO:tasks.workunit.client.1.smithi192.stdout:7/713: chown d2/db/f60 0 1 2024-04-24T22:45:39.455 INFO:tasks.workunit.client.1.smithi192.stdout:7/714: dread - d2/db/fa6 zero size 2024-04-24T22:45:39.455 INFO:tasks.workunit.client.0.smithi063.stdout:5/780: dread d5/f6f [0,4194304] 0 2024-04-24T22:45:39.456 INFO:tasks.workunit.client.1.smithi192.stdout:5/544: creat d0/d13/d3b/fbd x:0 0 0 2024-04-24T22:45:39.457 INFO:tasks.workunit.client.1.smithi192.stdout:7/715: chown d2/db/d21/d42/d5b/d63 707327833 1 2024-04-24T22:45:39.459 INFO:tasks.workunit.client.1.smithi192.stdout:8/639: rename db/dc/d29/d9f/fc0 to db/d75/fc9 0 2024-04-24T22:45:39.460 INFO:tasks.workunit.client.0.smithi063.stdout:0/758: truncate d6/d64/d21/d25/d62/d63/f6e 3273668 0 2024-04-24T22:45:39.461 INFO:tasks.workunit.client.0.smithi063.stdout:0/759: readlink d6/d64/d21/ldb 0 2024-04-24T22:45:39.461 INFO:tasks.workunit.client.1.smithi192.stdout:7/716: stat d2/db/d21/d42/d67/d9b/d6e/d77 0 2024-04-24T22:45:39.462 INFO:tasks.workunit.client.1.smithi192.stdout:5/545: dread - d0/d1b/d62/fa7 zero size 2024-04-24T22:45:39.464 INFO:tasks.workunit.client.0.smithi063.stdout:3/820: mknod da/d18/d47/d4d/dc2/c11a 0 2024-04-24T22:45:39.464 INFO:tasks.workunit.client.0.smithi063.stdout:3/821: chown da/d39/d94/f110 88 1 2024-04-24T22:45:39.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/868: link d3/db5/d21/d9f/dd5/d106/d11/f111 d3/db5/d21/d9f/dd5/d106/d39/d115/f119 0 2024-04-24T22:45:39.467 INFO:tasks.workunit.client.0.smithi063.stdout:4/799: symlink d2/d19/dce/l106 0 2024-04-24T22:45:39.468 INFO:tasks.workunit.client.1.smithi192.stdout:6/580: link d0/d8/f2a d0/d8/d29/d5a/d63/fc9 0 2024-04-24T22:45:39.468 INFO:tasks.workunit.client.0.smithi063.stdout:9/824: dread - d6/d2f/dcd/d1f/d20/d2a/d4b/fdc zero size 2024-04-24T22:45:39.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/546: mknod d0/d13/d19/d64/d4f/d52/cbe 0 2024-04-24T22:45:39.470 INFO:tasks.workunit.client.1.smithi192.stdout:5/547: fdatasync d0/d13/d3b/f3a 0 2024-04-24T22:45:39.472 INFO:tasks.workunit.client.1.smithi192.stdout:8/640: mkdir db/dc/d22/dca 0 2024-04-24T22:45:39.475 INFO:tasks.workunit.client.0.smithi063.stdout:5/781: creat d5/d24/d49/d3b/f10b x:0 0 0 2024-04-24T22:45:39.475 INFO:tasks.workunit.client.1.smithi192.stdout:5/548: truncate d0/d13/d53/f70 801844 0 2024-04-24T22:45:39.478 INFO:tasks.workunit.client.0.smithi063.stdout:0/760: symlink d6/d64/d24/dd1/l103 0 2024-04-24T22:45:39.480 INFO:tasks.workunit.client.1.smithi192.stdout:9/746: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:39.482 INFO:tasks.workunit.client.1.smithi192.stdout:7/717: rename d2/db/d21/d42/d67/d9b/d6e/fb8 to d2/db/d25/fe3 0 2024-04-24T22:45:39.484 INFO:tasks.workunit.client.1.smithi192.stdout:5/549: mkdir d0/d13/d19/d71/dbf 0 2024-04-24T22:45:39.485 INFO:tasks.workunit.client.0.smithi063.stdout:1/869: mknod d3/de0/d79/d96/df8/c11a 0 2024-04-24T22:45:39.488 INFO:tasks.workunit.client.0.smithi063.stdout:4/800: mknod d2/d8/d5c/c107 0 2024-04-24T22:45:39.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/825: write d6/d2f/dcd/db/d6b/d4f/dad/d36/fe5 [3730657,13010] 0 2024-04-24T22:45:39.491 INFO:tasks.workunit.client.0.smithi063.stdout:4/801: dread d2/f2e [0,4194304] 0 2024-04-24T22:45:39.497 INFO:tasks.workunit.client.1.smithi192.stdout:6/581: truncate d0/d8/f2a 869489 0 2024-04-24T22:45:39.500 INFO:tasks.workunit.client.1.smithi192.stdout:9/747: mknod d3/dd/de/d17/d89/d8b/ded/c104 0 2024-04-24T22:45:39.501 INFO:tasks.workunit.client.1.smithi192.stdout:9/748: truncate d3/d59/dbf/fe9 387798 0 2024-04-24T22:45:39.502 INFO:tasks.workunit.client.1.smithi192.stdout:6/582: dread d0/d8/f76 [0,4194304] 0 2024-04-24T22:45:39.506 INFO:tasks.workunit.client.0.smithi063.stdout:6/869: getdents db/d1d/d8d 0 2024-04-24T22:45:39.507 INFO:tasks.workunit.client.1.smithi192.stdout:7/718: creat d2/db/d21/d42/d67/d9b/d6e/fe4 x:0 0 0 2024-04-24T22:45:39.510 INFO:tasks.workunit.client.1.smithi192.stdout:5/550: dread d0/fc [0,4194304] 0 2024-04-24T22:45:39.515 INFO:tasks.workunit.client.1.smithi192.stdout:9/749: creat d3/d59/d7d/d9b/dd1/f105 x:0 0 0 2024-04-24T22:45:39.516 INFO:tasks.workunit.client.1.smithi192.stdout:9/750: write d3/f45 [4612438,67299] 0 2024-04-24T22:45:39.518 INFO:tasks.workunit.client.0.smithi063.stdout:8/870: write d1/d11/f47 [443381,79815] 0 2024-04-24T22:45:39.519 INFO:tasks.workunit.client.1.smithi192.stdout:7/719: readlink d2/db/d21/l35 0 2024-04-24T22:45:39.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/720: write d2/db/d21/fc6 [382592,123835] 0 2024-04-24T22:45:39.523 INFO:tasks.workunit.client.0.smithi063.stdout:2/912: sync 2024-04-24T22:45:39.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/761: mknod d6/d64/d21/d25/d62/d63/c104 0 2024-04-24T22:45:39.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/751: dwrite d3/d24/f97 [0,4194304] 0 2024-04-24T22:45:39.525 INFO:tasks.workunit.client.0.smithi063.stdout:0/762: chown d6/da/d2b/fa7 50158735 1 2024-04-24T22:45:39.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/763: stat d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:39.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/870: dwrite d3/de0/f43 [4194304,4194304] 0 2024-04-24T22:45:39.527 INFO:tasks.workunit.client.1.smithi192.stdout:5/551: truncate d0/d13/d53/f94 1012373 0 2024-04-24T22:45:39.528 INFO:tasks.workunit.client.0.smithi063.stdout:1/871: chown d3/db5/d21/d9f/dd5/d106/df5/f103 0 1 2024-04-24T22:45:39.528 INFO:tasks.workunit.client.1.smithi192.stdout:5/552: write d0/d1b/d50/fb1 [146087,88019] 0 2024-04-24T22:45:39.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/553: readlink d0/d13/d3b/l5d 0 2024-04-24T22:45:39.532 INFO:tasks.workunit.client.0.smithi063.stdout:6/870: symlink db/d85/dd2/dd7/l123 0 2024-04-24T22:45:39.533 INFO:tasks.workunit.client.1.smithi192.stdout:7/721: fsync d2/db/d25/fe3 0 2024-04-24T22:45:39.533 INFO:tasks.workunit.client.1.smithi192.stdout:6/583: dwrite d0/d8/f20 [0,4194304] 0 2024-04-24T22:45:39.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/782: dwrite d5/d11/f2f [0,4194304] 0 2024-04-24T22:45:39.534 INFO:tasks.workunit.client.1.smithi192.stdout:6/584: chown d0/l1a 31 1 2024-04-24T22:45:39.535 INFO:tasks.workunit.client.0.smithi063.stdout:9/826: creat d6/d2f/dcd/db/d1c/d78/d9a/ddd/f113 x:0 0 0 2024-04-24T22:45:39.535 INFO:tasks.workunit.client.0.smithi063.stdout:5/783: read - d5/d24/db6/ff6 zero size 2024-04-24T22:45:39.535 INFO:tasks.workunit.client.1.smithi192.stdout:6/585: chown d0/d8/f17 754298 1 2024-04-24T22:45:39.536 INFO:tasks.workunit.client.1.smithi192.stdout:9/752: rmdir d3/d59/d7d/d88 39 2024-04-24T22:45:39.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/784: truncate d5/d24/d44/fe4 801156 0 2024-04-24T22:45:39.539 INFO:tasks.workunit.client.0.smithi063.stdout:3/822: truncate da/d18/d3a/f69 1092744 0 2024-04-24T22:45:39.545 INFO:tasks.workunit.client.1.smithi192.stdout:9/753: dwrite d3/dd/de/d17/f68 [0,4194304] 0 2024-04-24T22:45:39.549 INFO:tasks.workunit.client.0.smithi063.stdout:1/872: fdatasync d3/db5/f71 0 2024-04-24T22:45:39.551 INFO:tasks.workunit.client.1.smithi192.stdout:5/554: mknod d0/d13/cc0 0 2024-04-24T22:45:39.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/761: sync 2024-04-24T22:45:39.552 INFO:tasks.workunit.client.0.smithi063.stdout:6/871: rmdir db/d1c 39 2024-04-24T22:45:39.553 INFO:tasks.workunit.client.1.smithi192.stdout:7/722: symlink d2/db/d21/d42/d67/le5 0 2024-04-24T22:45:39.554 INFO:tasks.workunit.client.0.smithi063.stdout:7/762: write d0/d4/de/d6f/fd6 [969396,87479] 0 2024-04-24T22:45:39.554 INFO:tasks.workunit.client.1.smithi192.stdout:7/723: chown d2/db/d36/c46 31 1 2024-04-24T22:45:39.555 INFO:tasks.workunit.client.1.smithi192.stdout:9/754: mkdir d3/d59/dbf/d106 0 2024-04-24T22:45:39.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/755: chown d3/d5d/c9d 1804075979 1 2024-04-24T22:45:39.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/827: rename d6/d2f/dcd/db/d93/da0/f106 to d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/db9/f114 0 2024-04-24T22:45:39.560 INFO:tasks.workunit.client.0.smithi063.stdout:3/823: creat da/d18/d9e/da3/dc1/f11b x:0 0 0 2024-04-24T22:45:39.561 INFO:tasks.workunit.client.1.smithi192.stdout:7/724: dread d2/db/d21/d42/d67/d9b/dc1/fc3 [0,4194304] 0 2024-04-24T22:45:39.562 INFO:tasks.workunit.client.1.smithi192.stdout:7/725: dread - d2/db/d21/d42/d67/d9b/d6e/db5/fdf zero size 2024-04-24T22:45:39.563 INFO:tasks.workunit.client.0.smithi063.stdout:4/802: link d2/d8/d3c/c41 d2/d80/d81/d90/c108 0 2024-04-24T22:45:39.564 INFO:tasks.workunit.client.0.smithi063.stdout:4/803: chown d2/d19/d33/dc1/d50/dc3/fdd 1955 1 2024-04-24T22:45:39.565 INFO:tasks.workunit.client.1.smithi192.stdout:7/726: dread d2/f91 [4194304,4194304] 0 2024-04-24T22:45:39.570 INFO:tasks.workunit.client.1.smithi192.stdout:9/756: creat d3/dd/de/d17/d89/db4/f107 x:0 0 0 2024-04-24T22:45:39.570 INFO:tasks.workunit.client.1.smithi192.stdout:9/757: fsync d3/d59/ff5 0 2024-04-24T22:45:39.572 INFO:tasks.workunit.client.0.smithi063.stdout:2/913: write d0/d2/d2d/d31/f5b [3141556,81619] 0 2024-04-24T22:45:39.572 INFO:tasks.workunit.client.0.smithi063.stdout:2/914: chown d0/d2/d39/d11a 1572 1 2024-04-24T22:45:39.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/758: readlink d3/d59/d7d/d9b/dd1/l102 0 2024-04-24T22:45:39.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/759: chown d3/d59/ff5 1 1 2024-04-24T22:45:39.575 INFO:tasks.workunit.client.1.smithi192.stdout:6/586: getdents d0/d8/d29/d5a 0 2024-04-24T22:45:39.577 INFO:tasks.workunit.client.0.smithi063.stdout:1/873: fdatasync d3/db5/f7f 0 2024-04-24T22:45:39.579 INFO:tasks.workunit.client.1.smithi192.stdout:7/727: dread d2/db/d21/d42/f93 [0,4194304] 0 2024-04-24T22:45:39.580 INFO:tasks.workunit.client.0.smithi063.stdout:6/872: creat db/d1d/d20/d29/dab/f124 x:0 0 0 2024-04-24T22:45:39.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/785: dread d5/d24/d36/f6c [0,4194304] 0 2024-04-24T22:45:39.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/728: dread - d2/db/d21/d42/d67/d9b/d6e/da2/fd1 zero size 2024-04-24T22:45:39.584 INFO:tasks.workunit.client.0.smithi063.stdout:1/874: dwrite d3/f23 [0,4194304] 0 2024-04-24T22:45:39.585 INFO:tasks.workunit.client.1.smithi192.stdout:5/555: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-24T22:45:39.587 INFO:tasks.workunit.client.0.smithi063.stdout:7/763: read d0/d4/de/d17/d35/f79 [2178177,69482] 0 2024-04-24T22:45:39.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/764: chown d0/d4/de/d6f/d5c/c87 6871 1 2024-04-24T22:45:39.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/627: sync 2024-04-24T22:45:39.622 INFO:tasks.workunit.client.1.smithi192.stdout:4/628: readlink d2/d9/l2a 0 2024-04-24T22:45:39.700 INFO:tasks.workunit.client.0.smithi063.stdout:0/764: write d6/d64/d21/f48 [1477385,52993] 0 2024-04-24T22:45:39.704 INFO:tasks.workunit.client.0.smithi063.stdout:9/828: unlink d6/d2f/dcd/db/d6b/d4f/dad/d36/f40 0 2024-04-24T22:45:39.704 INFO:tasks.workunit.client.0.smithi063.stdout:9/829: dread - d6/d2f/dcd/db/d6b/d4f/dad/d48/fe9 zero size 2024-04-24T22:45:39.707 INFO:tasks.workunit.client.1.smithi192.stdout:9/760: truncate d3/dd/de/f51 640387 0 2024-04-24T22:45:39.710 INFO:tasks.workunit.client.1.smithi192.stdout:7/729: readlink d2/db/ldc 0 2024-04-24T22:45:39.716 INFO:tasks.workunit.client.1.smithi192.stdout:9/761: dwrite d3/dd/d53/f5c [0,4194304] 0 2024-04-24T22:45:39.726 INFO:tasks.workunit.client.0.smithi063.stdout:2/915: dread d0/d2/d2d/d31/d89/d7/f5c [0,4194304] 0 2024-04-24T22:45:39.727 INFO:tasks.workunit.client.1.smithi192.stdout:7/730: truncate d2/db/d21/d42/d67/d9b/f48 1744067 0 2024-04-24T22:45:39.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/629: mknod d2/db/d75/da1/d36/d5b/cd4 0 2024-04-24T22:45:39.734 INFO:tasks.workunit.client.1.smithi192.stdout:7/731: dwrite d2/db/d2a/d57/fd9 [0,4194304] 0 2024-04-24T22:45:39.737 INFO:tasks.workunit.client.1.smithi192.stdout:9/762: write d3/d59/d7d/d88/dc2/fd8 [389520,45364] 0 2024-04-24T22:45:39.738 INFO:tasks.workunit.client.0.smithi063.stdout:5/786: dread - d5/d11/d23/d2d/d62/d73/d95/fcb zero size 2024-04-24T22:45:39.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/763: fsync d3/d59/ff5 0 2024-04-24T22:45:39.740 INFO:tasks.workunit.client.1.smithi192.stdout:9/764: dread - d3/dd/de/d17/d89/db4/f107 zero size 2024-04-24T22:45:39.743 INFO:tasks.workunit.client.1.smithi192.stdout:4/630: mknod d2/d5c/cd5 0 2024-04-24T22:45:39.743 INFO:tasks.workunit.client.1.smithi192.stdout:7/732: dwrite d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 [0,4194304] 0 2024-04-24T22:45:39.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/830: mknod d6/dbd/c115 0 2024-04-24T22:45:39.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/831: chown d6/d2f/dcd/db/d6b 33 1 2024-04-24T22:45:39.753 INFO:tasks.workunit.client.0.smithi063.stdout:8/871: getdents d1/d11/d32/d38/db1 0 2024-04-24T22:45:39.763 INFO:tasks.workunit.client.0.smithi063.stdout:2/916: symlink d0/d2/d2d/d31/d89/d7/d86/da0/deb/d10e/l11d 0 2024-04-24T22:45:39.764 INFO:tasks.workunit.client.0.smithi063.stdout:2/917: stat d0/d2/d35/c79 0 2024-04-24T22:45:39.768 INFO:tasks.workunit.client.0.smithi063.stdout:4/804: mkdir d2/d19/d33/dc1/d109 0 2024-04-24T22:45:39.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/631: creat d2/fd6 x:0 0 0 2024-04-24T22:45:39.771 INFO:tasks.workunit.client.1.smithi192.stdout:9/765: getdents d3/d59/d7d/d88 0 2024-04-24T22:45:39.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/766: chown d3/dd/de/dba/dbc/ddb 7580483 1 2024-04-24T22:45:39.772 INFO:tasks.workunit.client.1.smithi192.stdout:7/733: dread d2/db/d21/d42/d67/d9b/fcd [0,4194304] 0 2024-04-24T22:45:39.774 INFO:tasks.workunit.client.0.smithi063.stdout:3/824: creat da/d18/d9e/f11c x:0 0 0 2024-04-24T22:45:39.775 INFO:tasks.workunit.client.1.smithi192.stdout:7/734: write d2/db/d21/d42/d67/da8/dc7/fbd [3870428,106768] 0 2024-04-24T22:45:39.776 INFO:tasks.workunit.client.1.smithi192.stdout:4/632: rename d2/db/d75/da1/d36/d5b/l30 to d2/d5c/dbb/ld7 0 2024-04-24T22:45:39.776 INFO:tasks.workunit.client.1.smithi192.stdout:7/735: chown d2/db/f15 404 1 2024-04-24T22:45:39.777 INFO:tasks.workunit.client.1.smithi192.stdout:9/767: dread d3/dd/de/da7/fd3 [0,4194304] 0 2024-04-24T22:45:39.779 INFO:tasks.workunit.client.0.smithi063.stdout:2/918: write d0/d2/d39/d11a/de2/de4/f112 [795314,42294] 0 2024-04-24T22:45:39.781 INFO:tasks.workunit.client.1.smithi192.stdout:4/633: creat d2/d5c/dce/fd8 x:0 0 0 2024-04-24T22:45:39.782 INFO:tasks.workunit.client.0.smithi063.stdout:5/787: fdatasync d5/d24/d44/fa5 0 2024-04-24T22:45:39.783 INFO:tasks.workunit.client.1.smithi192.stdout:6/587: creat d0/d8/d18/fca x:0 0 0 2024-04-24T22:45:39.784 INFO:tasks.workunit.client.1.smithi192.stdout:9/768: rmdir d3/dd/d53/d60 39 2024-04-24T22:45:39.786 INFO:tasks.workunit.client.0.smithi063.stdout:4/805: fsync d2/d19/fa6 0 2024-04-24T22:45:39.787 INFO:tasks.workunit.client.0.smithi063.stdout:4/806: fsync d2/d8/d40/d51/f66 0 2024-04-24T22:45:39.787 INFO:tasks.workunit.client.1.smithi192.stdout:4/634: readlink d2/l26 0 2024-04-24T22:45:39.788 INFO:tasks.workunit.client.0.smithi063.stdout:7/765: creat d0/d4/de/ffb x:0 0 0 2024-04-24T22:45:39.790 INFO:tasks.workunit.client.1.smithi192.stdout:9/769: fsync d3/dd/de/da7/fcb 0 2024-04-24T22:45:39.793 INFO:tasks.workunit.client.1.smithi192.stdout:4/635: symlink d2/db/d75/da1/d36/d70/ld9 0 2024-04-24T22:45:39.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/919: truncate d0/d2/d2d/d31/d89/d1c/d3f/f76 1526813 0 2024-04-24T22:45:39.796 INFO:tasks.workunit.client.0.smithi063.stdout:2/920: read d0/fa6 [1154467,120523] 0 2024-04-24T22:45:39.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/770: dwrite d3/d59/d7d/d88/dc2/fd8 [0,4194304] 0 2024-04-24T22:45:39.801 INFO:tasks.workunit.client.0.smithi063.stdout:7/766: dread d0/d2f/f37 [0,4194304] 0 2024-04-24T22:45:39.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/765: sync 2024-04-24T22:45:39.824 INFO:tasks.workunit.client.0.smithi063.stdout:0/766: dread d6/d64/d24/d42/fbe [0,4194304] 0 2024-04-24T22:45:39.826 INFO:tasks.workunit.client.1.smithi192.stdout:2/774: sync 2024-04-24T22:45:39.830 INFO:tasks.workunit.client.1.smithi192.stdout:2/775: fdatasync d4/d9/d10/d6c/d6f/da8/fac 0 2024-04-24T22:45:39.837 INFO:tasks.workunit.client.0.smithi063.stdout:3/825: mknod da/d39/c11d 0 2024-04-24T22:45:39.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/776: creat d4/d9/d10/d6c/d6f/da8/ffc x:0 0 0 2024-04-24T22:45:39.839 INFO:tasks.workunit.client.1.smithi192.stdout:2/777: chown d4/d9/d37/de5 261275 1 2024-04-24T22:45:39.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/588: getdents d0 0 2024-04-24T22:45:39.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/589: readlink d0/d8/d29/d4c/d4d/lbc 0 2024-04-24T22:45:39.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/590: rename d0/d8/d3d/d3e/d7e/d9b to d0/d8/d22/dcb 0 2024-04-24T22:45:39.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/767: mkdir d6/d64/d21/d80/dde/d105 0 2024-04-24T22:45:39.861 INFO:tasks.workunit.client.0.smithi063.stdout:3/826: readlink da/d18/d47/d4d/d62/l90 0 2024-04-24T22:45:39.862 INFO:tasks.workunit.client.0.smithi063.stdout:8/872: getdents d1/d11/d32/d6a 0 2024-04-24T22:45:39.865 INFO:tasks.workunit.client.0.smithi063.stdout:7/767: truncate d0/d4/de/d17/dd7/f84 293388 0 2024-04-24T22:45:39.868 INFO:tasks.workunit.client.0.smithi063.stdout:4/807: mkdir d2/d9f/df8/d10a 0 2024-04-24T22:45:39.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/873: symlink d1/d11/d32/d6a/l115 0 2024-04-24T22:45:39.870 INFO:tasks.workunit.client.0.smithi063.stdout:4/808: dread - d2/d8/d40/fe6 zero size 2024-04-24T22:45:39.871 INFO:tasks.workunit.client.0.smithi063.stdout:7/768: dwrite d0/d4/de/d6f/d2e/d76/ff3 [0,4194304] 0 2024-04-24T22:45:39.879 INFO:tasks.workunit.client.0.smithi063.stdout:8/874: rename d1/d22/d33/dbd/f10a to d1/d11/d32/d38/db7/f116 0 2024-04-24T22:45:39.879 INFO:tasks.workunit.client.0.smithi063.stdout:8/875: chown d1/d22/d33/da6/d54/fad 626 1 2024-04-24T22:45:39.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/769: unlink d0/d4/de/d6f/d2e/l65 0 2024-04-24T22:45:39.883 INFO:tasks.workunit.client.0.smithi063.stdout:0/768: dread d6/d64/d24/f6b [0,4194304] 0 2024-04-24T22:45:39.890 INFO:tasks.workunit.client.0.smithi063.stdout:7/770: rename d0/d4/de/d17/d28/d31/dad to d0/d4/de/dfc 0 2024-04-24T22:45:39.894 INFO:tasks.workunit.client.0.smithi063.stdout:7/771: mkdir d0/df9/dfd 0 2024-04-24T22:45:39.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/772: mkdir d0/d4/de/dfc/dfe 0 2024-04-24T22:45:39.898 INFO:tasks.workunit.client.0.smithi063.stdout:7/773: symlink d0/d4/de/dfc/de7/lff 0 2024-04-24T22:45:39.900 INFO:tasks.workunit.client.0.smithi063.stdout:7/774: write d0/d4/de/d17/d35/d69/fbc [4754724,51453] 0 2024-04-24T22:45:39.925 INFO:tasks.workunit.client.1.smithi192.stdout:5/556: dwrite d0/d13/d19/d64/d5f/f72 [0,4194304] 0 2024-04-24T22:45:39.927 INFO:tasks.workunit.client.1.smithi192.stdout:5/557: dread - d0/d13/d19/d83/f87 zero size 2024-04-24T22:45:39.930 INFO:tasks.workunit.client.0.smithi063.stdout:6/873: write db/d1d/d20/d3e/fe1 [837486,105591] 0 2024-04-24T22:45:39.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/558: dwrite d0/d13/d19/d64/d4f/d6c/f9c [0,4194304] 0 2024-04-24T22:45:39.935 INFO:tasks.workunit.client.0.smithi063.stdout:6/874: creat db/d1d/d8d/dfc/d11c/f125 x:0 0 0 2024-04-24T22:45:39.936 INFO:tasks.workunit.client.1.smithi192.stdout:5/559: rmdir d0/d13/d19/d71/daf 39 2024-04-24T22:45:39.937 INFO:tasks.workunit.client.1.smithi192.stdout:5/560: mkdir d0/d1b/d62/dc1 0 2024-04-24T22:45:39.938 INFO:tasks.workunit.client.0.smithi063.stdout:3/827: sync 2024-04-24T22:45:39.943 INFO:tasks.workunit.client.1.smithi192.stdout:5/561: dread d0/f8b [0,4194304] 0 2024-04-24T22:45:39.943 INFO:tasks.workunit.client.0.smithi063.stdout:3/828: creat da/d39/d94/db3/df3/f11e x:0 0 0 2024-04-24T22:45:39.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/875: creat d3/da/f11b x:0 0 0 2024-04-24T22:45:39.944 INFO:tasks.workunit.client.0.smithi063.stdout:3/829: dread - da/d18/d9e/fed zero size 2024-04-24T22:45:39.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/875: link db/d1d/d20/d3f/d5f/d93/la8 db/d1d/d8d/dfc/d11c/l126 0 2024-04-24T22:45:39.946 INFO:tasks.workunit.client.0.smithi063.stdout:6/876: fsync db/d1d/d20/d29/fcc 0 2024-04-24T22:45:39.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/562: dread d0/d13/d19/d64/d4f/d52/f67 [0,4194304] 0 2024-04-24T22:45:39.948 INFO:tasks.workunit.client.1.smithi192.stdout:5/563: chown d0/d1b/d62/fb4 51235 1 2024-04-24T22:45:39.950 INFO:tasks.workunit.client.0.smithi063.stdout:1/876: unlink d3/da/d34/lca 0 2024-04-24T22:45:39.951 INFO:tasks.workunit.client.0.smithi063.stdout:3/830: symlink da/d39/d94/de4/l11f 0 2024-04-24T22:45:39.952 INFO:tasks.workunit.client.0.smithi063.stdout:6/877: read db/d1d/d2d/d69/dd4/de4/fd5 [93892,75859] 0 2024-04-24T22:45:39.954 INFO:tasks.workunit.client.0.smithi063.stdout:3/831: mkdir da/d18/d47/d4d/dc2/d120 0 2024-04-24T22:45:39.956 INFO:tasks.workunit.client.0.smithi063.stdout:3/832: stat da/d39/f98 0 2024-04-24T22:45:39.961 INFO:tasks.workunit.client.0.smithi063.stdout:6/878: dwrite db/d1c/dc2/feb [0,4194304] 0 2024-04-24T22:45:39.964 INFO:tasks.workunit.client.1.smithi192.stdout:3/694: sync 2024-04-24T22:45:39.964 INFO:tasks.workunit.client.1.smithi192.stdout:0/666: sync 2024-04-24T22:45:39.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/832: dwrite d6/d2f/dcd/d1f/d20/d2a/d4b/fdc [0,4194304] 0 2024-04-24T22:45:39.966 INFO:tasks.workunit.client.0.smithi063.stdout:9/833: readlink d6/d2f/dcd/db/d1c/d78/lf1 0 2024-04-24T22:45:39.967 INFO:tasks.workunit.client.1.smithi192.stdout:3/695: mkdir d2/d4/d12/d1a/d1d/d7b/ddb 0 2024-04-24T22:45:39.971 INFO:tasks.workunit.client.0.smithi063.stdout:9/834: mknod d6/dbd/dcf/c116 0 2024-04-24T22:45:39.972 INFO:tasks.workunit.client.1.smithi192.stdout:3/696: creat d2/d27/fdc x:0 0 0 2024-04-24T22:45:39.978 INFO:tasks.workunit.client.1.smithi192.stdout:0/667: rename d3/d27/d3c/d6c/ldc to d3/d27/d28/dcc/le1 0 2024-04-24T22:45:39.979 INFO:tasks.workunit.client.0.smithi063.stdout:9/835: symlink d6/d2f/dcd/d1f/d20/de6/l117 0 2024-04-24T22:45:39.979 INFO:tasks.workunit.client.0.smithi063.stdout:9/836: readlink d6/d2f/dcd/l29 0 2024-04-24T22:45:39.980 INFO:tasks.workunit.client.0.smithi063.stdout:9/837: readlink d6/d2f/dcd/db/d1c/lf4 0 2024-04-24T22:45:39.981 INFO:tasks.workunit.client.1.smithi192.stdout:9/771: creat d3/dd/de/da7/f108 x:0 0 0 2024-04-24T22:45:39.982 INFO:tasks.workunit.client.1.smithi192.stdout:5/564: dread d0/f18 [0,4194304] 0 2024-04-24T22:45:39.983 INFO:tasks.workunit.client.1.smithi192.stdout:4/636: dwrite d2/d9/d6b/fac [0,4194304] 0 2024-04-24T22:45:39.986 INFO:tasks.workunit.client.0.smithi063.stdout:9/838: unlink d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fd1 0 2024-04-24T22:45:39.989 INFO:tasks.workunit.client.0.smithi063.stdout:6/879: dread db/d1d/d20/d3e/f45 [0,4194304] 0 2024-04-24T22:45:39.989 INFO:tasks.workunit.client.1.smithi192.stdout:3/697: rename d2/d4/d12/d1a/d1d/d55/d5f/ld8 to d2/d27/d28/d38/ldd 0 2024-04-24T22:45:39.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/565: fdatasync d0/d13/f39 0 2024-04-24T22:45:39.992 INFO:tasks.workunit.client.0.smithi063.stdout:2/921: write d0/d2/d2d/d31/d89/fbb [338101,11186] 0 2024-04-24T22:45:39.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/880: creat db/d1c/dc3/f127 x:0 0 0 2024-04-24T22:45:39.996 INFO:tasks.workunit.client.1.smithi192.stdout:4/637: mkdir d2/db/d75/da1/d7d/d9c/dda 0 2024-04-24T22:45:39.999 INFO:tasks.workunit.client.1.smithi192.stdout:3/698: fsync d2/d4/d12/f49 0 2024-04-24T22:45:40.002 INFO:tasks.workunit.client.0.smithi063.stdout:6/881: symlink db/d1d/d20/d29/d5d/df5/l128 0 2024-04-24T22:45:40.003 INFO:tasks.workunit.client.0.smithi063.stdout:9/839: link d6/d2f/dcd/cb7 d6/d2f/d42/d81/def/c118 0 2024-04-24T22:45:40.005 INFO:tasks.workunit.client.0.smithi063.stdout:5/788: write d5/d11/d76/fbc [548637,42641] 0 2024-04-24T22:45:40.006 INFO:tasks.workunit.client.1.smithi192.stdout:2/778: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:40.006 INFO:tasks.workunit.client.1.smithi192.stdout:4/638: readlink d2/l84 0 2024-04-24T22:45:40.007 INFO:tasks.workunit.client.1.smithi192.stdout:4/639: readlink d2/l84 0 2024-04-24T22:45:40.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/882: creat db/d1d/d20/d29/f129 x:0 0 0 2024-04-24T22:45:40.009 INFO:tasks.workunit.client.1.smithi192.stdout:6/591: truncate d0/d8/d22/fb4 3665354 0 2024-04-24T22:45:40.015 INFO:tasks.workunit.client.0.smithi063.stdout:5/789: mkdir d5/d11/d23/d2d/d62/d81/d10c 0 2024-04-24T22:45:40.018 INFO:tasks.workunit.client.0.smithi063.stdout:6/883: rename db/d85/db5 to db/d1d/d8d/dce/d12a 0 2024-04-24T22:45:40.020 INFO:tasks.workunit.client.0.smithi063.stdout:2/922: getdents d0/d2/d2f/ddc 0 2024-04-24T22:45:40.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/592: rmdir d0/d8/d18/d2e 39 2024-04-24T22:45:40.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/593: dread - d0/f71 zero size 2024-04-24T22:45:40.025 INFO:tasks.workunit.client.0.smithi063.stdout:5/790: unlink d5/d1d/d7d/l61 0 2024-04-24T22:45:40.027 INFO:tasks.workunit.client.0.smithi063.stdout:2/923: write d0/d2/fa8 [2869708,37390] 0 2024-04-24T22:45:40.027 INFO:tasks.workunit.client.0.smithi063.stdout:4/809: truncate d2/d19/f4d 450350 0 2024-04-24T22:45:40.028 INFO:tasks.workunit.client.0.smithi063.stdout:4/810: readlink d2/d19/d82/le0 0 2024-04-24T22:45:40.029 INFO:tasks.workunit.client.0.smithi063.stdout:5/791: readlink d5/d1d/dc0/lb3 0 2024-04-24T22:45:40.031 INFO:tasks.workunit.client.0.smithi063.stdout:6/884: creat db/d85/dd2/de6/f12b x:0 0 0 2024-04-24T22:45:40.033 INFO:tasks.workunit.client.1.smithi192.stdout:6/594: creat d0/d8/d29/d4c/db0/fcc x:0 0 0 2024-04-24T22:45:40.034 INFO:tasks.workunit.client.0.smithi063.stdout:5/792: stat d5/d11/c104 0 2024-04-24T22:45:40.035 INFO:tasks.workunit.client.1.smithi192.stdout:3/699: creat d2/d4/fde x:0 0 0 2024-04-24T22:45:40.035 INFO:tasks.workunit.client.0.smithi063.stdout:5/793: fdatasync d5/d1d/f93 0 2024-04-24T22:45:40.035 INFO:tasks.workunit.client.0.smithi063.stdout:8/876: dwrite d1/d22/f31 [0,4194304] 0 2024-04-24T22:45:40.036 INFO:tasks.workunit.client.0.smithi063.stdout:5/794: fsync d5/d11/d23/d2d/ff9 0 2024-04-24T22:45:40.037 INFO:tasks.workunit.client.1.smithi192.stdout:5/566: getdents d0/d13 0 2024-04-24T22:45:40.038 INFO:tasks.workunit.client.0.smithi063.stdout:0/769: truncate d6/d64/d21/fb3 1607567 0 2024-04-24T22:45:40.040 INFO:tasks.workunit.client.0.smithi063.stdout:2/924: rename d0/d2/d2d/d31/d89/d48/f6f to d0/d2/d2d/d31/d89/d1c/f11e 0 2024-04-24T22:45:40.051 INFO:tasks.workunit.client.1.smithi192.stdout:2/779: rename d4/d9/d10/d6c/d6f/c7a to d4/d9/dd4/cfd 0 2024-04-24T22:45:40.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/567: mkdir d0/d1b/d62/dc1/dc2 0 2024-04-24T22:45:40.055 INFO:tasks.workunit.client.0.smithi063.stdout:4/811: mknod d2/d19/d33/dc1/d50/dfa/c10b 0 2024-04-24T22:45:40.059 INFO:tasks.workunit.client.1.smithi192.stdout:5/568: rename d0/d13/d19/d64/d5f/f8a to d0/d13/d19/d64/d5f/d2c/d3c/fc3 0 2024-04-24T22:45:40.062 INFO:tasks.workunit.client.1.smithi192.stdout:6/595: getdents d0/d8/d3d/d3e/d7e 0 2024-04-24T22:45:40.063 INFO:tasks.workunit.client.1.smithi192.stdout:3/700: dread d2/d4/f16 [0,4194304] 0 2024-04-24T22:45:40.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/780: unlink d4/d9/c83 0 2024-04-24T22:45:40.065 INFO:tasks.workunit.client.0.smithi063.stdout:2/925: fdatasync d0/d2/d35/f52 0 2024-04-24T22:45:40.068 INFO:tasks.workunit.client.1.smithi192.stdout:5/569: fdatasync d0/d13/d19/f4b 0 2024-04-24T22:45:40.069 INFO:tasks.workunit.client.0.smithi063.stdout:7/775: link d0/d4/de/d17/d28/l2d d0/d4/de/d17/d35/l100 0 2024-04-24T22:45:40.070 INFO:tasks.workunit.client.0.smithi063.stdout:7/776: fsync d0/d4/de/d6f/d2e/f85 0 2024-04-24T22:45:40.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/701: creat d2/d4/d12/d39/d97/fdf x:0 0 0 2024-04-24T22:45:40.076 INFO:tasks.workunit.client.1.smithi192.stdout:3/702: truncate d2/d62/db2/f85 1023874 0 2024-04-24T22:45:40.076 INFO:tasks.workunit.client.1.smithi192.stdout:6/596: dread d0/d8/f2b [4194304,4194304] 0 2024-04-24T22:45:40.077 INFO:tasks.workunit.client.0.smithi063.stdout:0/770: symlink d6/d64/d21/d25/d62/d63/d89/l106 0 2024-04-24T22:45:40.081 INFO:tasks.workunit.client.0.smithi063.stdout:3/833: mkdir da/d18/d3a/d121 0 2024-04-24T22:45:40.083 INFO:tasks.workunit.client.0.smithi063.stdout:2/926: fsync d0/d2/d39/d11a/de2/f74 0 2024-04-24T22:45:40.087 INFO:tasks.workunit.client.1.smithi192.stdout:2/781: mknod d4/d9/d10/d6c/cfe 0 2024-04-24T22:45:40.088 INFO:tasks.workunit.client.1.smithi192.stdout:2/782: chown d4/d29/ld0 0 1 2024-04-24T22:45:40.089 INFO:tasks.workunit.client.0.smithi063.stdout:2/927: dread d0/dd/d72/d77/dda/d7c/fb9 [0,4194304] 0 2024-04-24T22:45:40.089 INFO:tasks.workunit.client.0.smithi063.stdout:1/877: symlink d3/db5/d21/d9f/dd5/d106/d11/l11c 0 2024-04-24T22:45:40.090 INFO:tasks.workunit.client.1.smithi192.stdout:6/597: symlink d0/d8/d29/d4c/d4d/d83/lcd 0 2024-04-24T22:45:40.092 INFO:tasks.workunit.client.0.smithi063.stdout:7/777: mknod d0/d4/de/d6f/d2e/d55/ddc/c101 0 2024-04-24T22:45:40.094 INFO:tasks.workunit.client.1.smithi192.stdout:3/703: rename d2/d27/dc6/d4b/c6b to d2/d4/d12/d1a/d1d/d55/d5f/d9b/ce0 0 2024-04-24T22:45:40.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/928: dwrite d0/d2/d39/ffd [0,4194304] 0 2024-04-24T22:45:40.097 INFO:tasks.workunit.client.0.smithi063.stdout:5/795: getdents d5/d24/d49/d3b/dc5 0 2024-04-24T22:45:40.098 INFO:tasks.workunit.client.1.smithi192.stdout:2/783: mkdir d4/d29/d60/d9a/dff 0 2024-04-24T22:45:40.098 INFO:tasks.workunit.client.1.smithi192.stdout:5/570: dread d0/d13/d19/d64/d5f/d2c/d3c/f5a [0,4194304] 0 2024-04-24T22:45:40.101 INFO:tasks.workunit.client.1.smithi192.stdout:3/704: dwrite d2/d4/fd2 [0,4194304] 0 2024-04-24T22:45:40.101 INFO:tasks.workunit.client.0.smithi063.stdout:0/771: dread d6/d64/d21/d25/f55 [0,4194304] 0 2024-04-24T22:45:40.102 INFO:tasks.workunit.client.0.smithi063.stdout:0/772: chown d6/da/dc/f1e 12 1 2024-04-24T22:45:40.102 INFO:tasks.workunit.client.0.smithi063.stdout:8/877: rename d1/d2/d3/f82 to d1/d11/f117 0 2024-04-24T22:45:40.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/773: dwrite d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:40.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/878: dwrite d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:40.116 INFO:tasks.workunit.client.0.smithi063.stdout:0/774: read d6/da/d2b/f40 [1135786,10087] 0 2024-04-24T22:45:40.116 INFO:tasks.workunit.client.0.smithi063.stdout:0/775: write d6/d64/d21/d9c/fbd [1638771,801] 0 2024-04-24T22:45:40.120 INFO:tasks.workunit.client.1.smithi192.stdout:6/598: dread d0/d8/d3d/f55 [0,4194304] 0 2024-04-24T22:45:40.122 INFO:tasks.workunit.client.1.smithi192.stdout:3/705: read d2/d27/dc6/d4f/f5a [276737,91702] 0 2024-04-24T22:45:40.123 INFO:tasks.workunit.client.1.smithi192.stdout:5/571: creat d0/d13/d19/d64/d5f/fc4 x:0 0 0 2024-04-24T22:45:40.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/784: rmdir d4/d29/d60/d9a/dff 0 2024-04-24T22:45:40.127 INFO:tasks.workunit.client.1.smithi192.stdout:6/599: dread - d0/d8/d29/d4c/d4d/f72 zero size 2024-04-24T22:45:40.130 INFO:tasks.workunit.client.1.smithi192.stdout:5/572: unlink d0/d13/d19/d71/d7f/d9a/fa5 0 2024-04-24T22:45:40.135 INFO:tasks.workunit.client.0.smithi063.stdout:8/879: creat d1/d21/f118 x:0 0 0 2024-04-24T22:45:40.136 INFO:tasks.workunit.client.1.smithi192.stdout:0/668: dwrite d3/d27/d28/dcc/d5c/f3d [0,4194304] 0 2024-04-24T22:45:40.137 INFO:tasks.workunit.client.1.smithi192.stdout:0/669: chown d3/d27/d28/dcc/da8 113646 1 2024-04-24T22:45:40.141 INFO:tasks.workunit.client.0.smithi063.stdout:0/776: unlink d6/da/d2b/fa7 0 2024-04-24T22:45:40.141 INFO:tasks.workunit.client.0.smithi063.stdout:0/777: readlink d6/d64/d24/lac 0 2024-04-24T22:45:40.146 INFO:tasks.workunit.client.0.smithi063.stdout:9/840: dwrite d6/d2f/d42/d81/f96 [0,4194304] 0 2024-04-24T22:45:40.147 INFO:tasks.workunit.client.1.smithi192.stdout:5/573: creat d0/fc5 x:0 0 0 2024-04-24T22:45:40.148 INFO:tasks.workunit.client.1.smithi192.stdout:5/574: truncate d0/fc5 344961 0 2024-04-24T22:45:40.150 INFO:tasks.workunit.client.1.smithi192.stdout:4/640: truncate d2/db/d75/da1/d36/f62 569557 0 2024-04-24T22:45:40.153 INFO:tasks.workunit.client.1.smithi192.stdout:4/641: dread d2/db/d75/da1/d36/d5b/d28/d2f/fb1 [0,4194304] 0 2024-04-24T22:45:40.156 INFO:tasks.workunit.client.0.smithi063.stdout:7/778: rename d0/c29 to d0/d4/de/d17/d35/d69/da0/c102 0 2024-04-24T22:45:40.157 INFO:tasks.workunit.client.1.smithi192.stdout:0/670: link d3/d27/d28/d2e/f53 d3/d27/d28/dcc/d5c/fe2 0 2024-04-24T22:45:40.158 INFO:tasks.workunit.client.0.smithi063.stdout:6/885: write db/d1d/d20/d3e/f5e [2520797,120422] 0 2024-04-24T22:45:40.159 INFO:tasks.workunit.client.0.smithi063.stdout:6/886: dread - db/d1d/d8d/dfc/d11c/f125 zero size 2024-04-24T22:45:40.167 INFO:tasks.workunit.client.0.smithi063.stdout:0/778: fdatasync d6/d64/d21/d25/d62/d63/d89/d95/da4/fc2 0 2024-04-24T22:45:40.171 INFO:tasks.workunit.client.1.smithi192.stdout:7/736: sync 2024-04-24T22:45:40.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/641: sync 2024-04-24T22:45:40.171 INFO:tasks.workunit.client.1.smithi192.stdout:1/563: sync 2024-04-24T22:45:40.172 INFO:tasks.workunit.client.1.smithi192.stdout:4/642: creat d2/d9/fdb x:0 0 0 2024-04-24T22:45:40.174 INFO:tasks.workunit.client.1.smithi192.stdout:4/643: dread - d2/da5/fb9 zero size 2024-04-24T22:45:40.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/737: dread d2/db/d21/d42/f4d [0,4194304] 0 2024-04-24T22:45:40.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/642: mkdir db/d5d/dcb 0 2024-04-24T22:45:40.182 INFO:tasks.workunit.client.0.smithi063.stdout:3/834: getdents da/d39/d94/de4 0 2024-04-24T22:45:40.183 INFO:tasks.workunit.client.1.smithi192.stdout:1/564: symlink d9/dd/d74/lba 0 2024-04-24T22:45:40.185 INFO:tasks.workunit.client.1.smithi192.stdout:0/671: dread d3/f39 [0,4194304] 0 2024-04-24T22:45:40.186 INFO:tasks.workunit.client.0.smithi063.stdout:6/887: chown db/d1d/d20/d3f/d7f/d68/d70/cb8 167 1 2024-04-24T22:45:40.186 INFO:tasks.workunit.client.1.smithi192.stdout:0/672: chown d3/d27/d28/dcc/d5c/f61 95854 1 2024-04-24T22:45:40.188 INFO:tasks.workunit.client.1.smithi192.stdout:4/644: unlink d2/db/d75/da1/d36/d5b/d28/d2f/fa2 0 2024-04-24T22:45:40.191 INFO:tasks.workunit.client.0.smithi063.stdout:6/888: dwrite db/f10 [0,4194304] 0 2024-04-24T22:45:40.192 INFO:tasks.workunit.client.1.smithi192.stdout:8/643: chown db/dc/le 3162265 1 2024-04-24T22:45:40.193 INFO:tasks.workunit.client.1.smithi192.stdout:1/565: readlink d9/dd/l5a 0 2024-04-24T22:45:40.194 INFO:tasks.workunit.client.0.smithi063.stdout:0/779: write d6/d74/fab [669425,101989] 0 2024-04-24T22:45:40.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/841: creat d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d107/f119 x:0 0 0 2024-04-24T22:45:40.201 INFO:tasks.workunit.client.1.smithi192.stdout:1/566: dread d9/d15/d2d/f6c [0,4194304] 0 2024-04-24T22:45:40.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/673: dwrite d3/f39 [0,4194304] 0 2024-04-24T22:45:40.205 INFO:tasks.workunit.client.0.smithi063.stdout:6/889: creat db/d85/dd2/de6/f12c x:0 0 0 2024-04-24T22:45:40.206 INFO:tasks.workunit.client.0.smithi063.stdout:0/780: mknod d6/d64/d21/d9c/c107 0 2024-04-24T22:45:40.208 INFO:tasks.workunit.client.1.smithi192.stdout:8/644: symlink db/d5d/d74/da0/lcc 0 2024-04-24T22:45:40.209 INFO:tasks.workunit.client.0.smithi063.stdout:2/929: sync 2024-04-24T22:45:40.210 INFO:tasks.workunit.client.0.smithi063.stdout:0/781: dread d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:40.211 INFO:tasks.workunit.client.1.smithi192.stdout:4/645: mknod d2/d9/cdc 0 2024-04-24T22:45:40.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/835: link da/d39/d94/db3/fdf da/d18/d47/f122 0 2024-04-24T22:45:40.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/779: write d0/d4/de/d6f/d5c/d9c/ffa [2701,109861] 0 2024-04-24T22:45:40.215 INFO:tasks.workunit.client.1.smithi192.stdout:1/567: rename d9/d15/d2d/d78 to d9/d15/d2d/d48/d93/da4/dbb 0 2024-04-24T22:45:40.215 INFO:tasks.workunit.client.1.smithi192.stdout:4/646: rename d2/db to d2/db/d5d/d77/d82/d8d/ddd 22 2024-04-24T22:45:40.217 INFO:tasks.workunit.client.0.smithi063.stdout:6/890: write db/d1d/d2d/d69/d82/f9a [2214602,75693] 0 2024-04-24T22:45:40.218 INFO:tasks.workunit.client.1.smithi192.stdout:4/647: chown d2/db/d75/da1/d36/d5b/d28/l56 240812 1 2024-04-24T22:45:40.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/891: chown db/d1d/d20/d29/l8a 377818265 1 2024-04-24T22:45:40.218 INFO:tasks.workunit.client.1.smithi192.stdout:4/648: write d2/d9/fc8 [320040,38235] 0 2024-04-24T22:45:40.221 INFO:tasks.workunit.client.1.smithi192.stdout:4/649: write d2/db/d75/da1/d7d/d9c/fc4 [907275,25853] 0 2024-04-24T22:45:40.222 INFO:tasks.workunit.client.1.smithi192.stdout:8/645: mkdir db/d43/d7c/d85/dcd 0 2024-04-24T22:45:40.223 INFO:tasks.workunit.client.1.smithi192.stdout:8/646: readlink db/dc/l67 0 2024-04-24T22:45:40.224 INFO:tasks.workunit.client.0.smithi063.stdout:1/878: dwrite d3/de0/d79/d96/d112/f92 [0,4194304] 0 2024-04-24T22:45:40.225 INFO:tasks.workunit.client.1.smithi192.stdout:1/568: creat d9/d15/d5e/d90/fbc x:0 0 0 2024-04-24T22:45:40.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/930: mkdir d0/d2/d2d/d31/d89/d48/d11f 0 2024-04-24T22:45:40.229 INFO:tasks.workunit.client.0.smithi063.stdout:0/782: creat d6/d64/d21/d9c/f108 x:0 0 0 2024-04-24T22:45:40.230 INFO:tasks.workunit.client.0.smithi063.stdout:1/879: dwrite d3/de0/d79/fc1 [0,4194304] 0 2024-04-24T22:45:40.230 INFO:tasks.workunit.client.1.smithi192.stdout:4/650: creat d2/d9/d25/d2e/fde x:0 0 0 2024-04-24T22:45:40.231 INFO:tasks.workunit.client.0.smithi063.stdout:3/836: truncate da/d18/d47/d4d/d62/f6f 272796 0 2024-04-24T22:45:40.241 INFO:tasks.workunit.client.1.smithi192.stdout:2/785: dwrite d4/f5e [0,4194304] 0 2024-04-24T22:45:40.241 INFO:tasks.workunit.client.0.smithi063.stdout:9/842: creat d6/d2f/dcd/d1f/d20/d22/f11a x:0 0 0 2024-04-24T22:45:40.242 INFO:tasks.workunit.client.0.smithi063.stdout:7/780: dread d0/d4/de/d6f/d5c/d9c/fd8 [0,4194304] 0 2024-04-24T22:45:40.243 INFO:tasks.workunit.client.0.smithi063.stdout:9/843: write d6/d2f/dcd/db/d6b/d4f/f79 [3180590,34690] 0 2024-04-24T22:45:40.244 INFO:tasks.workunit.client.1.smithi192.stdout:6/600: truncate d0/d8/f2b 7326414 0 2024-04-24T22:45:40.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/812: truncate d2/d19/f4d 1291733 0 2024-04-24T22:45:40.251 INFO:tasks.workunit.client.0.smithi063.stdout:0/783: rename d6/d3a/dba/fe0 to d6/d64/d21/d25/d62/d63/d7c/f109 0 2024-04-24T22:45:40.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/796: truncate d5/f14 1277862 0 2024-04-24T22:45:40.254 INFO:tasks.workunit.client.0.smithi063.stdout:3/837: chown da/l66 2989 1 2024-04-24T22:45:40.255 INFO:tasks.workunit.client.1.smithi192.stdout:0/674: getdents d3/d27/d28/dcc/d5c 0 2024-04-24T22:45:40.256 INFO:tasks.workunit.client.1.smithi192.stdout:5/575: write d0/d13/f39 [583040,19033] 0 2024-04-24T22:45:40.257 INFO:tasks.workunit.client.1.smithi192.stdout:5/576: write d0/d13/d19/d64/d5f/f72 [2803509,20555] 0 2024-04-24T22:45:40.258 INFO:tasks.workunit.client.0.smithi063.stdout:1/880: creat d3/db5/d21/d9f/dd5/d106/df5/f11d x:0 0 0 2024-04-24T22:45:40.258 INFO:tasks.workunit.client.1.smithi192.stdout:5/577: chown d0/d13/d19/d64/d4f/d52/f61 52 1 2024-04-24T22:45:40.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/892: dread db/d1d/d20/d3f/fc5 [0,4194304] 0 2024-04-24T22:45:40.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/893: fsync db/d1d/d2d/f76 0 2024-04-24T22:45:40.263 INFO:tasks.workunit.client.0.smithi063.stdout:7/781: creat d0/d4/de/d6f/d5c/d9c/f103 x:0 0 0 2024-04-24T22:45:40.264 INFO:tasks.workunit.client.1.smithi192.stdout:8/647: getdents db/dc/d22/d31/d33/db7 0 2024-04-24T22:45:40.269 INFO:tasks.workunit.client.0.smithi063.stdout:2/931: symlink d0/l120 0 2024-04-24T22:45:40.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/880: write d1/d22/f5b [1857450,96729] 0 2024-04-24T22:45:40.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/932: write d0/dd/d72/d77/f78 [632938,103570] 0 2024-04-24T22:45:40.272 INFO:tasks.workunit.client.1.smithi192.stdout:5/578: creat d0/d13/d19/d83/fc6 x:0 0 0 2024-04-24T22:45:40.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/933: read d0/d2/f47 [3078646,17464] 0 2024-04-24T22:45:40.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/934: fdatasync d0/d2/d2d/d31/d89/d1c/d3f/f110 0 2024-04-24T22:45:40.275 INFO:tasks.workunit.client.1.smithi192.stdout:7/738: dwrite d2/db/f45 [0,4194304] 0 2024-04-24T22:45:40.276 INFO:tasks.workunit.client.1.smithi192.stdout:4/651: getdents d2/db 0 2024-04-24T22:45:40.276 INFO:tasks.workunit.client.0.smithi063.stdout:2/935: write d0/d2/d39/d11a/de2/de4/fe9 [443176,108686] 0 2024-04-24T22:45:40.276 INFO:tasks.workunit.client.0.smithi063.stdout:2/936: chown d0/d2/d2f/ddc 126898336 1 2024-04-24T22:45:40.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/784: creat d6/d3a/dba/f10a x:0 0 0 2024-04-24T22:45:40.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/648: mknod db/dc/d29/d9f/cce 0 2024-04-24T22:45:40.282 INFO:tasks.workunit.client.1.smithi192.stdout:5/579: unlink d0/fc 0 2024-04-24T22:45:40.287 INFO:tasks.workunit.client.0.smithi063.stdout:1/881: mknod d3/db5/d21/d9f/dd5/d106/df5/c11e 0 2024-04-24T22:45:40.288 INFO:tasks.workunit.client.0.smithi063.stdout:6/894: mkdir db/d1d/d20/d29/d5d/d12d 0 2024-04-24T22:45:40.289 INFO:tasks.workunit.client.0.smithi063.stdout:6/895: write db/d1d/d20/f38 [1491472,40196] 0 2024-04-24T22:45:40.290 INFO:tasks.workunit.client.0.smithi063.stdout:7/782: write d0/d4/de/d6f/d5c/d9c/fd8 [1318322,17378] 0 2024-04-24T22:45:40.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/896: chown db/d1d/d20/d3f/d7f/d68/fb0 38223 1 2024-04-24T22:45:40.292 INFO:tasks.workunit.client.0.smithi063.stdout:6/897: stat db/d1d/d20/d3f/d7f/d68/d70/ce2 0 2024-04-24T22:45:40.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/844: symlink d6/d2f/dcd/d1f/d20/d2a/df2/l11b 0 2024-04-24T22:45:40.293 INFO:tasks.workunit.client.1.smithi192.stdout:8/649: dread - db/dc/f8e zero size 2024-04-24T22:45:40.294 INFO:tasks.workunit.client.0.smithi063.stdout:6/898: truncate db/d1d/d20/d29/dab/f124 559593 0 2024-04-24T22:45:40.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/650: fdatasync db/dc/d22/d31/d33/d81/fb1 0 2024-04-24T22:45:40.296 INFO:tasks.workunit.client.1.smithi192.stdout:8/651: dread - db/dc/d29/f86 zero size 2024-04-24T22:45:40.297 INFO:tasks.workunit.client.1.smithi192.stdout:6/601: getdents d0/d8/d22/dcb 0 2024-04-24T22:45:40.301 INFO:tasks.workunit.client.1.smithi192.stdout:1/569: write d9/d15/d2d/d31/d2f/f73 [814914,127157] 0 2024-04-24T22:45:40.302 INFO:tasks.workunit.client.0.smithi063.stdout:8/881: dwrite d1/d11/d8b/dd6/ff0 [4194304,4194304] 0 2024-04-24T22:45:40.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/937: creat d0/d2/d2d/d31/d89/d48/f121 x:0 0 0 2024-04-24T22:45:40.308 INFO:tasks.workunit.client.1.smithi192.stdout:8/652: mknod db/dc/d22/d31/d33/d81/ccf 0 2024-04-24T22:45:40.309 INFO:tasks.workunit.client.0.smithi063.stdout:0/785: chown d6/d64/d24/l46 69238784 1 2024-04-24T22:45:40.309 INFO:tasks.workunit.client.1.smithi192.stdout:4/652: dread d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 [0,4194304] 0 2024-04-24T22:45:40.310 INFO:tasks.workunit.client.1.smithi192.stdout:6/602: fsync d0/f82 0 2024-04-24T22:45:40.311 INFO:tasks.workunit.client.1.smithi192.stdout:6/603: chown d0/d8/d29/ca7 3041686 1 2024-04-24T22:45:40.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/570: rmdir d9/d15/d2d/d63/d68 39 2024-04-24T22:45:40.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/580: creat d0/d1b/fc7 x:0 0 0 2024-04-24T22:45:40.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/581: read d0/f18 [767552,106241] 0 2024-04-24T22:45:40.319 INFO:tasks.workunit.client.1.smithi192.stdout:0/675: dwrite d3/d27/d28/dcc/d5c/d80/d94/fbe [0,4194304] 0 2024-04-24T22:45:40.321 INFO:tasks.workunit.client.1.smithi192.stdout:8/653: dread - db/dc/d22/d31/d33/d8f/fa2 zero size 2024-04-24T22:45:40.321 INFO:tasks.workunit.client.1.smithi192.stdout:8/654: fsync db/fae 0 2024-04-24T22:45:40.324 INFO:tasks.workunit.client.1.smithi192.stdout:4/653: unlink d2/d9/d25/f68 0 2024-04-24T22:45:40.324 INFO:tasks.workunit.client.1.smithi192.stdout:4/654: chown d2/db/d75/cb8 857880 1 2024-04-24T22:45:40.328 INFO:tasks.workunit.client.1.smithi192.stdout:7/739: getdents d2/db/d2a/d57 0 2024-04-24T22:45:40.329 INFO:tasks.workunit.client.1.smithi192.stdout:7/740: truncate d2/db/d21/fd7 29663 0 2024-04-24T22:45:40.330 INFO:tasks.workunit.client.1.smithi192.stdout:6/604: creat d0/d8/d29/d4c/fce x:0 0 0 2024-04-24T22:45:40.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/582: rename d0/d13/d3b/d2f/l93 to d0/d13/d19/d71/d7f/d9a/lc8 0 2024-04-24T22:45:40.343 INFO:tasks.workunit.client.1.smithi192.stdout:8/655: creat db/d5d/d74/d96/fd0 x:0 0 0 2024-04-24T22:45:40.345 INFO:tasks.workunit.client.1.smithi192.stdout:5/583: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-24T22:45:40.346 INFO:tasks.workunit.client.1.smithi192.stdout:7/741: symlink d2/db/d21/d42/le6 0 2024-04-24T22:45:40.356 INFO:tasks.workunit.client.1.smithi192.stdout:8/656: creat db/d43/d76/d92/fd1 x:0 0 0 2024-04-24T22:45:40.358 INFO:tasks.workunit.client.1.smithi192.stdout:4/655: mkdir d2/db/d75/da1/ddf 0 2024-04-24T22:45:40.358 INFO:tasks.workunit.client.1.smithi192.stdout:1/571: dwrite d9/f14 [0,4194304] 0 2024-04-24T22:45:40.358 INFO:tasks.workunit.client.1.smithi192.stdout:4/656: fsync d2/fd6 0 2024-04-24T22:45:40.360 INFO:tasks.workunit.client.1.smithi192.stdout:8/657: read f5 [6080061,63674] 0 2024-04-24T22:45:40.365 INFO:tasks.workunit.client.1.smithi192.stdout:0/676: creat d3/d27/d3c/fe3 x:0 0 0 2024-04-24T22:45:40.367 INFO:tasks.workunit.client.1.smithi192.stdout:0/677: write d3/d27/d28/d2e/f4c [2458708,94844] 0 2024-04-24T22:45:40.368 INFO:tasks.workunit.client.1.smithi192.stdout:0/678: write d3/d27/d28/d31/fb5 [1145340,117] 0 2024-04-24T22:45:40.372 INFO:tasks.workunit.client.1.smithi192.stdout:4/657: write d2/db/d75/da1/d36/d5b/f37 [1186204,75001] 0 2024-04-24T22:45:40.374 INFO:tasks.workunit.client.1.smithi192.stdout:8/658: creat db/d43/d7c/fd2 x:0 0 0 2024-04-24T22:45:40.375 INFO:tasks.workunit.client.1.smithi192.stdout:8/659: dread - db/dc/d22/d31/d33/fbc zero size 2024-04-24T22:45:40.376 INFO:tasks.workunit.client.1.smithi192.stdout:7/742: symlink d2/db/d25/le7 0 2024-04-24T22:45:40.377 INFO:tasks.workunit.client.1.smithi192.stdout:7/743: dread - d2/db/d21/d42/d67/d9b/d6e/fe4 zero size 2024-04-24T22:45:40.383 INFO:tasks.workunit.client.1.smithi192.stdout:0/679: rename d3/d27/d28/f1d to d3/d27/d28/d31/dbc/fe4 0 2024-04-24T22:45:40.383 INFO:tasks.workunit.client.1.smithi192.stdout:0/680: fsync d3/d27/d28/f34 0 2024-04-24T22:45:40.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/681: write d3/d27/d3c/d3b/fd1 [704381,31774] 0 2024-04-24T22:45:40.389 INFO:tasks.workunit.client.1.smithi192.stdout:7/744: truncate d2/db/d21/d42/d67/d9b/d6e/f9e 571925 0 2024-04-24T22:45:40.389 INFO:tasks.workunit.client.1.smithi192.stdout:6/605: write d0/d8/d29/d5a/d63/fc9 [1088366,105522] 0 2024-04-24T22:45:40.392 INFO:tasks.workunit.client.1.smithi192.stdout:6/606: dread - d0/d8/d29/d4c/f6d zero size 2024-04-24T22:45:40.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/584: dwrite d0/d13/d19/d83/f87 [0,4194304] 0 2024-04-24T22:45:40.395 INFO:tasks.workunit.client.1.smithi192.stdout:0/682: creat d3/d27/d6a/fe5 x:0 0 0 2024-04-24T22:45:40.401 INFO:tasks.workunit.client.1.smithi192.stdout:7/745: mknod d2/db/d21/d42/d67/dae/ce8 0 2024-04-24T22:45:40.402 INFO:tasks.workunit.client.0.smithi063.stdout:1/882: chown d3/db5/d21/f91 118406305 1 2024-04-24T22:45:40.403 INFO:tasks.workunit.client.1.smithi192.stdout:6/607: creat d0/d8/d3d/d3e/d40/fcf x:0 0 0 2024-04-24T22:45:40.405 INFO:tasks.workunit.client.1.smithi192.stdout:5/585: truncate d0/d13/f43 406416 0 2024-04-24T22:45:40.406 INFO:tasks.workunit.client.0.smithi063.stdout:1/883: read d3/de0/d79/f52 [1664011,77525] 0 2024-04-24T22:45:40.407 INFO:tasks.workunit.client.1.smithi192.stdout:0/683: symlink d3/d27/d28/d31/dbc/le6 0 2024-04-24T22:45:40.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/660: creat db/dc/d22/d31/fd3 x:0 0 0 2024-04-24T22:45:40.409 INFO:tasks.workunit.client.0.smithi063.stdout:1/884: write d3/db5/d21/d9f/dd5/d106/d39/f66 [105837,26894] 0 2024-04-24T22:45:40.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/661: fsync db/d5d/f84 0 2024-04-24T22:45:40.410 INFO:tasks.workunit.client.1.smithi192.stdout:9/772: sync 2024-04-24T22:45:40.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/662: fdatasync db/dc/d22/d31/d33/d81/faf 0 2024-04-24T22:45:40.411 INFO:tasks.workunit.client.1.smithi192.stdout:8/663: write db/f34 [6684893,64764] 0 2024-04-24T22:45:40.413 INFO:tasks.workunit.client.1.smithi192.stdout:4/658: rename d2/db/d75/da1/d36/d5b/lb5 to d2/le0 0 2024-04-24T22:45:40.414 INFO:tasks.workunit.client.1.smithi192.stdout:4/659: dread - d2/d5c/dbb/fcd zero size 2024-04-24T22:45:40.414 INFO:tasks.workunit.client.0.smithi063.stdout:6/899: symlink db/d85/dd2/l12e 0 2024-04-24T22:45:40.415 INFO:tasks.workunit.client.1.smithi192.stdout:4/660: write d2/fd0 [851249,127046] 0 2024-04-24T22:45:40.417 INFO:tasks.workunit.client.1.smithi192.stdout:6/608: creat d0/d8/d3d/d9d/fd0 x:0 0 0 2024-04-24T22:45:40.417 INFO:tasks.workunit.client.0.smithi063.stdout:4/813: link d2/d19/d82/l9c d2/d19/d33/dc1/d109/l10c 0 2024-04-24T22:45:40.421 INFO:tasks.workunit.client.1.smithi192.stdout:9/773: mknod d3/dd/de/da7/c109 0 2024-04-24T22:45:40.422 INFO:tasks.workunit.client.0.smithi063.stdout:8/882: creat d1/d2/d3/db6/f119 x:0 0 0 2024-04-24T22:45:40.427 INFO:tasks.workunit.client.0.smithi063.stdout:0/786: creat d6/d64/d21/d25/d62/d63/d7c/f10b x:0 0 0 2024-04-24T22:45:40.429 INFO:tasks.workunit.client.0.smithi063.stdout:3/838: creat da/d18/f123 x:0 0 0 2024-04-24T22:45:40.429 INFO:tasks.workunit.client.1.smithi192.stdout:7/746: mknod d2/db/d49/dca/ce9 0 2024-04-24T22:45:40.431 INFO:tasks.workunit.client.1.smithi192.stdout:4/661: readlink d2/l6 0 2024-04-24T22:45:40.432 INFO:tasks.workunit.client.0.smithi063.stdout:9/845: dwrite d6/d2f/dcd/db/d1c/d78/d9a/daa/fc3 [0,4194304] 0 2024-04-24T22:45:40.433 INFO:tasks.workunit.client.1.smithi192.stdout:0/684: symlink d3/d27/d28/d2e/d3f/dd7/le7 0 2024-04-24T22:45:40.434 INFO:tasks.workunit.client.0.smithi063.stdout:7/783: mknod d0/c104 0 2024-04-24T22:45:40.437 INFO:tasks.workunit.client.1.smithi192.stdout:6/609: rename d0/d8/d22/c25 to d0/d8/d29/d5a/cd1 0 2024-04-24T22:45:40.438 INFO:tasks.workunit.client.1.smithi192.stdout:6/610: dread - d0/d8/d29/f95 zero size 2024-04-24T22:45:40.439 INFO:tasks.workunit.client.1.smithi192.stdout:6/611: fdatasync d0/d8/d3d/f49 0 2024-04-24T22:45:40.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/885: symlink d3/db5/d21/d9f/dd5/d106/d39/dbf/l11f 0 2024-04-24T22:45:40.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/747: creat d2/db/d21/d42/d67/d9b/d6e/db5/fea x:0 0 0 2024-04-24T22:45:40.445 INFO:tasks.workunit.client.0.smithi063.stdout:6/900: mkdir db/d1d/d20/d3f/d5f/d93/d12f 0 2024-04-24T22:45:40.446 INFO:tasks.workunit.client.1.smithi192.stdout:4/662: mknod d2/db/d75/da1/d36/d5b/d28/d2f/ce1 0 2024-04-24T22:45:40.447 INFO:tasks.workunit.client.1.smithi192.stdout:8/664: dwrite db/dc/d29/f86 [0,4194304] 0 2024-04-24T22:45:40.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/663: fsync d2/db/d75/da1/d7d/fca 0 2024-04-24T22:45:40.448 INFO:tasks.workunit.client.1.smithi192.stdout:8/665: stat db/dc/d29/l51 0 2024-04-24T22:45:40.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/883: dread d1/d11/d32/fb2 [0,4194304] 0 2024-04-24T22:45:40.457 INFO:tasks.workunit.client.1.smithi192.stdout:6/612: rename d0/cd to d0/d8/d3d/d9d/cd2 0 2024-04-24T22:45:40.457 INFO:tasks.workunit.client.1.smithi192.stdout:9/774: write d3/dd/de/dba/f42 [1616948,62662] 0 2024-04-24T22:45:40.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/839: creat da/d40/d111/f124 x:0 0 0 2024-04-24T22:45:40.459 INFO:tasks.workunit.client.0.smithi063.stdout:3/840: readlink da/d18/d47/d4d/d62/le8 0 2024-04-24T22:45:40.460 INFO:tasks.workunit.client.0.smithi063.stdout:5/797: dread d5/f14 [0,4194304] 0 2024-04-24T22:45:40.461 INFO:tasks.workunit.client.1.smithi192.stdout:7/748: read d2/db/d21/d42/d67/d9b/f48 [917228,68938] 0 2024-04-24T22:45:40.461 INFO:tasks.workunit.client.0.smithi063.stdout:7/784: mknod d0/d4/de/d17/d28/d31/d99/c105 0 2024-04-24T22:45:40.462 INFO:tasks.workunit.client.0.smithi063.stdout:1/886: symlink d3/da/d34/d99/l120 0 2024-04-24T22:45:40.463 INFO:tasks.workunit.client.0.smithi063.stdout:6/901: mknod db/d1d/d20/d3f/d94/c130 0 2024-04-24T22:45:40.465 INFO:tasks.workunit.client.0.smithi063.stdout:4/814: rename d2/d19/f4d to d2/d9f/df8/d10a/f10d 0 2024-04-24T22:45:40.465 INFO:tasks.workunit.client.1.smithi192.stdout:9/775: mkdir d3/dd/de/d17/d89/db4/db9/d10a 0 2024-04-24T22:45:40.466 INFO:tasks.workunit.client.0.smithi063.stdout:4/815: write d2/d8/d40/d51/dd5/ff2 [574408,33894] 0 2024-04-24T22:45:40.468 INFO:tasks.workunit.client.0.smithi063.stdout:2/938: creat d0/d2/d27/f122 x:0 0 0 2024-04-24T22:45:40.471 INFO:tasks.workunit.client.1.smithi192.stdout:0/685: getdents d3/d27/d3c/d64 0 2024-04-24T22:45:40.473 INFO:tasks.workunit.client.0.smithi063.stdout:2/939: dwrite d0/d2/d2d/d31/d89/d1c/d3f/f110 [0,4194304] 0 2024-04-24T22:45:40.477 INFO:tasks.workunit.client.1.smithi192.stdout:9/776: dread d3/dd/de/d17/f35 [0,4194304] 0 2024-04-24T22:45:40.478 INFO:tasks.workunit.client.0.smithi063.stdout:9/846: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d100/d11c 0 2024-04-24T22:45:40.478 INFO:tasks.workunit.client.1.smithi192.stdout:0/686: dwrite d3/f48 [0,4194304] 0 2024-04-24T22:45:40.480 INFO:tasks.workunit.client.1.smithi192.stdout:4/664: getdents d2/db/d5d/d77/d82 0 2024-04-24T22:45:40.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/787: rename d6/d64/d21/d25/d62/d63 to d6/d64/d21/d25/d62/d63/d89/d95/d10c 22 2024-04-24T22:45:40.489 INFO:tasks.workunit.client.1.smithi192.stdout:9/777: mknod d3/d59/dbf/c10b 0 2024-04-24T22:45:40.490 INFO:tasks.workunit.client.1.smithi192.stdout:0/687: stat d3/d27/d28/d31/dbc/fe4 0 2024-04-24T22:45:40.490 INFO:tasks.workunit.client.1.smithi192.stdout:9/778: write d3/d59/d7d/d88/dc2/fd8 [1558766,119601] 0 2024-04-24T22:45:40.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/884: dwrite d1/d2/d3/f59 [0,4194304] 0 2024-04-24T22:45:40.492 INFO:tasks.workunit.client.0.smithi063.stdout:3/841: truncate da/d18/d9e/f10a 8153745 0 2024-04-24T22:45:40.492 INFO:tasks.workunit.client.1.smithi192.stdout:9/779: fsync d3/dd/de/dba/f101 0 2024-04-24T22:45:40.493 INFO:tasks.workunit.client.1.smithi192.stdout:6/613: truncate d0/d8/d29/f95 459728 0 2024-04-24T22:45:40.493 INFO:tasks.workunit.client.1.smithi192.stdout:6/614: readlink d0/d8/l57 0 2024-04-24T22:45:40.494 INFO:tasks.workunit.client.0.smithi063.stdout:5/798: mkdir d5/d11/d76/d92/d10d 0 2024-04-24T22:45:40.494 INFO:tasks.workunit.client.1.smithi192.stdout:6/615: write d0/d8/d3d/f9c [4787571,26195] 0 2024-04-24T22:45:40.496 INFO:tasks.workunit.client.0.smithi063.stdout:2/940: symlink d0/d2/d2d/d31/d89/d7/d86/da4/l123 0 2024-04-24T22:45:40.497 INFO:tasks.workunit.client.0.smithi063.stdout:2/941: stat d0/dd/dcd/ld6 0 2024-04-24T22:45:40.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/942: chown d0/d2/d2d/d31/d89/d1c/d3f/c8f 640 1 2024-04-24T22:45:40.500 INFO:tasks.workunit.client.0.smithi063.stdout:2/943: fsync d0/d2/d2d/d31/d89/d7/d86/d8e/fbe 0 2024-04-24T22:45:40.500 INFO:tasks.workunit.client.1.smithi192.stdout:9/780: unlink d3/dd/lc7 0 2024-04-24T22:45:40.501 INFO:tasks.workunit.client.0.smithi063.stdout:0/788: fdatasync d6/d64/d24/d42/dd4/f96 0 2024-04-24T22:45:40.503 INFO:tasks.workunit.client.0.smithi063.stdout:4/816: truncate d2/fd 2907845 0 2024-04-24T22:45:40.505 INFO:tasks.workunit.client.1.smithi192.stdout:6/616: dwrite d0/d8/d3d/f49 [0,4194304] 0 2024-04-24T22:45:40.507 INFO:tasks.workunit.client.1.smithi192.stdout:0/688: dwrite d3/d27/d28/dcc/d5c/d80/fe0 [0,4194304] 0 2024-04-24T22:45:40.507 INFO:tasks.workunit.client.0.smithi063.stdout:4/817: write d2/d19/d82/d96/fa9 [3029672,71238] 0 2024-04-24T22:45:40.508 INFO:tasks.workunit.client.0.smithi063.stdout:3/842: mkdir da/d18/d47/d4d/d9b/d125 0 2024-04-24T22:45:40.509 INFO:tasks.workunit.client.1.smithi192.stdout:0/689: fsync d3/d27/d28/dcc/d5c/d80/d94/fbe 0 2024-04-24T22:45:40.510 INFO:tasks.workunit.client.1.smithi192.stdout:4/665: dread d2/db/d75/da1/d36/d5b/d28/f2b [0,4194304] 0 2024-04-24T22:45:40.512 INFO:tasks.workunit.client.0.smithi063.stdout:9/847: symlink d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d100/d11c/l11d 0 2024-04-24T22:45:40.512 INFO:tasks.workunit.client.0.smithi063.stdout:1/887: dread d3/db5/d21/d9f/dd5/d106/d11/f46 [0,4194304] 0 2024-04-24T22:45:40.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/902: link db/d1d/d20/d3f/d5f/c61 db/d1c/ddb/c131 0 2024-04-24T22:45:40.514 INFO:tasks.workunit.client.0.smithi063.stdout:0/789: dwrite d6/d3a/dba/f10a [0,4194304] 0 2024-04-24T22:45:40.515 INFO:tasks.workunit.client.0.smithi063.stdout:2/944: creat d0/d2/d2d/d31/d89/d48/f124 x:0 0 0 2024-04-24T22:45:40.518 INFO:tasks.workunit.client.0.smithi063.stdout:8/885: mknod d1/c11a 0 2024-04-24T22:45:40.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/749: truncate d2/db/f45 2866268 0 2024-04-24T22:45:40.520 INFO:tasks.workunit.client.1.smithi192.stdout:4/666: creat d2/db/d75/da1/d7d/d9c/fe2 x:0 0 0 2024-04-24T22:45:40.523 INFO:tasks.workunit.client.0.smithi063.stdout:6/903: dwrite db/fdf [0,4194304] 0 2024-04-24T22:45:40.524 INFO:tasks.workunit.client.0.smithi063.stdout:6/904: write db/d1d/d8d/dfc/f10c [1089912,39488] 0 2024-04-24T22:45:40.526 INFO:tasks.workunit.client.1.smithi192.stdout:0/690: mknod d3/d27/ce8 0 2024-04-24T22:45:40.527 INFO:tasks.workunit.client.1.smithi192.stdout:0/691: fdatasync d3/d27/d28/d2e/dc7/dd4/fdf 0 2024-04-24T22:45:40.528 INFO:tasks.workunit.client.0.smithi063.stdout:5/799: mkdir d5/d24/d49/d3b/dc5/d10e 0 2024-04-24T22:45:40.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/750: rmdir d2/db/d21/d42/d67/d9b/dc1 39 2024-04-24T22:45:40.529 INFO:tasks.workunit.client.1.smithi192.stdout:4/667: dwrite d2/db/d10/f11 [0,4194304] 0 2024-04-24T22:45:40.529 INFO:tasks.workunit.client.0.smithi063.stdout:4/818: truncate d2/d8/f31 1421035 0 2024-04-24T22:45:40.532 INFO:tasks.workunit.client.0.smithi063.stdout:4/819: write d2/d19/d33/dc1/d50/dc3/ff9 [564004,9952] 0 2024-04-24T22:45:40.534 INFO:tasks.workunit.client.1.smithi192.stdout:0/692: creat d3/d27/d3c/d6c/db3/fe9 x:0 0 0 2024-04-24T22:45:40.540 INFO:tasks.workunit.client.1.smithi192.stdout:7/751: creat d2/db/d21/d42/d67/da8/dc7/feb x:0 0 0 2024-04-24T22:45:40.541 INFO:tasks.workunit.client.0.smithi063.stdout:1/888: rmdir d3/db5/d21/d9f/dd5/d106/d11/d5e 39 2024-04-24T22:45:40.542 INFO:tasks.workunit.client.1.smithi192.stdout:0/693: dwrite d3/d27/d28/f20 [4194304,4194304] 0 2024-04-24T22:45:40.548 INFO:tasks.workunit.client.0.smithi063.stdout:3/843: symlink da/d18/l126 0 2024-04-24T22:45:40.549 INFO:tasks.workunit.client.1.smithi192.stdout:9/781: write d3/dd/d62/d6d/f86 [129064,7627] 0 2024-04-24T22:45:40.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/694: dread d3/d27/d28/d31/f3a [0,4194304] 0 2024-04-24T22:45:40.554 INFO:tasks.workunit.client.1.smithi192.stdout:0/695: write d3/d27/d28/dcc/f4d [7024268,39398] 0 2024-04-24T22:45:40.555 INFO:tasks.workunit.client.1.smithi192.stdout:0/696: readlink d3/l4f 0 2024-04-24T22:45:40.558 INFO:tasks.workunit.client.1.smithi192.stdout:4/668: link d2/db/d10/f11 d2/d9/d25/fe3 0 2024-04-24T22:45:40.560 INFO:tasks.workunit.client.0.smithi063.stdout:4/820: mknod d2/d19/d82/c10e 0 2024-04-24T22:45:40.561 INFO:tasks.workunit.client.1.smithi192.stdout:6/617: dwrite d0/d8/d3d/d3e/d40/d44/f73 [0,4194304] 0 2024-04-24T22:45:40.563 INFO:tasks.workunit.client.1.smithi192.stdout:6/618: stat d0/d8/d29/d4c/fce 0 2024-04-24T22:45:40.564 INFO:tasks.workunit.client.1.smithi192.stdout:9/782: rmdir d3/d59 39 2024-04-24T22:45:40.565 INFO:tasks.workunit.client.1.smithi192.stdout:9/783: stat d3/d5d/da1/d85/fb7 0 2024-04-24T22:45:40.568 INFO:tasks.workunit.client.0.smithi063.stdout:8/886: mkdir d1/d2/d11b 0 2024-04-24T22:45:40.570 INFO:tasks.workunit.client.0.smithi063.stdout:1/889: creat d3/db5/d21/d9f/daa/db7/f121 x:0 0 0 2024-04-24T22:45:40.571 INFO:tasks.workunit.client.1.smithi192.stdout:0/697: mkdir d3/d27/d28/d31/dbc/dea 0 2024-04-24T22:45:40.574 INFO:tasks.workunit.client.1.smithi192.stdout:4/669: symlink d2/db/d5d/d77/d82/le4 0 2024-04-24T22:45:40.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/800: mknod d5/d24/d36/c10f 0 2024-04-24T22:45:40.575 INFO:tasks.workunit.client.1.smithi192.stdout:4/670: stat d2/db/d5d/d77/d82/d8d 0 2024-04-24T22:45:40.576 INFO:tasks.workunit.client.1.smithi192.stdout:0/698: read d3/d27/d28/dcc/d5c/f61 [1658192,61623] 0 2024-04-24T22:45:40.577 INFO:tasks.workunit.client.0.smithi063.stdout:7/785: sync 2024-04-24T22:45:40.577 INFO:tasks.workunit.client.1.smithi192.stdout:6/619: creat d0/d8/d29/d4c/d4d/d83/fd3 x:0 0 0 2024-04-24T22:45:40.578 INFO:tasks.workunit.client.1.smithi192.stdout:4/671: truncate d2/d5c/dbb/fcd 144613 0 2024-04-24T22:45:40.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/821: mknod d2/d8/dd4/c10f 0 2024-04-24T22:45:40.578 INFO:tasks.workunit.client.1.smithi192.stdout:0/699: truncate d3/d27/d28/dcc/fbb 4567272 0 2024-04-24T22:45:40.580 INFO:tasks.workunit.client.1.smithi192.stdout:9/784: unlink d3/f71 0 2024-04-24T22:45:40.584 INFO:tasks.workunit.client.0.smithi063.stdout:5/801: mkdir d5/d24/d36/d65/d110 0 2024-04-24T22:45:40.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/620: mknod d0/d8/d3d/d3e/d7e/cd4 0 2024-04-24T22:45:40.587 INFO:tasks.workunit.client.1.smithi192.stdout:0/700: rmdir d3/d27/d28/dba 39 2024-04-24T22:45:40.588 INFO:tasks.workunit.client.0.smithi063.stdout:7/786: mkdir d0/d4/de/d6f/d2e/d55/d106 0 2024-04-24T22:45:40.590 INFO:tasks.workunit.client.0.smithi063.stdout:2/945: write d0/d2/d2d/d31/d89/d7/d86/da0/fe1 [573444,31629] 0 2024-04-24T22:45:40.590 INFO:tasks.workunit.client.0.smithi063.stdout:9/848: dwrite d6/d2f/dcd/db/d93/da0/fdf [0,4194304] 0 2024-04-24T22:45:40.592 INFO:tasks.workunit.client.0.smithi063.stdout:9/849: readlink d6/d2f/dcd/d1f/d20/de6/l117 0 2024-04-24T22:45:40.594 INFO:tasks.workunit.client.1.smithi192.stdout:7/752: dread d2/db/f20 [4194304,4194304] 0 2024-04-24T22:45:40.595 INFO:tasks.workunit.client.0.smithi063.stdout:9/850: fdatasync d6/d2f/dcd/d1f/ddb/ff8 0 2024-04-24T22:45:40.598 INFO:tasks.workunit.client.1.smithi192.stdout:6/621: write d0/d8/d3d/d3e/d7e/f8e [4041595,46564] 0 2024-04-24T22:45:40.598 INFO:tasks.workunit.client.0.smithi063.stdout:1/890: dread d3/db5/d21/d9f/daa/db7/f73 [0,4194304] 0 2024-04-24T22:45:40.599 INFO:tasks.workunit.client.0.smithi063.stdout:0/790: dwrite d6/d64/d21/d25/d62/d63/d89/d95/da4/f8f [0,4194304] 0 2024-04-24T22:45:40.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/786: sync 2024-04-24T22:45:40.601 INFO:tasks.workunit.client.1.smithi192.stdout:3/706: sync 2024-04-24T22:45:40.605 INFO:tasks.workunit.client.1.smithi192.stdout:4/672: rmdir d2/db/d75/dd3 0 2024-04-24T22:45:40.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/905: dwrite db/d1d/d20/d3f/d5f/d93/f54 [0,4194304] 0 2024-04-24T22:45:40.608 INFO:tasks.workunit.client.1.smithi192.stdout:7/753: fsync d2/db/d21/d42/f4d 0 2024-04-24T22:45:40.609 INFO:tasks.workunit.client.0.smithi063.stdout:6/906: read db/d1d/d20/f2b [4289607,92436] 0 2024-04-24T22:45:40.609 INFO:tasks.workunit.client.0.smithi063.stdout:3/844: dread da/f15 [0,4194304] 0 2024-04-24T22:45:40.613 INFO:tasks.workunit.client.1.smithi192.stdout:9/785: dread d3/fb [0,4194304] 0 2024-04-24T22:45:40.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/851: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d11e 0 2024-04-24T22:45:40.616 INFO:tasks.workunit.client.0.smithi063.stdout:9/852: truncate d6/d2f/dcd/d1f/ddb/ff5 528746 0 2024-04-24T22:45:40.619 INFO:tasks.workunit.client.1.smithi192.stdout:3/707: symlink d2/d27/dc6/da0/le1 0 2024-04-24T22:45:40.622 INFO:tasks.workunit.client.1.smithi192.stdout:4/673: symlink d2/d5c/dce/le5 0 2024-04-24T22:45:40.629 INFO:tasks.workunit.client.0.smithi063.stdout:4/822: write d2/d8/dd4/dda/fd3 [1093745,7606] 0 2024-04-24T22:45:40.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/887: dwrite d1/d2/d3/f56 [0,4194304] 0 2024-04-24T22:45:40.631 INFO:tasks.workunit.client.0.smithi063.stdout:3/845: creat da/d18/d47/d4d/d9b/dde/df9/f127 x:0 0 0 2024-04-24T22:45:40.631 INFO:tasks.workunit.client.0.smithi063.stdout:5/802: write d5/d24/d49/d3b/f45 [1473588,17943] 0 2024-04-24T22:45:40.633 INFO:tasks.workunit.client.1.smithi192.stdout:2/787: mknod d4/d9/d10/d6c/c100 0 2024-04-24T22:45:40.635 INFO:tasks.workunit.client.1.smithi192.stdout:3/708: unlink d2/d4/d12/d39/fd6 0 2024-04-24T22:45:40.636 INFO:tasks.workunit.client.0.smithi063.stdout:0/791: mknod d6/d64/da0/d33/c10d 0 2024-04-24T22:45:40.637 INFO:tasks.workunit.client.1.smithi192.stdout:4/674: dread - d2/db/d75/da1/d53/fc5 zero size 2024-04-24T22:45:40.637 INFO:tasks.workunit.client.0.smithi063.stdout:2/946: dwrite d0/d2/d2d/d31/d89/d7/d86/fa2 [0,4194304] 0 2024-04-24T22:45:40.638 INFO:tasks.workunit.client.0.smithi063.stdout:2/947: chown d0/d2/d2d/f81 2268831 1 2024-04-24T22:45:40.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/754: fsync d2/db/d21/d42/f8c 0 2024-04-24T22:45:40.641 INFO:tasks.workunit.client.1.smithi192.stdout:0/701: getdents d3/d27/d28/dcc/d5c/d80/d83 0 2024-04-24T22:45:40.641 INFO:tasks.workunit.client.0.smithi063.stdout:4/823: symlink d2/d80/d8d/l110 0 2024-04-24T22:45:40.645 INFO:tasks.workunit.client.0.smithi063.stdout:8/888: unlink d1/d2/l28 0 2024-04-24T22:45:40.648 INFO:tasks.workunit.client.1.smithi192.stdout:0/702: dwrite d3/d27/d3c/d6c/db3/fe9 [0,4194304] 0 2024-04-24T22:45:40.648 INFO:tasks.workunit.client.0.smithi063.stdout:5/803: write d5/de5/fed [519990,98645] 0 2024-04-24T22:45:40.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/675: rename d2/db/d75/da1/d7d/f9b to d2/db/d75/da1/d36/d5b/d28/d2f/fe6 0 2024-04-24T22:45:40.649 INFO:tasks.workunit.client.0.smithi063.stdout:2/948: dwrite d0/d2/d2d/d31/d89/d7/d86/d8e/fbe [0,4194304] 0 2024-04-24T22:45:40.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/755: mkdir d2/db/d21/d42/d67/da8/dc7/dec 0 2024-04-24T22:45:40.653 INFO:tasks.workunit.client.1.smithi192.stdout:4/676: dread d2/d9/fc8 [0,4194304] 0 2024-04-24T22:45:40.654 INFO:tasks.workunit.client.0.smithi063.stdout:3/846: fdatasync da/f14 0 2024-04-24T22:45:40.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/824: dwrite d2/d8/d40/d51/f66 [0,4194304] 0 2024-04-24T22:45:40.656 INFO:tasks.workunit.client.0.smithi063.stdout:8/889: dread d1/d11/d8b/dd6/ff0 [4194304,4194304] 0 2024-04-24T22:45:40.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/703: rename d3/d27/d28/d2e/d3f/dd7 to d3/d27/d6a/deb 0 2024-04-24T22:45:40.660 INFO:tasks.workunit.client.0.smithi063.stdout:9/853: link d6/d2f/dcd/db/d1c/d78/d9a/daa/fab d6/d2f/dcd/db/d1c/d78/d9a/f11f 0 2024-04-24T22:45:40.661 INFO:tasks.workunit.client.0.smithi063.stdout:0/792: dwrite d6/d64/d21/d9c/f9d [0,4194304] 0 2024-04-24T22:45:40.661 INFO:tasks.workunit.client.1.smithi192.stdout:7/756: chown d2/l43 19404 1 2024-04-24T22:45:40.663 INFO:tasks.workunit.client.0.smithi063.stdout:2/949: symlink d0/d2/d2d/d31/d89/d7/d86/da4/l125 0 2024-04-24T22:45:40.665 INFO:tasks.workunit.client.0.smithi063.stdout:5/804: getdents d5/d24/d44/dc9 0 2024-04-24T22:45:40.665 INFO:tasks.workunit.client.0.smithi063.stdout:0/793: fdatasync d6/d64/d21/d25/d31/fdd 0 2024-04-24T22:45:40.665 INFO:tasks.workunit.client.0.smithi063.stdout:8/890: write d1/d22/d33/da6/d54/fad [2965357,93987] 0 2024-04-24T22:45:40.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/704: rename d3/d27/d28/d31/l38 to d3/d27/d6a/lec 0 2024-04-24T22:45:40.667 INFO:tasks.workunit.client.0.smithi063.stdout:4/825: mkdir d2/d8/d3c/d111 0 2024-04-24T22:45:40.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/788: getdents d4/d9/d10/da7/df5 0 2024-04-24T22:45:40.668 INFO:tasks.workunit.client.0.smithi063.stdout:3/847: dwrite da/d18/d47/d4d/d9b/dde/df9/f127 [0,4194304] 0 2024-04-24T22:45:40.670 INFO:tasks.workunit.client.1.smithi192.stdout:7/757: mknod d2/db/d21/d42/d67/da8/ced 0 2024-04-24T22:45:40.671 INFO:tasks.workunit.client.1.smithi192.stdout:4/677: symlink d2/db/d5d/le7 0 2024-04-24T22:45:40.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/805: creat d5/d24/db6/f111 x:0 0 0 2024-04-24T22:45:40.675 INFO:tasks.workunit.client.1.smithi192.stdout:4/678: mkdir d2/d5c/dbb/de8 0 2024-04-24T22:45:40.676 INFO:tasks.workunit.client.0.smithi063.stdout:6/907: sync 2024-04-24T22:45:40.678 INFO:tasks.workunit.client.0.smithi063.stdout:0/794: truncate d6/da/d2b/f40 2954325 0 2024-04-24T22:45:40.680 INFO:tasks.workunit.client.1.smithi192.stdout:4/679: mknod d2/db/d5d/d77/d82/ce9 0 2024-04-24T22:45:40.681 INFO:tasks.workunit.client.0.smithi063.stdout:5/806: creat d5/d24/d49/d3b/dc5/dda/f112 x:0 0 0 2024-04-24T22:45:40.682 INFO:tasks.workunit.client.1.smithi192.stdout:7/758: dread d2/db/f6f [0,4194304] 0 2024-04-24T22:45:40.684 INFO:tasks.workunit.client.1.smithi192.stdout:4/680: symlink d2/d9/lea 0 2024-04-24T22:45:40.685 INFO:tasks.workunit.client.1.smithi192.stdout:7/759: symlink d2/db/d2a/d57/lee 0 2024-04-24T22:45:40.685 INFO:tasks.workunit.client.1.smithi192.stdout:7/760: chown d2/db/d21/d42/l4a 72 1 2024-04-24T22:45:40.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/681: mkdir d2/db/d75/da1/d36/d5b/d28/deb 0 2024-04-24T22:45:40.688 INFO:tasks.workunit.client.1.smithi192.stdout:7/761: write d2/db/d21/f22 [12835792,40649] 0 2024-04-24T22:45:40.689 INFO:tasks.workunit.client.1.smithi192.stdout:4/682: write d2/db/d75/da1/d36/d5b/f24 [1236801,12257] 0 2024-04-24T22:45:40.691 INFO:tasks.workunit.client.0.smithi063.stdout:3/848: getdents da/d39/d94/dbc 0 2024-04-24T22:45:40.693 INFO:tasks.workunit.client.0.smithi063.stdout:6/908: link db/d1d/d20/d29/cbb db/d1d/d20/c132 0 2024-04-24T22:45:40.694 INFO:tasks.workunit.client.1.smithi192.stdout:4/683: creat d2/db/d75/da1/d36/d5b/d28/fec x:0 0 0 2024-04-24T22:45:40.694 INFO:tasks.workunit.client.1.smithi192.stdout:4/684: stat d2/d5c/dce 0 2024-04-24T22:45:40.695 INFO:tasks.workunit.client.0.smithi063.stdout:9/854: dread d6/d2f/d42/d81/fa9 [0,4194304] 0 2024-04-24T22:45:40.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/762: mkdir d2/db/d21/d42/d5b/def 0 2024-04-24T22:45:40.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/849: creat da/d40/f128 x:0 0 0 2024-04-24T22:45:40.696 INFO:tasks.workunit.client.0.smithi063.stdout:8/891: dread d1/d11/f29 [0,4194304] 0 2024-04-24T22:45:40.697 INFO:tasks.workunit.client.1.smithi192.stdout:4/685: truncate d2/db/d75/da1/d36/d5b/d28/fec 44195 0 2024-04-24T22:45:40.697 INFO:tasks.workunit.client.0.smithi063.stdout:6/909: write db/d1d/d20/d3e/f4f [1660717,1298] 0 2024-04-24T22:45:40.697 INFO:tasks.workunit.client.0.smithi063.stdout:9/855: creat d6/d2f/dcd/d8a/f120 x:0 0 0 2024-04-24T22:45:40.698 INFO:tasks.workunit.client.1.smithi192.stdout:7/763: read - d2/db/d21/d42/d67/d9b/f99 zero size 2024-04-24T22:45:40.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/686: symlink d2/db/d10/led 0 2024-04-24T22:45:40.699 INFO:tasks.workunit.client.1.smithi192.stdout:4/687: fdatasync d2/db/d10/fb6 0 2024-04-24T22:45:40.700 INFO:tasks.workunit.client.0.smithi063.stdout:9/856: read d6/d2f/d42/d81/fc6 [2731218,41453] 0 2024-04-24T22:45:40.700 INFO:tasks.workunit.client.1.smithi192.stdout:7/764: creat d2/db/d36/ff0 x:0 0 0 2024-04-24T22:45:40.702 INFO:tasks.workunit.client.1.smithi192.stdout:4/688: symlink d2/db/d75/da1/d36/d5b/d28/lee 0 2024-04-24T22:45:40.705 INFO:tasks.workunit.client.0.smithi063.stdout:6/910: dwrite db/d1d/d20/d84/fae [0,4194304] 0 2024-04-24T22:45:40.706 INFO:tasks.workunit.client.1.smithi192.stdout:7/765: rename d2/db/d21/d42/d67/d9b/d6e/db5/db6/ccf to d2/db/d21/d42/d67/d9b/d6e/d77/d8e/cf1 0 2024-04-24T22:45:40.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/766: stat d2/db/d2a/f3c 0 2024-04-24T22:45:40.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/911: mkdir db/d1d/d20/d84/d100/d133 0 2024-04-24T22:45:40.710 INFO:tasks.workunit.client.0.smithi063.stdout:0/795: dread d6/d64/d21/d25/d31/f35 [0,4194304] 0 2024-04-24T22:45:40.710 INFO:tasks.workunit.client.0.smithi063.stdout:8/892: link d1/d11/d32/d38/c5a d1/d22/d33/da6/d54/d7b/c11c 0 2024-04-24T22:45:40.712 INFO:tasks.workunit.client.0.smithi063.stdout:9/857: getdents d6/d2f/dcd/db/d1c 0 2024-04-24T22:45:40.713 INFO:tasks.workunit.client.0.smithi063.stdout:8/893: dread - d1/d11/dcb/f105 zero size 2024-04-24T22:45:40.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/850: dread da/f28 [0,4194304] 0 2024-04-24T22:45:40.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/851: fdatasync da/d40/d111/f113 0 2024-04-24T22:45:40.717 INFO:tasks.workunit.client.0.smithi063.stdout:0/796: rename d6/d64/c70 to d6/d64/d21/d25/d31/c10e 0 2024-04-24T22:45:40.718 INFO:tasks.workunit.client.0.smithi063.stdout:0/797: chown d6/d64/d21/d25/d62/d63/d89/d95/da4/ff9 2549296 1 2024-04-24T22:45:40.719 INFO:tasks.workunit.client.0.smithi063.stdout:8/894: mkdir d1/d11/d8b/dd6/d11d 0 2024-04-24T22:45:40.720 INFO:tasks.workunit.client.0.smithi063.stdout:9/858: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/f121 x:0 0 0 2024-04-24T22:45:40.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/852: mknod da/d39/c129 0 2024-04-24T22:45:40.722 INFO:tasks.workunit.client.1.smithi192.stdout:4/689: dread d2/d5c/fbf [0,4194304] 0 2024-04-24T22:45:40.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/895: rmdir d1/d11/d32/d38/db7/dc1 39 2024-04-24T22:45:40.725 INFO:tasks.workunit.client.1.smithi192.stdout:4/690: symlink d2/d5c/lef 0 2024-04-24T22:45:40.726 INFO:tasks.workunit.client.0.smithi063.stdout:8/896: creat d1/d2/d3/d86/f11e x:0 0 0 2024-04-24T22:45:40.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/691: write d2/db/d10/fb6 [509781,26966] 0 2024-04-24T22:45:40.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/692: chown d2/d5c/fbf 157 1 2024-04-24T22:45:40.728 INFO:tasks.workunit.client.0.smithi063.stdout:9/859: dwrite d6/d2f/dcd/d53/f7a [0,4194304] 0 2024-04-24T22:45:40.729 INFO:tasks.workunit.client.1.smithi192.stdout:4/693: mkdir d2/db/d75/da1/dc9/df0 0 2024-04-24T22:45:40.730 INFO:tasks.workunit.client.0.smithi063.stdout:8/897: link d1/l10d d1/d11/d8b/l11f 0 2024-04-24T22:45:40.732 INFO:tasks.workunit.client.1.smithi192.stdout:4/694: dread d2/f1f [0,4194304] 0 2024-04-24T22:45:40.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/860: dwrite d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/fe7 [0,4194304] 0 2024-04-24T22:45:40.738 INFO:tasks.workunit.client.1.smithi192.stdout:4/695: dread d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:40.739 INFO:tasks.workunit.client.0.smithi063.stdout:8/898: dwrite d1/d21/fb3 [0,4194304] 0 2024-04-24T22:45:40.739 INFO:tasks.workunit.client.1.smithi192.stdout:4/696: write d2/fd0 [1099356,99607] 0 2024-04-24T22:45:40.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/697: symlink d2/d5c/lf1 0 2024-04-24T22:45:40.741 INFO:tasks.workunit.client.0.smithi063.stdout:9/861: unlink d6/d2f/dcd/d1f/d20/d2a/dcb/f33 0 2024-04-24T22:45:40.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/698: read d2/db/d75/da1/d53/fcc [1348843,110744] 0 2024-04-24T22:45:40.745 INFO:tasks.workunit.client.0.smithi063.stdout:9/862: mkdir d6/d2f/dcd/db/d6b/d4f/dad/d48/d122 0 2024-04-24T22:45:40.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/699: write d2/db/d5d/d77/f92 [6894985,111070] 0 2024-04-24T22:45:40.749 INFO:tasks.workunit.client.1.smithi192.stdout:4/700: mkdir d2/db/d75/da1/d36/d5b/d28/d9f/df2 0 2024-04-24T22:45:40.750 INFO:tasks.workunit.client.1.smithi192.stdout:4/701: mknod d2/d9/d6b/cf3 0 2024-04-24T22:45:40.751 INFO:tasks.workunit.client.1.smithi192.stdout:4/702: readlink d2/db/d5d/d77/d82/le4 0 2024-04-24T22:45:40.761 INFO:tasks.workunit.client.0.smithi063.stdout:8/899: dread d1/d11/d32/d38/f67 [0,4194304] 0 2024-04-24T22:45:40.762 INFO:tasks.workunit.client.0.smithi063.stdout:8/900: chown d1/d22/d33/da6/d54/d79 40 1 2024-04-24T22:45:40.765 INFO:tasks.workunit.client.0.smithi063.stdout:8/901: unlink d1/d22/d33/da6/d54/l9c 0 2024-04-24T22:45:40.765 INFO:tasks.workunit.client.0.smithi063.stdout:8/902: chown d1 5307 1 2024-04-24T22:45:40.798 INFO:tasks.workunit.client.0.smithi063.stdout:1/891: write d3/db5/d21/d9f/daa/db7/f73 [1734965,12054] 0 2024-04-24T22:45:40.802 INFO:tasks.workunit.client.1.smithi192.stdout:6/622: dwrite d0/d8/d29/d5a/d63/f81 [0,4194304] 0 2024-04-24T22:45:40.806 INFO:tasks.workunit.client.0.smithi063.stdout:1/892: link d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f113 d3/db5/d21/d9f/daa/d108/f122 0 2024-04-24T22:45:40.808 INFO:tasks.workunit.client.1.smithi192.stdout:6/623: symlink d0/d8/d22/dcb/ld5 0 2024-04-24T22:45:40.813 INFO:tasks.workunit.client.1.smithi192.stdout:6/624: getdents d0/d8/d3d/d3e 0 2024-04-24T22:45:40.814 INFO:tasks.workunit.client.1.smithi192.stdout:6/625: write d0/d8/d29/d4c/d4d/d83/fd3 [952708,100428] 0 2024-04-24T22:45:40.815 INFO:tasks.workunit.client.0.smithi063.stdout:2/950: dread d0/d2/d2d/ff7 [0,4194304] 0 2024-04-24T22:45:40.816 INFO:tasks.workunit.client.1.smithi192.stdout:6/626: mkdir d0/d8/d3d/d9d/dd6 0 2024-04-24T22:45:40.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/627: rmdir d0/d8/d29/d58 39 2024-04-24T22:45:40.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/628: rename d0/d8/d29/d4c/fc5 to d0/d8/d29/d4c/d4d/d83/fd7 0 2024-04-24T22:45:40.824 INFO:tasks.workunit.client.1.smithi192.stdout:6/629: symlink d0/d8/d22/ld8 0 2024-04-24T22:45:40.829 INFO:tasks.workunit.client.0.smithi063.stdout:7/787: truncate d0/d4/de/d17/d28/d31/f6a 2887093 0 2024-04-24T22:45:40.832 INFO:tasks.workunit.client.0.smithi063.stdout:7/788: fdatasync d0/d4/de/dfc/ff7 0 2024-04-24T22:45:40.832 INFO:tasks.workunit.client.0.smithi063.stdout:9/863: fsync d6/d2f/dcd/d53/f7a 0 2024-04-24T22:45:40.834 INFO:tasks.workunit.client.0.smithi063.stdout:7/789: creat d0/d2f/f107 x:0 0 0 2024-04-24T22:45:40.836 INFO:tasks.workunit.client.0.smithi063.stdout:9/864: truncate d6/d2f/dcd/db/d6b/f46 1776799 0 2024-04-24T22:45:40.836 INFO:tasks.workunit.client.0.smithi063.stdout:7/790: write d0/d4/de/d17/d35/d69/db1/fd3 [4896390,43795] 0 2024-04-24T22:45:40.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/865: rmdir d6/d2f/dcd/db/d6b/d4f/d59 39 2024-04-24T22:45:40.840 INFO:tasks.workunit.client.0.smithi063.stdout:7/791: symlink d0/d4/de/d17/d35/d6c/d7a/l108 0 2024-04-24T22:45:40.841 INFO:tasks.workunit.client.1.smithi192.stdout:3/709: dwrite d2/f29 [0,4194304] 0 2024-04-24T22:45:40.845 INFO:tasks.workunit.client.0.smithi063.stdout:1/893: dread d3/da/d34/f9a [0,4194304] 0 2024-04-24T22:45:40.850 INFO:tasks.workunit.client.0.smithi063.stdout:9/866: dwrite d6/d2f/d42/d81/fa9 [4194304,4194304] 0 2024-04-24T22:45:40.850 INFO:tasks.workunit.client.0.smithi063.stdout:1/894: truncate d3/db5/d21/d9f/daa/feb 1154026 0 2024-04-24T22:45:40.850 INFO:tasks.workunit.client.0.smithi063.stdout:9/867: fsync d6/d2f/dcd/db/d6b/d4f/dad/f105 0 2024-04-24T22:45:40.850 INFO:tasks.workunit.client.1.smithi192.stdout:3/710: rename d2/d4/d12/f49 to d2/d4/d12/d1a/d1d/d55/d5f/d9b/fe2 0 2024-04-24T22:45:40.853 INFO:tasks.workunit.client.0.smithi063.stdout:1/895: fdatasync d3/da/d34/d99/fe5 0 2024-04-24T22:45:40.853 INFO:tasks.workunit.client.1.smithi192.stdout:3/711: symlink d2/d27/dc6/da0/le3 0 2024-04-24T22:45:40.854 INFO:tasks.workunit.client.0.smithi063.stdout:1/896: write d3/db5/d21/d9f/dd5/d106/df5/f103 [833858,67586] 0 2024-04-24T22:45:40.858 INFO:tasks.workunit.client.0.smithi063.stdout:9/868: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d123 0 2024-04-24T22:45:40.860 INFO:tasks.workunit.client.0.smithi063.stdout:1/897: creat d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f123 x:0 0 0 2024-04-24T22:45:40.864 INFO:tasks.workunit.client.1.smithi192.stdout:3/712: getdents d2/d4/d12/d1a 0 2024-04-24T22:45:40.865 INFO:tasks.workunit.client.0.smithi063.stdout:4/826: write d2/d8/d5c/fe2 [486566,22906] 0 2024-04-24T22:45:40.869 INFO:tasks.workunit.client.0.smithi063.stdout:9/869: dwrite d6/dbd/dcf/fec [0,4194304] 0 2024-04-24T22:45:40.869 INFO:tasks.workunit.client.1.smithi192.stdout:3/713: creat d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/fe4 x:0 0 0 2024-04-24T22:45:40.870 INFO:tasks.workunit.client.0.smithi063.stdout:1/898: dwrite d3/db5/d21/d9f/dd5/f102 [0,4194304] 0 2024-04-24T22:45:40.871 INFO:tasks.workunit.client.0.smithi063.stdout:9/870: fdatasync d6/d2f/dcd/db/d6b/d4f/dad/d36/fe5 0 2024-04-24T22:45:40.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/714: symlink d2/d4/d4e/le5 0 2024-04-24T22:45:40.875 INFO:tasks.workunit.client.1.smithi192.stdout:3/715: creat d2/d27/dc6/d41/dac/fe6 x:0 0 0 2024-04-24T22:45:40.877 INFO:tasks.workunit.client.0.smithi063.stdout:9/871: fsync d6/d2f/dcd/db/d1c/db3/fb6 0 2024-04-24T22:45:40.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/872: dread - d6/d2f/dcd/db/d6b/d4f/dad/f105 zero size 2024-04-24T22:45:40.880 INFO:tasks.workunit.client.0.smithi063.stdout:4/827: creat d2/d8/dd4/dda/f112 x:0 0 0 2024-04-24T22:45:40.882 INFO:tasks.workunit.client.0.smithi063.stdout:5/807: write d5/f6f [1159602,96265] 0 2024-04-24T22:45:40.884 INFO:tasks.workunit.client.0.smithi063.stdout:5/808: write d5/d11/d23/d2d/ff9 [240012,61482] 0 2024-04-24T22:45:40.884 INFO:tasks.workunit.client.0.smithi063.stdout:9/873: dwrite d6/d2f/dcd/f108 [0,4194304] 0 2024-04-24T22:45:40.885 INFO:tasks.workunit.client.1.smithi192.stdout:3/716: dwrite d2/d4/d12/fd5 [0,4194304] 0 2024-04-24T22:45:40.886 INFO:tasks.workunit.client.0.smithi063.stdout:1/899: rename d3/db5/d21/d9f/dd5/d106/d11/f89 to d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/ddf/f124 0 2024-04-24T22:45:40.888 INFO:tasks.workunit.client.1.smithi192.stdout:3/717: fsync d2/d4/d4e/f8f 0 2024-04-24T22:45:40.888 INFO:tasks.workunit.client.1.smithi192.stdout:3/718: fsync d2/d27/d28/dab/dca/fcf 0 2024-04-24T22:45:40.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/828: creat d2/d80/d81/f113 x:0 0 0 2024-04-24T22:45:40.892 INFO:tasks.workunit.client.0.smithi063.stdout:4/829: write d2/d80/fb0 [1295328,106391] 0 2024-04-24T22:45:40.896 INFO:tasks.workunit.client.1.smithi192.stdout:3/719: creat d2/d27/fe7 x:0 0 0 2024-04-24T22:45:40.901 INFO:tasks.workunit.client.0.smithi063.stdout:9/874: rename d6/d2f/dcd/db/d93/ff9 to d6/d2f/dcd/db/d6b/d4f/dad/d48/f124 0 2024-04-24T22:45:40.902 INFO:tasks.workunit.client.1.smithi192.stdout:3/720: truncate d2/d27/f6f 889699 0 2024-04-24T22:45:40.903 INFO:tasks.workunit.client.0.smithi063.stdout:9/875: fdatasync d6/d2f/dcd/d53/dd5/fe1 0 2024-04-24T22:45:40.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/830: symlink d2/d19/d33/d9d/l114 0 2024-04-24T22:45:40.908 INFO:tasks.workunit.client.0.smithi063.stdout:5/809: mknod d5/d11/d76/c113 0 2024-04-24T22:45:40.909 INFO:tasks.workunit.client.0.smithi063.stdout:6/912: write db/d1c/fbf [438761,54727] 0 2024-04-24T22:45:40.911 INFO:tasks.workunit.client.0.smithi063.stdout:9/876: dwrite d6/d2f/d42/d81/fc2 [0,4194304] 0 2024-04-24T22:45:40.913 INFO:tasks.workunit.client.0.smithi063.stdout:1/900: dread d3/da/d34/d68/fa8 [0,4194304] 0 2024-04-24T22:45:40.914 INFO:tasks.workunit.client.1.smithi192.stdout:7/767: write d2/f1a [2117547,68833] 0 2024-04-24T22:45:40.918 INFO:tasks.workunit.client.0.smithi063.stdout:3/853: write da/fb4 [760319,31127] 0 2024-04-24T22:45:40.920 INFO:tasks.workunit.client.0.smithi063.stdout:6/913: dread db/d1d/d20/d29/f2e [0,4194304] 0 2024-04-24T22:45:40.923 INFO:tasks.workunit.client.0.smithi063.stdout:6/914: readlink db/d1d/d20/d29/d5d/df5/l128 0 2024-04-24T22:45:40.924 INFO:tasks.workunit.client.0.smithi063.stdout:0/798: dwrite d6/da/f3d [0,4194304] 0 2024-04-24T22:45:40.924 INFO:tasks.workunit.client.0.smithi063.stdout:5/810: dread d5/d11/d18/f19 [0,4194304] 0 2024-04-24T22:45:40.925 INFO:tasks.workunit.client.0.smithi063.stdout:1/901: write d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f113 [3714753,30631] 0 2024-04-24T22:45:40.929 INFO:tasks.workunit.client.0.smithi063.stdout:3/854: dwrite da/f1a [4194304,4194304] 0 2024-04-24T22:45:40.958 INFO:tasks.workunit.client.0.smithi063.stdout:9/877: rename d6/d2f/dcd/d1f/d20/c8d to d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d107/c125 0 2024-04-24T22:45:40.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/831: getdents d2/d19/d33/dc1/d109 0 2024-04-24T22:45:40.962 INFO:tasks.workunit.client.0.smithi063.stdout:6/915: rmdir db/d1d/d8d/dce 39 2024-04-24T22:45:40.969 INFO:tasks.workunit.client.1.smithi192.stdout:4/703: dwrite d2/d9/d25/f32 [0,4194304] 0 2024-04-24T22:45:40.971 INFO:tasks.workunit.client.0.smithi063.stdout:8/903: dwrite d1/d22/d33/da6/d54/f8e [0,4194304] 0 2024-04-24T22:45:40.977 INFO:tasks.workunit.client.0.smithi063.stdout:1/902: truncate d3/db5/d21/d9f/dd5/d106/d39/dbf/fdc 825571 0 2024-04-24T22:45:40.979 INFO:tasks.workunit.client.1.smithi192.stdout:6/630: write d0/d8/d29/d4c/d4d/f9e [226477,44018] 0 2024-04-24T22:45:40.981 INFO:tasks.workunit.client.1.smithi192.stdout:6/631: write d0/d8/d29/d4c/f6d [495097,40515] 0 2024-04-24T22:45:40.982 INFO:tasks.workunit.client.0.smithi063.stdout:3/855: rename da/d18/d47/fe9 to da/d18/d115/f12a 0 2024-04-24T22:45:40.982 INFO:tasks.workunit.client.0.smithi063.stdout:2/951: dwrite d0/fa6 [0,4194304] 0 2024-04-24T22:45:40.984 INFO:tasks.workunit.client.0.smithi063.stdout:2/952: readlink d0/d2/d39/d11a/l10c 0 2024-04-24T22:45:40.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/792: truncate d0/d4/de/d17/d35/d69/db1/fd3 4640278 0 2024-04-24T22:45:40.993 INFO:tasks.workunit.client.0.smithi063.stdout:4/832: dread d2/d19/d33/dc1/f6d [0,4194304] 0 2024-04-24T22:45:40.997 INFO:tasks.workunit.client.0.smithi063.stdout:8/904: chown d1/d22/d33/c80 3 1 2024-04-24T22:45:40.999 INFO:tasks.workunit.client.0.smithi063.stdout:1/903: creat d3/db5/d21/d49/f125 x:0 0 0 2024-04-24T22:45:41.002 INFO:tasks.workunit.client.0.smithi063.stdout:3/856: stat da/d18/d3a/ca6 0 2024-04-24T22:45:41.003 INFO:tasks.workunit.client.0.smithi063.stdout:8/905: dread d1/d22/fab [0,4194304] 0 2024-04-24T22:45:41.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/916: dread db/d1c/f89 [0,4194304] 0 2024-04-24T22:45:41.006 INFO:tasks.workunit.client.0.smithi063.stdout:2/953: symlink d0/d2/d2f/l126 0 2024-04-24T22:45:41.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/917: chown db/d1d/d20/d3f 1 1 2024-04-24T22:45:41.013 INFO:tasks.workunit.client.0.smithi063.stdout:1/904: creat d3/db5/d21/d9f/daa/f126 x:0 0 0 2024-04-24T22:45:41.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/768: rmdir d2/db/d21/d42/d5b 39 2024-04-24T22:45:41.023 INFO:tasks.workunit.client.0.smithi063.stdout:6/918: rmdir db/d1d/d20/d29/d5d 39 2024-04-24T22:45:41.027 INFO:tasks.workunit.client.1.smithi192.stdout:4/704: symlink d2/db/d75/da1/dc9/df0/lf4 0 2024-04-24T22:45:41.028 INFO:tasks.workunit.client.1.smithi192.stdout:6/632: getdents d0/d8/d3d/d9d/dd6 0 2024-04-24T22:45:41.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/769: mknod d2/db/d21/dd5/cf2 0 2024-04-24T22:45:41.031 INFO:tasks.workunit.client.1.smithi192.stdout:7/770: stat d2/db/l1b 0 2024-04-24T22:45:41.033 INFO:tasks.workunit.client.0.smithi063.stdout:6/919: rename db/d1d/d2d/f36 to db/d1d/d20/d29/dab/dd0/d105/f134 0 2024-04-24T22:45:41.034 INFO:tasks.workunit.client.1.smithi192.stdout:4/705: creat d2/db/d75/da1/d36/d5b/d28/d2f/ff5 x:0 0 0 2024-04-24T22:45:41.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/920: truncate db/fdf 5819306 0 2024-04-24T22:45:41.035 INFO:tasks.workunit.client.1.smithi192.stdout:4/706: write d2/db/f7a [5115415,50099] 0 2024-04-24T22:45:41.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/799: dwrite d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:41.040 INFO:tasks.workunit.client.0.smithi063.stdout:2/954: getdents d0/d2/d2d/d31/d89/d1c/d3f 0 2024-04-24T22:45:41.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/811: dwrite d5/d11/d23/f69 [0,4194304] 0 2024-04-24T22:45:41.042 INFO:tasks.workunit.client.1.smithi192.stdout:4/707: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/ff5 [0,4194304] 0 2024-04-24T22:45:41.043 INFO:tasks.workunit.client.0.smithi063.stdout:5/812: write d5/d24/d49/d3b/f10b [185173,71910] 0 2024-04-24T22:45:41.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/633: creat d0/d8/d18/d2e/fd9 x:0 0 0 2024-04-24T22:45:41.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/800: creat d6/d64/d21/d25/d62/f10f x:0 0 0 2024-04-24T22:45:41.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/771: mkdir d2/db/d25/dda/df3 0 2024-04-24T22:45:41.053 INFO:tasks.workunit.client.1.smithi192.stdout:4/708: mkdir d2/db/d75/da1/d36/d5b/d28/df6 0 2024-04-24T22:45:41.060 INFO:tasks.workunit.client.1.smithi192.stdout:4/709: creat d2/db/d75/da1/d36/d5b/d64/ff7 x:0 0 0 2024-04-24T22:45:41.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/921: mknod db/d1c/c135 0 2024-04-24T22:45:41.062 INFO:tasks.workunit.client.1.smithi192.stdout:4/710: truncate d2/db/d75/da1/d36/d5b/d28/fec 563425 0 2024-04-24T22:45:41.069 INFO:tasks.workunit.client.1.smithi192.stdout:4/711: symlink d2/db/d75/da1/dc9/df0/lf8 0 2024-04-24T22:45:41.070 INFO:tasks.workunit.client.1.smithi192.stdout:6/634: link d0/d8/d29/d4c/f74 d0/d8/d22/fda 0 2024-04-24T22:45:41.071 INFO:tasks.workunit.client.0.smithi063.stdout:9/878: truncate d6/d2f/dcd/db/d93/f109 68011 0 2024-04-24T22:45:41.073 INFO:tasks.workunit.client.0.smithi063.stdout:2/955: symlink d0/d2/d27/d100/l127 0 2024-04-24T22:45:41.074 INFO:tasks.workunit.client.1.smithi192.stdout:4/712: symlink d2/db/d5d/lf9 0 2024-04-24T22:45:41.076 INFO:tasks.workunit.client.1.smithi192.stdout:6/635: mkdir d0/d8/d3d/d9d/da5/dc8/ddb 0 2024-04-24T22:45:41.079 INFO:tasks.workunit.client.0.smithi063.stdout:5/813: rename d5/d11/d76/cf8 to d5/d11/d23/d2d/d62/d73/d95/d10a/c114 0 2024-04-24T22:45:41.079 INFO:tasks.workunit.client.1.smithi192.stdout:4/713: fdatasync d2/db/d75/da1/d36/d5b/d28/f2b 0 2024-04-24T22:45:41.081 INFO:tasks.workunit.client.0.smithi063.stdout:8/906: truncate d1/d2/d3/f46 3241873 0 2024-04-24T22:45:41.081 INFO:tasks.workunit.client.1.smithi192.stdout:3/721: dwrite d2/d27/f6f [0,4194304] 0 2024-04-24T22:45:41.081 INFO:tasks.workunit.client.0.smithi063.stdout:3/857: dwrite da/d18/f27 [0,4194304] 0 2024-04-24T22:45:41.086 INFO:tasks.workunit.client.0.smithi063.stdout:2/956: dread d0/d2/f1b [0,4194304] 0 2024-04-24T22:45:41.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/905: truncate d3/f3c 7778 0 2024-04-24T22:45:41.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/636: mkdir d0/d8/d3d/d9d/dd6/ddc 0 2024-04-24T22:45:41.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/722: rmdir d2/d4/d12/dbc 39 2024-04-24T22:45:41.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/714: creat d2/db/d5d/d77/d82/d8d/ffa x:0 0 0 2024-04-24T22:45:41.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/879: dread d6/d2f/fa4 [0,4194304] 0 2024-04-24T22:45:41.103 INFO:tasks.workunit.client.1.smithi192.stdout:6/637: chown d0/d8/d29/d5a/cd1 130038 1 2024-04-24T22:45:41.105 INFO:tasks.workunit.client.1.smithi192.stdout:3/723: creat d2/d27/dc6/d41/fe8 x:0 0 0 2024-04-24T22:45:41.107 INFO:tasks.workunit.client.1.smithi192.stdout:7/772: write d2/db/d21/d42/f8c [709424,23811] 0 2024-04-24T22:45:41.108 INFO:tasks.workunit.client.1.smithi192.stdout:4/715: write d2/d9/d25/fe3 [1996715,42554] 0 2024-04-24T22:45:41.115 INFO:tasks.workunit.client.1.smithi192.stdout:7/773: dwrite d2/f1a [0,4194304] 0 2024-04-24T22:45:41.116 INFO:tasks.workunit.client.0.smithi063.stdout:5/814: mkdir d5/d1d/d7d/d67/d115 0 2024-04-24T22:45:41.117 INFO:tasks.workunit.client.0.smithi063.stdout:5/815: read d5/d24/d44/fe4 [737737,130971] 0 2024-04-24T22:45:41.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/801: write d6/dd/f4d [750494,16338] 0 2024-04-24T22:45:41.120 INFO:tasks.workunit.client.0.smithi063.stdout:9/880: symlink d6/d2f/dcd/db/d6b/d4f/dad/l126 0 2024-04-24T22:45:41.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/638: dread d0/d8/f14 [0,4194304] 0 2024-04-24T22:45:41.128 INFO:tasks.workunit.client.1.smithi192.stdout:6/639: symlink d0/d8/d3d/d3e/d40/d44/ldd 0 2024-04-24T22:45:41.131 INFO:tasks.workunit.client.0.smithi063.stdout:9/881: mkdir d6/d2f/dcd/db/dca/d127 0 2024-04-24T22:45:41.134 INFO:tasks.workunit.client.0.smithi063.stdout:0/802: mkdir d6/d110 0 2024-04-24T22:45:41.140 INFO:tasks.workunit.client.0.smithi063.stdout:6/922: truncate db/d1d/d2d/d48/f51 156757 0 2024-04-24T22:45:41.142 INFO:tasks.workunit.client.1.smithi192.stdout:4/716: getdents d2/db/d5d/d77/d82/d8d 0 2024-04-24T22:45:41.142 INFO:tasks.workunit.client.0.smithi063.stdout:3/858: write da/d18/fea [756501,113573] 0 2024-04-24T22:45:41.143 INFO:tasks.workunit.client.0.smithi063.stdout:9/882: dread d6/d2f/dcd/d53/f99 [0,4194304] 0 2024-04-24T22:45:41.143 INFO:tasks.workunit.client.0.smithi063.stdout:6/923: mknod db/d1d/d20/d11e/c136 0 2024-04-24T22:45:41.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/717: mknod d2/db/d75/da1/d36/cfb 0 2024-04-24T22:45:41.144 INFO:tasks.workunit.client.0.smithi063.stdout:2/957: write d0/dd/fcc [352479,27409] 0 2024-04-24T22:45:41.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/859: creat da/d39/d94/dbc/f12b x:0 0 0 2024-04-24T22:45:41.147 INFO:tasks.workunit.client.0.smithi063.stdout:2/958: fsync d0/d2/d2d/d31/d89/d7/d86/da0/d94/ff9 0 2024-04-24T22:45:41.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/860: creat da/d48/f12c x:0 0 0 2024-04-24T22:45:41.150 INFO:tasks.workunit.client.1.smithi192.stdout:3/724: dwrite d2/d4/d12/f1b [4194304,4194304] 0 2024-04-24T22:45:41.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/906: dwrite d3/db5/d21/d9f/dd5/d106/d11/f5f [0,4194304] 0 2024-04-24T22:45:41.152 INFO:tasks.workunit.client.0.smithi063.stdout:5/816: dread d5/d24/d36/f5d [0,4194304] 0 2024-04-24T22:45:41.153 INFO:tasks.workunit.client.0.smithi063.stdout:1/907: write d3/ff2 [931425,54346] 0 2024-04-24T22:45:41.154 INFO:tasks.workunit.client.0.smithi063.stdout:5/817: chown d5/d11/d23/d2d/d62/d73 761971750 1 2024-04-24T22:45:41.155 INFO:tasks.workunit.client.0.smithi063.stdout:1/908: stat d3/db5/d21/d9f/dd5/d106/d11/d5e/c7c 0 2024-04-24T22:45:41.156 INFO:tasks.workunit.client.0.smithi063.stdout:1/909: chown d3/db5/d21/d49/lef 123026134 1 2024-04-24T22:45:41.157 INFO:tasks.workunit.client.0.smithi063.stdout:1/910: write d3/db5/d21/d9f/f118 [249545,89288] 0 2024-04-24T22:45:41.165 INFO:tasks.workunit.client.0.smithi063.stdout:1/911: dwrite d3/db5/d21/d9f/daa/f126 [0,4194304] 0 2024-04-24T22:45:41.165 INFO:tasks.workunit.client.0.smithi063.stdout:1/912: chown d3/de0/d79/f2e 689994701 1 2024-04-24T22:45:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:40 smithi192 ceph-mon[33991]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 85 MiB/s rd, 201 MiB/s wr, 425 op/s 2024-04-24T22:45:41.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:40 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:41.190 INFO:tasks.workunit.client.0.smithi063.stdout:4/833: sync 2024-04-24T22:45:41.190 INFO:tasks.workunit.client.0.smithi063.stdout:7/793: sync 2024-04-24T22:45:41.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/794: write d0/d4/de/d6f/d2e/d76/ff3 [3400445,82238] 0 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:9/786: sync 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/705: sync 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:5/586: sync 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:1/572: sync 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:2/789: sync 2024-04-24T22:45:41.206 INFO:tasks.workunit.client.1.smithi192.stdout:8/666: sync 2024-04-24T22:45:41.208 INFO:tasks.workunit.client.1.smithi192.stdout:3/725: dwrite d2/d62/db2/f85 [0,4194304] 0 2024-04-24T22:45:41.209 INFO:tasks.workunit.client.0.smithi063.stdout:6/924: symlink db/d1d/d20/d29/d5d/l137 0 2024-04-24T22:45:41.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/907: truncate d1/d22/fab 195721 0 2024-04-24T22:45:41.211 INFO:tasks.workunit.client.1.smithi192.stdout:8/667: stat db/dc/d29/c95 0 2024-04-24T22:45:41.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/908: write d1/d2/d3/db6/ffc [5055984,83386] 0 2024-04-24T22:45:41.212 INFO:tasks.workunit.client.1.smithi192.stdout:5/587: read d0/d13/d19/d83/f87 [1752024,46148] 0 2024-04-24T22:45:41.213 INFO:tasks.workunit.client.0.smithi063.stdout:2/959: unlink d0/d2/d2f/ddc/ff3 0 2024-04-24T22:45:41.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/668: write db/dc/fb0 [191618,76043] 0 2024-04-24T22:45:41.217 INFO:tasks.workunit.client.0.smithi063.stdout:3/861: rmdir da/d39/d94/db3/df3 39 2024-04-24T22:45:41.221 INFO:tasks.workunit.client.1.smithi192.stdout:0/706: symlink d3/d6d/led 0 2024-04-24T22:45:41.222 INFO:tasks.workunit.client.1.smithi192.stdout:7/774: dwrite d2/db/d25/f69 [0,4194304] 0 2024-04-24T22:45:41.224 INFO:tasks.workunit.client.1.smithi192.stdout:5/588: dwrite d0/d13/d19/da9/fab [0,4194304] 0 2024-04-24T22:45:41.225 INFO:tasks.workunit.client.0.smithi063.stdout:1/913: chown d3/db5/d21/d9f/dd5/d106/d39/c45 3 1 2024-04-24T22:45:41.226 INFO:tasks.workunit.client.0.smithi063.stdout:0/803: sync 2024-04-24T22:45:41.228 INFO:tasks.workunit.client.1.smithi192.stdout:5/589: write d0/d13/d19/d64/d5f/f20 [3262667,7947] 0 2024-04-24T22:45:41.229 INFO:tasks.workunit.client.1.smithi192.stdout:3/726: symlink d2/d4/d12/d1a/le9 0 2024-04-24T22:45:41.233 INFO:tasks.workunit.client.1.smithi192.stdout:6/640: write d0/d8/d22/fda [1121027,26394] 0 2024-04-24T22:45:41.233 INFO:tasks.workunit.client.0.smithi063.stdout:4/834: truncate d2/d19/d82/d96/fad 133045 0 2024-04-24T22:45:41.234 INFO:tasks.workunit.client.1.smithi192.stdout:6/641: stat d0/d8/d18/d2e/c45 0 2024-04-24T22:45:41.236 INFO:tasks.workunit.client.1.smithi192.stdout:5/590: dwrite d0/f73 [0,4194304] 0 2024-04-24T22:45:41.236 INFO:tasks.workunit.client.0.smithi063.stdout:7/795: creat d0/d4/de/dfc/f109 x:0 0 0 2024-04-24T22:45:41.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/909: unlink d1/d22/d33/da6/dae/de7/ce9 0 2024-04-24T22:45:41.246 INFO:tasks.workunit.client.0.smithi063.stdout:8/910: truncate d1/f111 354402 0 2024-04-24T22:45:41.247 INFO:tasks.workunit.client.0.smithi063.stdout:2/960: creat d0/d2/d63/f128 x:0 0 0 2024-04-24T22:45:41.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/775: symlink d2/db/d21/d42/d67/d9b/d6e/db5/lf4 0 2024-04-24T22:45:41.251 INFO:tasks.workunit.client.0.smithi063.stdout:3/862: creat da/d39/d94/f12d x:0 0 0 2024-04-24T22:45:41.251 INFO:tasks.workunit.client.0.smithi063.stdout:8/911: read d1/d22/d33/da6/d54/f71 [2170060,77399] 0 2024-04-24T22:45:41.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/818: mknod d5/d11/d23/c116 0 2024-04-24T22:45:41.254 INFO:tasks.workunit.client.0.smithi063.stdout:9/883: write d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d8e/f98 [1682698,108839] 0 2024-04-24T22:45:41.254 INFO:tasks.workunit.client.0.smithi063.stdout:2/961: dwrite d0/d2/d2d/d31/d89/d7/f13 [0,4194304] 0 2024-04-24T22:45:41.255 INFO:tasks.workunit.client.0.smithi063.stdout:2/962: readlink d0/d2/lcf 0 2024-04-24T22:45:41.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/914: creat d3/de0/d79/f127 x:0 0 0 2024-04-24T22:45:41.266 INFO:tasks.workunit.client.1.smithi192.stdout:6/642: mkdir d0/d8/d29/d4c/d4d/d83/dde 0 2024-04-24T22:45:41.267 INFO:tasks.workunit.client.0.smithi063.stdout:4/835: stat d2/f97 0 2024-04-24T22:45:41.268 INFO:tasks.workunit.client.1.smithi192.stdout:2/790: write d4/d9/d10/d6c/fb5 [576627,75715] 0 2024-04-24T22:45:41.268 INFO:tasks.workunit.client.1.smithi192.stdout:0/707: dread d3/d27/d28/d31/dbc/fd8 [0,4194304] 0 2024-04-24T22:45:41.273 INFO:tasks.workunit.client.1.smithi192.stdout:1/573: creat d9/d15/d2d/d63/d68/da7/fbd x:0 0 0 2024-04-24T22:45:41.275 INFO:tasks.workunit.client.0.smithi063.stdout:6/925: mkdir db/d1d/d8d/dce/d12a/d10f/d138 0 2024-04-24T22:45:41.276 INFO:tasks.workunit.client.1.smithi192.stdout:8/669: write db/dc/d29/f4f [4576323,92584] 0 2024-04-24T22:45:41.278 INFO:tasks.workunit.client.0.smithi063.stdout:3/863: fsync da/d48/f5d 0 2024-04-24T22:45:41.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/727: dread d2/d27/dc6/d4f/f7a [0,4194304] 0 2024-04-24T22:45:41.280 INFO:tasks.workunit.client.1.smithi192.stdout:3/728: write d2/f33 [3386181,94968] 0 2024-04-24T22:45:41.282 INFO:tasks.workunit.client.1.smithi192.stdout:3/729: truncate d2/d4/d4e/f8f 4656243 0 2024-04-24T22:45:41.282 INFO:tasks.workunit.client.1.smithi192.stdout:6/643: mkdir d0/d8/d29/d5a/ddf 0 2024-04-24T22:45:41.284 INFO:tasks.workunit.client.1.smithi192.stdout:5/591: symlink d0/d1b/d62/dc1/dc2/lc9 0 2024-04-24T22:45:41.287 INFO:tasks.workunit.client.0.smithi063.stdout:2/963: creat d0/d2/d2f/dba/f129 x:0 0 0 2024-04-24T22:45:41.287 INFO:tasks.workunit.client.1.smithi192.stdout:0/708: dread - d3/d27/d28/dcc/d5c/d8a/fad zero size 2024-04-24T22:45:41.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/574: mknod d9/d15/d2d/d48/d93/cbe 0 2024-04-24T22:45:41.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/575: chown d9/d15/d2d/d31/d2f 27462058 1 2024-04-24T22:45:41.292 INFO:tasks.workunit.client.0.smithi063.stdout:1/915: rmdir d3/de0/d79 39 2024-04-24T22:45:41.292 INFO:tasks.workunit.client.0.smithi063.stdout:4/836: creat d2/d19/d33/d9d/da8/f115 x:0 0 0 2024-04-24T22:45:41.292 INFO:tasks.workunit.client.0.smithi063.stdout:0/804: write d6/d64/d21/f48 [2312097,73344] 0 2024-04-24T22:45:41.294 INFO:tasks.workunit.client.1.smithi192.stdout:8/670: rename db/dc/l3a to db/d5d/d74/da0/ld4 0 2024-04-24T22:45:41.295 INFO:tasks.workunit.client.1.smithi192.stdout:9/787: write d3/dd/de/f55 [928686,27985] 0 2024-04-24T22:45:41.296 INFO:tasks.workunit.client.1.smithi192.stdout:1/576: dread d9/f4f [0,4194304] 0 2024-04-24T22:45:41.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/788: readlink d3/dd/d62/d6d/d91/l9a 0 2024-04-24T22:45:41.298 INFO:tasks.workunit.client.1.smithi192.stdout:6/644: write d0/d8/d29/d5a/fb6 [5085128,78884] 0 2024-04-24T22:45:41.298 INFO:tasks.workunit.client.1.smithi192.stdout:9/789: readlink d3/dd/de/d17/l37 0 2024-04-24T22:45:41.302 INFO:tasks.workunit.client.1.smithi192.stdout:2/791: mknod d4/d9/d10/d6c/c101 0 2024-04-24T22:45:41.302 INFO:tasks.workunit.client.0.smithi063.stdout:3/864: stat da/d40/lb9 0 2024-04-24T22:45:41.303 INFO:tasks.workunit.client.1.smithi192.stdout:2/792: readlink d4/d9/d10/d6c/d8c/l9b 0 2024-04-24T22:45:41.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:40 smithi063 ceph-mon[26912]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 85 MiB/s rd, 201 MiB/s wr, 425 op/s 2024-04-24T22:45:41.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:40 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:41.305 INFO:tasks.workunit.client.0.smithi063.stdout:9/884: symlink d6/l128 0 2024-04-24T22:45:41.307 INFO:tasks.workunit.client.1.smithi192.stdout:8/671: symlink db/d43/d7c/ld5 0 2024-04-24T22:45:41.308 INFO:tasks.workunit.client.0.smithi063.stdout:8/912: dwrite d1/d11/d32/fb2 [0,4194304] 0 2024-04-24T22:45:41.309 INFO:tasks.workunit.client.0.smithi063.stdout:5/819: rename d5/d24/d36/f63 to d5/d24/d44/f117 0 2024-04-24T22:45:41.309 INFO:tasks.workunit.client.1.smithi192.stdout:3/730: mknod d2/d27/dc6/d41/dad/cea 0 2024-04-24T22:45:41.311 INFO:tasks.workunit.client.1.smithi192.stdout:1/577: mkdir d9/d15/d2d/d63/d68/da7/dbf 0 2024-04-24T22:45:41.312 INFO:tasks.workunit.client.1.smithi192.stdout:5/592: write d0/d13/d19/d64/d5f/d2c/f51 [331544,68687] 0 2024-04-24T22:45:41.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/885: dwrite d6/dbd/dcf/fec [4194304,4194304] 0 2024-04-24T22:45:41.313 INFO:tasks.workunit.client.1.smithi192.stdout:5/593: readlink d0/d1b/l26 0 2024-04-24T22:45:41.314 INFO:tasks.workunit.client.1.smithi192.stdout:6/645: mkdir d0/d8/d22/dcb/de0 0 2024-04-24T22:45:41.315 INFO:tasks.workunit.client.1.smithi192.stdout:6/646: chown d0/d8/d3d/d9d/fd0 2119063472 1 2024-04-24T22:45:41.320 INFO:tasks.workunit.client.1.smithi192.stdout:6/647: dread d0/d8/d29/d4c/d4d/f9e [0,4194304] 0 2024-04-24T22:45:41.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/776: getdents d2/db/d2a/d57 0 2024-04-24T22:45:41.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/578: creat d9/d15/d2d/d31/d2f/fc0 x:0 0 0 2024-04-24T22:45:41.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/790: rename d3/d24/f7e to d3/d59/d7d/d88/f10c 0 2024-04-24T22:45:41.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/791: chown d3/dd/d53/lc1 125256027 1 2024-04-24T22:45:41.334 INFO:tasks.workunit.client.0.smithi063.stdout:6/926: sync 2024-04-24T22:45:41.337 INFO:tasks.workunit.client.1.smithi192.stdout:2/793: write d4/d9/d10/d6c/d95/fa2 [872117,27463] 0 2024-04-24T22:45:41.337 INFO:tasks.workunit.client.1.smithi192.stdout:2/794: chown d4/c73 9952 1 2024-04-24T22:45:41.339 INFO:tasks.workunit.client.1.smithi192.stdout:2/795: write d4/d9/dd4/fea [428465,70185] 0 2024-04-24T22:45:41.340 INFO:tasks.workunit.client.1.smithi192.stdout:6/648: mkdir d0/d8/d3d/d3e/de1 0 2024-04-24T22:45:41.341 INFO:tasks.workunit.client.1.smithi192.stdout:7/777: truncate d2/db/f6f 507206 0 2024-04-24T22:45:41.342 INFO:tasks.workunit.client.1.smithi192.stdout:3/731: readlink d2/d4/d12/d1a/d1d/l58 0 2024-04-24T22:45:41.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/594: mknod d0/d13/d19/d83/dad/cca 0 2024-04-24T22:45:41.346 INFO:tasks.workunit.client.1.smithi192.stdout:1/579: dread d9/dd/f41 [0,4194304] 0 2024-04-24T22:45:41.348 INFO:tasks.workunit.client.1.smithi192.stdout:9/792: unlink d3/d5d/da1/d85/fb7 0 2024-04-24T22:45:41.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/709: link d3/d27/d3c/d3b/c44 d3/d27/d28/d31/dbc/cee 0 2024-04-24T22:45:41.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/672: dread db/dc/d22/d31/d33/db7/fc7 [0,4194304] 0 2024-04-24T22:45:41.352 INFO:tasks.workunit.client.1.smithi192.stdout:6/649: mknod d0/d8/d3d/d9d/ce2 0 2024-04-24T22:45:41.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/732: symlink d2/d27/dc6/d41/dad/leb 0 2024-04-24T22:45:41.362 INFO:tasks.workunit.client.1.smithi192.stdout:1/580: rename d9/d15/d2d/f6c to d9/d15/d5e/d90/da9/fc1 0 2024-04-24T22:45:41.362 INFO:tasks.workunit.client.1.smithi192.stdout:8/673: dread db/dc/d22/f5e [0,4194304] 0 2024-04-24T22:45:41.363 INFO:tasks.workunit.client.1.smithi192.stdout:8/674: chown db/dc/d29/c79 3160 1 2024-04-24T22:45:41.365 INFO:tasks.workunit.client.1.smithi192.stdout:0/710: fdatasync d3/d27/d28/d2e/f63 0 2024-04-24T22:45:41.371 INFO:tasks.workunit.client.0.smithi063.stdout:7/796: creat d0/d4/de/f10a x:0 0 0 2024-04-24T22:45:41.375 INFO:tasks.workunit.client.0.smithi063.stdout:3/865: write da/d18/d9e/fc4 [47308,22624] 0 2024-04-24T22:45:41.378 INFO:tasks.workunit.client.1.smithi192.stdout:5/595: mkdir d0/d1b/dcb 0 2024-04-24T22:45:41.380 INFO:tasks.workunit.client.0.smithi063.stdout:2/964: rename d0/d2/d39/d11a/de2/de4/l117 to d0/dd/d72/l12a 0 2024-04-24T22:45:41.382 INFO:tasks.workunit.client.0.smithi063.stdout:8/913: creat d1/d11/d8b/dd6/f120 x:0 0 0 2024-04-24T22:45:41.382 INFO:tasks.workunit.client.0.smithi063.stdout:3/866: dread da/d18/f8d [4194304,4194304] 0 2024-04-24T22:45:41.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/867: stat da/d18/dfb 0 2024-04-24T22:45:41.383 INFO:tasks.workunit.client.1.smithi192.stdout:2/796: write d4/d9/d37/f3e [1451071,58895] 0 2024-04-24T22:45:41.386 INFO:tasks.workunit.client.1.smithi192.stdout:8/675: creat db/d43/d71/fd6 x:0 0 0 2024-04-24T22:45:41.387 INFO:tasks.workunit.client.1.smithi192.stdout:8/676: fdatasync db/d43/f69 0 2024-04-24T22:45:41.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/837: write d2/d8/d36/f44 [2505107,99336] 0 2024-04-24T22:45:41.395 INFO:tasks.workunit.client.0.smithi063.stdout:5/820: rename d5/d24 to d5/d11/d23/d2d/d62/d73/d118 0 2024-04-24T22:45:41.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/733: mknod d2/d4/d12/dbc/cec 0 2024-04-24T22:45:41.400 INFO:tasks.workunit.client.0.smithi063.stdout:3/868: unlink da/d39/f7a 0 2024-04-24T22:45:41.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/581: symlink d9/d15/d2d/d48/d93/da4/dbb/dac/lc2 0 2024-04-24T22:45:41.402 INFO:tasks.workunit.client.1.smithi192.stdout:1/582: chown d9/d15/d2d/d31/d2f 11988681 1 2024-04-24T22:45:41.402 INFO:tasks.workunit.client.1.smithi192.stdout:7/778: dwrite d2/db/d21/d42/d5b/f5d [0,4194304] 0 2024-04-24T22:45:41.404 INFO:tasks.workunit.client.1.smithi192.stdout:1/583: write d9/f3c [4900539,65981] 0 2024-04-24T22:45:41.407 INFO:tasks.workunit.client.1.smithi192.stdout:1/584: read d9/d15/f1b [180016,104312] 0 2024-04-24T22:45:41.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/677: truncate db/dc/f80 189386 0 2024-04-24T22:45:41.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/678: chown db/d43/d76 114 1 2024-04-24T22:45:41.411 INFO:tasks.workunit.client.1.smithi192.stdout:0/711: write d3/d27/d28/dcc/d5c/d8a/fad [210149,119176] 0 2024-04-24T22:45:41.412 INFO:tasks.workunit.client.0.smithi063.stdout:7/797: mkdir d0/d4/d10b 0 2024-04-24T22:45:41.413 INFO:tasks.workunit.client.0.smithi063.stdout:7/798: fdatasync d0/d4/de/d6f/d5c/d9c/ffa 0 2024-04-24T22:45:41.414 INFO:tasks.workunit.client.1.smithi192.stdout:6/650: creat d0/d8/d3d/d3e/d40/d44/fe3 x:0 0 0 2024-04-24T22:45:41.420 INFO:tasks.workunit.client.1.smithi192.stdout:6/651: dwrite d0/d8/d29/d4c/db0/fcc [0,4194304] 0 2024-04-24T22:45:41.424 INFO:tasks.workunit.client.1.smithi192.stdout:9/793: getdents d3/d5d 0 2024-04-24T22:45:41.424 INFO:tasks.workunit.client.0.smithi063.stdout:5/821: dread d5/d11/d23/d2d/d62/d73/d118/d44/fbf [0,4194304] 0 2024-04-24T22:45:41.427 INFO:tasks.workunit.client.1.smithi192.stdout:6/652: dread d0/d8/f33 [0,4194304] 0 2024-04-24T22:45:41.427 INFO:tasks.workunit.client.1.smithi192.stdout:3/734: write d2/d27/d28/f30 [266602,101188] 0 2024-04-24T22:45:41.431 INFO:tasks.workunit.client.0.smithi063.stdout:6/927: creat db/d1c/dea/d116/f139 x:0 0 0 2024-04-24T22:45:41.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/797: mkdir d4/d9/d10/d102 0 2024-04-24T22:45:41.432 INFO:tasks.workunit.client.0.smithi063.stdout:1/916: getdents d3/da/d34 0 2024-04-24T22:45:41.434 INFO:tasks.workunit.client.0.smithi063.stdout:0/805: getdents d6/d64/da0/d98/dd6 0 2024-04-24T22:45:41.435 INFO:tasks.workunit.client.1.smithi192.stdout:8/679: symlink db/dc/d22/d31/d33/d81/d8a/ld7 0 2024-04-24T22:45:41.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/886: write d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/fe3 [659199,6611] 0 2024-04-24T22:45:41.436 INFO:tasks.workunit.client.1.smithi192.stdout:8/680: chown db/dc/d29/cb5 42682 1 2024-04-24T22:45:41.436 INFO:tasks.workunit.client.1.smithi192.stdout:7/779: write d2/db/d49/f6c [333882,27937] 0 2024-04-24T22:45:41.437 INFO:tasks.workunit.client.0.smithi063.stdout:0/806: stat d6/da/d2b/fe3 0 2024-04-24T22:45:41.437 INFO:tasks.workunit.client.1.smithi192.stdout:0/712: dread d3/d27/f71 [0,4194304] 0 2024-04-24T22:45:41.439 INFO:tasks.workunit.client.1.smithi192.stdout:7/780: write d2/db/d21/d42/f8c [5159233,129329] 0 2024-04-24T22:45:41.440 INFO:tasks.workunit.client.0.smithi063.stdout:4/838: rename d2/d19/lf3 to d2/d8/d40/d51/dd5/l116 0 2024-04-24T22:45:41.441 INFO:tasks.workunit.client.1.smithi192.stdout:0/713: write d3/d27/d28/dcc/f4d [2664517,32418] 0 2024-04-24T22:45:41.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/914: dwrite d1/d2/d3/f46 [0,4194304] 0 2024-04-24T22:45:41.442 INFO:tasks.workunit.client.1.smithi192.stdout:0/714: stat d3/d27/d3c/d3b/f8d 0 2024-04-24T22:45:41.442 INFO:tasks.workunit.client.0.smithi063.stdout:6/928: dwrite db/d1c/dc2/fd3 [4194304,4194304] 0 2024-04-24T22:45:41.443 INFO:tasks.workunit.client.1.smithi192.stdout:5/596: creat d0/d13/d19/d64/d5f/fcc x:0 0 0 2024-04-24T22:45:41.443 INFO:tasks.workunit.client.0.smithi063.stdout:2/965: creat d0/d2/d2d/d31/d89/f12b x:0 0 0 2024-04-24T22:45:41.444 INFO:tasks.workunit.client.0.smithi063.stdout:6/929: readlink db/d1d/d2d/d48/dfd/l103 0 2024-04-24T22:45:41.448 INFO:tasks.workunit.client.1.smithi192.stdout:3/735: creat d2/d27/d28/dab/dca/fed x:0 0 0 2024-04-24T22:45:41.448 INFO:tasks.workunit.client.1.smithi192.stdout:3/736: chown d2/d27/d28/da5/d93 257883 1 2024-04-24T22:45:41.452 INFO:tasks.workunit.client.1.smithi192.stdout:2/798: symlink d4/d29/d60/d9a/l103 0 2024-04-24T22:45:41.454 INFO:tasks.workunit.client.0.smithi063.stdout:9/887: mkdir d6/d2f/dcd/db/d1c/d78/d9a/d129 0 2024-04-24T22:45:41.456 INFO:tasks.workunit.client.0.smithi063.stdout:4/839: unlink d2/d8/d40/d51/dd5/ldc 0 2024-04-24T22:45:41.461 INFO:tasks.workunit.client.1.smithi192.stdout:8/681: rename db/dc/f13 to db/d43/d7c/d85/fd8 0 2024-04-24T22:45:41.462 INFO:tasks.workunit.client.0.smithi063.stdout:3/869: sync 2024-04-24T22:45:41.466 INFO:tasks.workunit.client.0.smithi063.stdout:6/930: symlink db/d1d/d2d/d69/d82/d96/l13a 0 2024-04-24T22:45:41.467 INFO:tasks.workunit.client.0.smithi063.stdout:7/799: dwrite d0/d4/de/d17/f9b [0,4194304] 0 2024-04-24T22:45:41.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/822: write d5/d11/d23/d2d/d62/d73/f74 [83047,62635] 0 2024-04-24T22:45:41.470 INFO:tasks.workunit.client.0.smithi063.stdout:0/807: truncate d6/d74/f7e 923646 0 2024-04-24T22:45:41.472 INFO:tasks.workunit.client.0.smithi063.stdout:1/917: write d3/db5/d21/d9f/dd5/d106/d11/f111 [1016786,5125] 0 2024-04-24T22:45:41.473 INFO:tasks.workunit.client.1.smithi192.stdout:6/653: creat d0/d8/d22/dcb/de0/fe4 x:0 0 0 2024-04-24T22:45:41.475 INFO:tasks.workunit.client.0.smithi063.stdout:4/840: rename d2/d9f/df8 to d2/d9f/d117 0 2024-04-24T22:45:41.476 INFO:tasks.workunit.client.0.smithi063.stdout:8/915: dread d1/d22/d33/da6/fa2 [0,4194304] 0 2024-04-24T22:45:41.477 INFO:tasks.workunit.client.1.smithi192.stdout:3/737: mkdir d2/d27/d28/dab/dca/dee 0 2024-04-24T22:45:41.477 INFO:tasks.workunit.client.1.smithi192.stdout:3/738: chown d2/d27/d28/da5/f66 1612456 1 2024-04-24T22:45:41.478 INFO:tasks.workunit.client.0.smithi063.stdout:2/966: dread d0/d2/d2f/f6d [0,4194304] 0 2024-04-24T22:45:41.479 INFO:tasks.workunit.client.1.smithi192.stdout:2/799: rename d4/d29/d60/c46 to d4/d9/d9f/c104 0 2024-04-24T22:45:41.486 INFO:tasks.workunit.client.1.smithi192.stdout:8/682: unlink db/dc/d22/d31/d33/d8f/fac 0 2024-04-24T22:45:41.486 INFO:tasks.workunit.client.1.smithi192.stdout:8/683: chown db/dc/d29/c79 4999 1 2024-04-24T22:45:41.488 INFO:tasks.workunit.client.1.smithi192.stdout:8/684: dread - db/dc/d22/d31/fc1 zero size 2024-04-24T22:45:41.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/823: symlink d5/d1d/da3/l119 0 2024-04-24T22:45:41.491 INFO:tasks.workunit.client.0.smithi063.stdout:5/824: chown d5/d11/d23/d2d/d62/d73/d118/d49/f6b 259402 1 2024-04-24T22:45:41.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/597: unlink d0/d13/d19/d64/d5f/fcc 0 2024-04-24T22:45:41.493 INFO:tasks.workunit.client.0.smithi063.stdout:9/888: mkdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d11e/d12a 0 2024-04-24T22:45:41.494 INFO:tasks.workunit.client.0.smithi063.stdout:9/889: dread - d6/d2f/dcd/db/d6b/d4f/dad/f105 zero size 2024-04-24T22:45:41.494 INFO:tasks.workunit.client.1.smithi192.stdout:6/654: fsync d0/d8/d18/d2e/fa0 0 2024-04-24T22:45:41.497 INFO:tasks.workunit.client.1.smithi192.stdout:0/715: dread d3/d27/d28/f24 [0,4194304] 0 2024-04-24T22:45:41.502 INFO:tasks.workunit.client.1.smithi192.stdout:2/800: dread d4/d9/d10/d6c/d8c/fc6 [0,4194304] 0 2024-04-24T22:45:41.503 INFO:tasks.workunit.client.0.smithi063.stdout:2/967: dread d0/d2/d39/d11a/de2/f6b [0,4194304] 0 2024-04-24T22:45:41.503 INFO:tasks.workunit.client.1.smithi192.stdout:7/781: link d2/db/d49/cb4 d2/db/d21/d42/d67/d9b/d6e/d77/cf5 0 2024-04-24T22:45:41.507 INFO:tasks.workunit.client.0.smithi063.stdout:6/931: dwrite db/d1d/d20/d3f/d5f/fd8 [0,4194304] 0 2024-04-24T22:45:41.509 INFO:tasks.workunit.client.1.smithi192.stdout:6/655: mkdir d0/d8/d29/d5a/d63/de5 0 2024-04-24T22:45:41.511 INFO:tasks.workunit.client.0.smithi063.stdout:0/808: creat d6/d110/f111 x:0 0 0 2024-04-24T22:45:41.511 INFO:tasks.workunit.client.0.smithi063.stdout:4/841: write d2/d8/d3c/f75 [2064542,103128] 0 2024-04-24T22:45:41.512 INFO:tasks.workunit.client.1.smithi192.stdout:3/739: mkdir d2/d4/d12/d39/def 0 2024-04-24T22:45:41.513 INFO:tasks.workunit.client.0.smithi063.stdout:5/825: rmdir d5/d11/d23 39 2024-04-24T22:45:41.521 INFO:tasks.workunit.client.0.smithi063.stdout:9/890: symlink d6/d2f/dcd/d1f/l12b 0 2024-04-24T22:45:41.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/916: rename d1/d21/f3a to d1/d11/d32/d38/db1/f121 0 2024-04-24T22:45:41.523 INFO:tasks.workunit.client.1.smithi192.stdout:6/656: dwrite d0/d8/d3d/d9d/fd0 [0,4194304] 0 2024-04-24T22:45:41.523 INFO:tasks.workunit.client.1.smithi192.stdout:8/685: dread db/dc/f41 [0,4194304] 0 2024-04-24T22:45:41.524 INFO:tasks.workunit.client.1.smithi192.stdout:5/598: dwrite d0/d1b/f69 [0,4194304] 0 2024-04-24T22:45:41.528 INFO:tasks.workunit.client.1.smithi192.stdout:0/716: write d3/d27/d28/fc3 [486501,122115] 0 2024-04-24T22:45:41.530 INFO:tasks.workunit.client.1.smithi192.stdout:0/717: stat d3/d27/d6a/dc4 0 2024-04-24T22:45:41.531 INFO:tasks.workunit.client.1.smithi192.stdout:8/686: dwrite db/dc/d22/d31/fc1 [0,4194304] 0 2024-04-24T22:45:41.535 INFO:tasks.workunit.client.1.smithi192.stdout:0/718: chown d3/d27/d3c/d3b/fd1 8976 1 2024-04-24T22:45:41.536 INFO:tasks.workunit.client.0.smithi063.stdout:6/932: creat db/d1d/d2d/d69/dd4/f13b x:0 0 0 2024-04-24T22:45:41.536 INFO:tasks.workunit.client.0.smithi063.stdout:6/933: write db/d1d/d20/d29/fcc [4565724,21069] 0 2024-04-24T22:45:41.537 INFO:tasks.workunit.client.0.smithi063.stdout:2/968: dread d0/d2/d39/f65 [0,4194304] 0 2024-04-24T22:45:41.537 INFO:tasks.workunit.client.0.smithi063.stdout:2/969: chown d0/f69 143 1 2024-04-24T22:45:41.539 INFO:tasks.workunit.client.1.smithi192.stdout:3/740: fdatasync d2/d4/d12/d1a/f81 0 2024-04-24T22:45:41.539 INFO:tasks.workunit.client.0.smithi063.stdout:2/970: chown d0/d2/d2d/d31/d89/f25 3590 1 2024-04-24T22:45:41.540 INFO:tasks.workunit.client.0.smithi063.stdout:0/809: creat d6/d64/d21/d25/d31/f112 x:0 0 0 2024-04-24T22:45:41.542 INFO:tasks.workunit.client.0.smithi063.stdout:1/918: creat d3/da/da2/f128 x:0 0 0 2024-04-24T22:45:41.542 INFO:tasks.workunit.client.1.smithi192.stdout:5/599: creat d0/d1b/d50/fcd x:0 0 0 2024-04-24T22:45:41.544 INFO:tasks.workunit.client.0.smithi063.stdout:3/870: link da/d18/d3a/d63/l7d da/d18/d9e/l12e 0 2024-04-24T22:45:41.545 INFO:tasks.workunit.client.0.smithi063.stdout:7/800: getdents d0/d4/de/d6f/d2e/d55 0 2024-04-24T22:45:41.546 INFO:tasks.workunit.client.0.smithi063.stdout:6/934: mkdir db/d1d/d2d/d69/d82/d13c 0 2024-04-24T22:45:41.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/782: creat d2/db/d21/d42/d67/ff6 x:0 0 0 2024-04-24T22:45:41.548 INFO:tasks.workunit.client.1.smithi192.stdout:7/783: stat d2/db/d36 0 2024-04-24T22:45:41.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/842: mkdir d2/d8/d36/d118 0 2024-04-24T22:45:41.549 INFO:tasks.workunit.client.1.smithi192.stdout:6/657: symlink d0/le6 0 2024-04-24T22:45:41.549 INFO:tasks.workunit.client.1.smithi192.stdout:8/687: dread db/d43/d71/f83 [0,4194304] 0 2024-04-24T22:45:41.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/871: dwrite da/d39/d94/f110 [0,4194304] 0 2024-04-24T22:45:41.550 INFO:tasks.workunit.client.1.smithi192.stdout:8/688: chown db/dc/d22/d31/f4d 49 1 2024-04-24T22:45:41.551 INFO:tasks.workunit.client.1.smithi192.stdout:0/719: dread d3/f10 [4194304,4194304] 0 2024-04-24T22:45:41.552 INFO:tasks.workunit.client.1.smithi192.stdout:0/720: write d3/fc0 [3164441,128954] 0 2024-04-24T22:45:41.553 INFO:tasks.workunit.client.0.smithi063.stdout:2/971: mkdir d0/d2/d35/d12c 0 2024-04-24T22:45:41.554 INFO:tasks.workunit.client.1.smithi192.stdout:3/741: symlink d2/d27/dc6/d41/dac/lf0 0 2024-04-24T22:45:41.555 INFO:tasks.workunit.client.0.smithi063.stdout:2/972: write d0/d2/d2d/d31/d89/d1c/d3f/f110 [1505842,78594] 0 2024-04-24T22:45:41.556 INFO:tasks.workunit.client.0.smithi063.stdout:5/826: rename d5/d11/d23/d2d/d62/d73/d118/l107 to d5/d11/d23/daf/l11a 0 2024-04-24T22:45:41.558 INFO:tasks.workunit.client.1.smithi192.stdout:0/721: read d3/fc0 [3849616,2102] 0 2024-04-24T22:45:41.559 INFO:tasks.workunit.client.1.smithi192.stdout:5/600: mknod d0/d13/d19/d64/d5f/d2c/cce 0 2024-04-24T22:45:41.561 INFO:tasks.workunit.client.0.smithi063.stdout:8/917: mkdir d1/d11/d32/d38/db7/d122 0 2024-04-24T22:45:41.561 INFO:tasks.workunit.client.1.smithi192.stdout:8/689: dwrite db/dc/d22/d31/fc1 [0,4194304] 0 2024-04-24T22:45:41.562 INFO:tasks.workunit.client.1.smithi192.stdout:2/801: getdents d4/d9/d10/d6c/d8c/da5 0 2024-04-24T22:45:41.562 INFO:tasks.workunit.client.1.smithi192.stdout:8/690: fsync db/d43/d71/fd6 0 2024-04-24T22:45:41.565 INFO:tasks.workunit.client.1.smithi192.stdout:7/784: creat d2/db/d2a/ff7 x:0 0 0 2024-04-24T22:45:41.566 INFO:tasks.workunit.client.0.smithi063.stdout:2/973: dread d0/dd/d72/d77/d87/f9c [0,4194304] 0 2024-04-24T22:45:41.569 INFO:tasks.workunit.client.0.smithi063.stdout:6/935: truncate db/d1d/d20/d3f/d7f/d68/d70/f67 1814685 0 2024-04-24T22:45:41.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/936: chown db 1233 1 2024-04-24T22:45:41.572 INFO:tasks.workunit.client.0.smithi063.stdout:3/872: mknod da/d18/d47/c12f 0 2024-04-24T22:45:41.575 INFO:tasks.workunit.client.0.smithi063.stdout:0/810: dwrite d6/da/dc/f71 [4194304,4194304] 0 2024-04-24T22:45:41.577 INFO:tasks.workunit.client.1.smithi192.stdout:3/742: creat d2/d4/d4e/ff1 x:0 0 0 2024-04-24T22:45:41.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/827: symlink d5/d1d/d7d/d67/l11b 0 2024-04-24T22:45:41.582 INFO:tasks.workunit.client.1.smithi192.stdout:2/802: write d4/d9/d10/d6c/fdb [500183,103667] 0 2024-04-24T22:45:41.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/918: rmdir d1/d2/d3/d86 39 2024-04-24T22:45:41.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/743: dwrite d2/d27/dc6/d4b/d54/fb3 [0,4194304] 0 2024-04-24T22:45:41.584 INFO:tasks.workunit.client.1.smithi192.stdout:8/691: mknod db/d43/d71/cd9 0 2024-04-24T22:45:41.586 INFO:tasks.workunit.client.0.smithi063.stdout:7/801: fsync d0/d4/de/f8d 0 2024-04-24T22:45:41.587 INFO:tasks.workunit.client.1.smithi192.stdout:7/785: creat d2/db/d21/ff8 x:0 0 0 2024-04-24T22:45:41.588 INFO:tasks.workunit.client.0.smithi063.stdout:2/974: creat d0/d2/d2d/d31/d89/d1c/f12d x:0 0 0 2024-04-24T22:45:41.592 INFO:tasks.workunit.client.1.smithi192.stdout:6/658: rename d0/d8/d29/d5a/d63/de5 to d0/d8/d22/dcb/de0/de7 0 2024-04-24T22:45:41.594 INFO:tasks.workunit.client.0.smithi063.stdout:4/843: dread d2/f74 [0,4194304] 0 2024-04-24T22:45:41.594 INFO:tasks.workunit.client.0.smithi063.stdout:9/891: sync 2024-04-24T22:45:41.594 INFO:tasks.workunit.client.0.smithi063.stdout:3/873: symlink da/d18/dfb/l130 0 2024-04-24T22:45:41.596 INFO:tasks.workunit.client.0.smithi063.stdout:6/937: dread db/d1d/d20/fa1 [0,4194304] 0 2024-04-24T22:45:41.596 INFO:tasks.workunit.client.1.smithi192.stdout:0/722: mkdir d3/d27/d28/dcc/da8/def 0 2024-04-24T22:45:41.597 INFO:tasks.workunit.client.0.smithi063.stdout:0/811: truncate d6/d64/d21/d25/d62/d63/f82 4318100 0 2024-04-24T22:45:41.599 INFO:tasks.workunit.client.0.smithi063.stdout:6/938: truncate db/d1d/d20/d29/d5d/df5/f117 462927 0 2024-04-24T22:45:41.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/601: mkdir d0/d13/d19/d71/dcf 0 2024-04-24T22:45:41.599 INFO:tasks.workunit.client.1.smithi192.stdout:0/723: write d3/d27/d3c/d3b/fd1 [891330,126287] 0 2024-04-24T22:45:41.600 INFO:tasks.workunit.client.1.smithi192.stdout:2/803: creat d4/d9/d10/d6c/d6f/da8/f105 x:0 0 0 2024-04-24T22:45:41.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/804: stat d4/d9/d10/da7/laf 0 2024-04-24T22:45:41.602 INFO:tasks.workunit.client.1.smithi192.stdout:2/805: readlink d4/d9/d10/d6c/d8c/l9b 0 2024-04-24T22:45:41.602 INFO:tasks.workunit.client.0.smithi063.stdout:1/919: rename d3/l8d to d3/da/d34/d99/l129 0 2024-04-24T22:45:41.603 INFO:tasks.workunit.client.1.smithi192.stdout:5/602: read d0/d13/d19/f60 [17673,100574] 0 2024-04-24T22:45:41.606 INFO:tasks.workunit.client.1.smithi192.stdout:8/692: rename db/dc/d22/c59 to db/d43/cda 0 2024-04-24T22:45:41.608 INFO:tasks.workunit.client.1.smithi192.stdout:8/693: write db/dc/fb0 [128965,70417] 0 2024-04-24T22:45:41.609 INFO:tasks.workunit.client.1.smithi192.stdout:2/806: dread d4/d9/f52 [0,4194304] 0 2024-04-24T22:45:41.611 INFO:tasks.workunit.client.0.smithi063.stdout:7/802: mkdir d0/d4/de/d17/dd7/d10c 0 2024-04-24T22:45:41.613 INFO:tasks.workunit.client.1.smithi192.stdout:6/659: dread d0/d8/d3d/f55 [0,4194304] 0 2024-04-24T22:45:41.613 INFO:tasks.workunit.client.1.smithi192.stdout:0/724: mkdir d3/d27/d3c/d6c/db3/df0 0 2024-04-24T22:45:41.617 INFO:tasks.workunit.client.1.smithi192.stdout:3/744: dread d2/d27/dc6/d4f/f52 [0,4194304] 0 2024-04-24T22:45:41.619 INFO:tasks.workunit.client.0.smithi063.stdout:4/844: dread d2/ff [0,4194304] 0 2024-04-24T22:45:41.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/786: mknod d2/db/d21/d42/d67/cf9 0 2024-04-24T22:45:41.620 INFO:tasks.workunit.client.0.smithi063.stdout:9/892: symlink d6/d2f/dcd/d53/df6/l12c 0 2024-04-24T22:45:41.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/893: chown d6/d2f/dcd/db/d1c/d78/d9a/ddd/f113 11810490 1 2024-04-24T22:45:41.622 INFO:tasks.workunit.client.0.smithi063.stdout:9/894: chown d6/d2f/dcd/db/d6b/d4f/c50 5930 1 2024-04-24T22:45:41.623 INFO:tasks.workunit.client.0.smithi063.stdout:9/895: readlink d6/d2f/dcd/db/d6b/d4f/dad/l64 0 2024-04-24T22:45:41.624 INFO:tasks.workunit.client.1.smithi192.stdout:8/694: truncate db/d43/d7c/f9d 285459 0 2024-04-24T22:45:41.625 INFO:tasks.workunit.client.0.smithi063.stdout:6/939: mkdir db/d1c/dea/d13d 0 2024-04-24T22:45:41.627 INFO:tasks.workunit.client.0.smithi063.stdout:1/920: unlink d3/db5/d21/d9f/l10c 0 2024-04-24T22:45:41.628 INFO:tasks.workunit.client.0.smithi063.stdout:9/896: dwrite d6/d2f/dcd/d53/dd5/f104 [0,4194304] 0 2024-04-24T22:45:41.629 INFO:tasks.workunit.client.0.smithi063.stdout:1/921: fdatasync d3/da/d34/d99/ff9 0 2024-04-24T22:45:41.629 INFO:tasks.workunit.client.1.smithi192.stdout:6/660: mkdir d0/d8/d18/d2e/de8 0 2024-04-24T22:45:41.629 INFO:tasks.workunit.client.1.smithi192.stdout:6/661: chown d0/d8/d3d/c46 4131534 1 2024-04-24T22:45:41.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/725: read d3/d27/d28/d31/f3a [464631,87160] 0 2024-04-24T22:45:41.631 INFO:tasks.workunit.client.0.smithi063.stdout:8/919: symlink d1/d2/l123 0 2024-04-24T22:45:41.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/726: read d3/d27/d28/dcc/f6b [2533020,76471] 0 2024-04-24T22:45:41.635 INFO:tasks.workunit.client.0.smithi063.stdout:2/975: mkdir d0/d2/d2f/dba/d11b/d12e 0 2024-04-24T22:45:41.637 INFO:tasks.workunit.client.1.smithi192.stdout:6/662: dwrite d0/d8/fe [0,4194304] 0 2024-04-24T22:45:41.638 INFO:tasks.workunit.client.1.smithi192.stdout:7/787: unlink d2/db/d21/d42/d67/d9b/d6e/d77/l7c 0 2024-04-24T22:45:41.639 INFO:tasks.workunit.client.1.smithi192.stdout:7/788: dread - d2/db/d36/ff0 zero size 2024-04-24T22:45:41.640 INFO:tasks.workunit.client.0.smithi063.stdout:4/845: mknod d2/d19/d33/c119 0 2024-04-24T22:45:41.641 INFO:tasks.workunit.client.0.smithi063.stdout:4/846: chown d2/d8/l3d 47423 1 2024-04-24T22:45:41.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/789: dread d2/db/d21/d42/f93 [0,4194304] 0 2024-04-24T22:45:41.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/790: stat d2/db/d21/d42 0 2024-04-24T22:45:41.642 INFO:tasks.workunit.client.0.smithi063.stdout:3/874: mkdir da/d18/d47/d4d/d9b/d125/d131 0 2024-04-24T22:45:41.647 INFO:tasks.workunit.client.1.smithi192.stdout:0/727: write d3/d27/d28/f56 [434107,100705] 0 2024-04-24T22:45:41.648 INFO:tasks.workunit.client.0.smithi063.stdout:5/828: creat d5/d11/d23/d2d/d62/d73/d118/f11c x:0 0 0 2024-04-24T22:45:41.649 INFO:tasks.workunit.client.1.smithi192.stdout:3/745: rename d2/d4/d4e/c7e to d2/d4/d12/dbc/db0/cf2 0 2024-04-24T22:45:41.650 INFO:tasks.workunit.client.1.smithi192.stdout:8/695: rename db/d43/d7c to db/d43/d7c/d85/dcd/ddb 22 2024-04-24T22:45:41.651 INFO:tasks.workunit.client.0.smithi063.stdout:9/897: fdatasync d6/d2f/dcd/d1f/d20/d2a/f6e 0 2024-04-24T22:45:41.653 INFO:tasks.workunit.client.1.smithi192.stdout:5/603: link d0/d13/d19/l63 d0/d13/d19/d83/dad/ld0 0 2024-04-24T22:45:41.653 INFO:tasks.workunit.client.0.smithi063.stdout:8/920: write d1/d11/d32/d38/db7/f116 [477981,47856] 0 2024-04-24T22:45:41.655 INFO:tasks.workunit.client.1.smithi192.stdout:7/791: mkdir d2/db/d21/d42/d67/da8/dfa 0 2024-04-24T22:45:41.655 INFO:tasks.workunit.client.1.smithi192.stdout:7/792: read d2/f1a [4052661,8764] 0 2024-04-24T22:45:41.656 INFO:tasks.workunit.client.1.smithi192.stdout:8/696: dread db/dc/d29/f86 [0,4194304] 0 2024-04-24T22:45:41.659 INFO:tasks.workunit.client.0.smithi063.stdout:2/976: read d0/dd/f75 [149766,17320] 0 2024-04-24T22:45:41.660 INFO:tasks.workunit.client.1.smithi192.stdout:5/604: dread d0/d13/d3b/d2f/f84 [0,4194304] 0 2024-04-24T22:45:41.660 INFO:tasks.workunit.client.0.smithi063.stdout:3/875: unlink da/d18/d9e/f11c 0 2024-04-24T22:45:41.666 INFO:tasks.workunit.client.0.smithi063.stdout:2/977: dwrite d0/dd/d72/d77/dda/d7c/f61 [4194304,4194304] 0 2024-04-24T22:45:41.668 INFO:tasks.workunit.client.0.smithi063.stdout:4/847: dread d2/d8/f32 [0,4194304] 0 2024-04-24T22:45:41.674 INFO:tasks.workunit.client.0.smithi063.stdout:9/898: rename d6/dbd/dcf to d6/dbd/d12d 0 2024-04-24T22:45:41.686 INFO:tasks.workunit.client.0.smithi063.stdout:3/876: mkdir da/d18/d47/d4d/d9b/dde/df9/d132 0 2024-04-24T22:45:41.687 INFO:tasks.workunit.client.0.smithi063.stdout:0/812: link d6/d64/da0/f84 d6/d64/d21/d25/d62/d63/d89/f113 0 2024-04-24T22:45:41.687 INFO:tasks.workunit.client.0.smithi063.stdout:3/877: chown da/d18/d47/d4d/d62/cdc 435 1 2024-04-24T22:45:41.689 INFO:tasks.workunit.client.0.smithi063.stdout:2/978: mknod d0/d2/d2d/d31/d89/d7/d86/c12f 0 2024-04-24T22:45:41.691 INFO:tasks.workunit.client.0.smithi063.stdout:5/829: unlink d5/d11/d23/d2d/d62/ffe 0 2024-04-24T22:45:41.693 INFO:tasks.workunit.client.0.smithi063.stdout:4/848: dread d2/d8/f3e [0,4194304] 0 2024-04-24T22:45:41.694 INFO:tasks.workunit.client.0.smithi063.stdout:9/899: rename d6/d2f/dcd/d53/dd5 to d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/d12e 0 2024-04-24T22:45:41.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/803: getdents d0/d4/de/d6f/d2e/d76/d9f/dbd 0 2024-04-24T22:45:41.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/830: read d5/d1d/f64 [356743,36212] 0 2024-04-24T22:45:41.700 INFO:tasks.workunit.client.0.smithi063.stdout:4/849: mkdir d2/d19/d82/d96/d11a 0 2024-04-24T22:45:41.701 INFO:tasks.workunit.client.0.smithi063.stdout:2/979: rename d0/d2/lcf to d0/d2/d6c/l130 0 2024-04-24T22:45:41.701 INFO:tasks.workunit.client.0.smithi063.stdout:7/804: rename d0/d4/de to d0/d4/de/d6f/d2e/d55/d106/d10d 22 2024-04-24T22:45:41.703 INFO:tasks.workunit.client.0.smithi063.stdout:6/940: write db/d1d/d20/d29/dab/dd0/d105/f134 [4769704,17401] 0 2024-04-24T22:45:41.705 INFO:tasks.workunit.client.1.smithi192.stdout:8/697: unlink db/d43/d71/fd6 0 2024-04-24T22:45:41.705 INFO:tasks.workunit.client.1.smithi192.stdout:7/793: creat d2/db/d21/d42/d67/dae/dbb/ffb x:0 0 0 2024-04-24T22:45:41.706 INFO:tasks.workunit.client.0.smithi063.stdout:5/831: creat d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/f11d x:0 0 0 2024-04-24T22:45:41.706 INFO:tasks.workunit.client.1.smithi192.stdout:6/663: dread d0/d8/d29/d5a/f60 [4194304,4194304] 0 2024-04-24T22:45:41.706 INFO:tasks.workunit.client.1.smithi192.stdout:2/807: truncate d4/f13 3121735 0 2024-04-24T22:45:41.707 INFO:tasks.workunit.client.1.smithi192.stdout:7/794: write d2/db/d21/d42/d67/dae/fc8 [1730544,579] 0 2024-04-24T22:45:41.710 INFO:tasks.workunit.client.0.smithi063.stdout:1/922: truncate d3/db5/d21/d9f/dd5/f102 1990458 0 2024-04-24T22:45:41.711 INFO:tasks.workunit.client.1.smithi192.stdout:5/605: mkdir d0/d13/dd1 0 2024-04-24T22:45:41.712 INFO:tasks.workunit.client.0.smithi063.stdout:1/923: read - d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f123 zero size 2024-04-24T22:45:41.713 INFO:tasks.workunit.client.0.smithi063.stdout:4/850: rename d2/d9f/fbd to d2/d8/dee/f11b 0 2024-04-24T22:45:41.714 INFO:tasks.workunit.client.0.smithi063.stdout:3/878: link da/d18/d47/c12f da/d18/d47/d4d/c133 0 2024-04-24T22:45:41.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/879: chown da/d39/f98 95 1 2024-04-24T22:45:41.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/880: stat da/d18/d3a/da1 0 2024-04-24T22:45:41.716 INFO:tasks.workunit.client.0.smithi063.stdout:7/805: dread d0/f7 [0,4194304] 0 2024-04-24T22:45:41.717 INFO:tasks.workunit.client.0.smithi063.stdout:6/941: fsync db/d1d/d20/d29/d5d/f99 0 2024-04-24T22:45:41.718 INFO:tasks.workunit.client.0.smithi063.stdout:7/806: stat d0/d4/de/d6f/d5c/d9c/ffa 0 2024-04-24T22:45:41.719 INFO:tasks.workunit.client.0.smithi063.stdout:8/921: truncate d1/d11/d32/d38/f67 575596 0 2024-04-24T22:45:41.719 INFO:tasks.workunit.client.0.smithi063.stdout:7/807: stat d0/d4/de/f7c 0 2024-04-24T22:45:41.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/881: dread da/d18/d9e/fc4 [0,4194304] 0 2024-04-24T22:45:41.720 INFO:tasks.workunit.client.1.smithi192.stdout:8/698: mknod db/dc/d29/cdc 0 2024-04-24T22:45:41.720 INFO:tasks.workunit.client.0.smithi063.stdout:0/813: write d6/d64/d24/f73 [1825954,43293] 0 2024-04-24T22:45:41.724 INFO:tasks.workunit.client.0.smithi063.stdout:6/942: dread db/d1d/d8d/dfc/f10c [0,4194304] 0 2024-04-24T22:45:41.725 INFO:tasks.workunit.client.1.smithi192.stdout:7/795: symlink d2/db/d21/d42/d67/d9b/d6e/db5/db6/lfc 0 2024-04-24T22:45:41.730 INFO:tasks.workunit.client.0.smithi063.stdout:5/832: dread d5/d11/d23/d2d/d62/d73/d118/d44/f46 [0,4194304] 0 2024-04-24T22:45:41.732 INFO:tasks.workunit.client.1.smithi192.stdout:7/796: dwrite d2/db/d21/ff8 [0,4194304] 0 2024-04-24T22:45:41.732 INFO:tasks.workunit.client.1.smithi192.stdout:2/808: creat d4/d9/d10/d6c/d8c/da5/de2/f106 x:0 0 0 2024-04-24T22:45:41.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/882: dread da/d48/fd7 [4194304,4194304] 0 2024-04-24T22:45:41.739 INFO:tasks.workunit.client.0.smithi063.stdout:2/980: creat d0/f131 x:0 0 0 2024-04-24T22:45:41.741 INFO:tasks.workunit.client.0.smithi063.stdout:8/922: mkdir d1/d2/d17/d124 0 2024-04-24T22:45:41.742 INFO:tasks.workunit.client.1.smithi192.stdout:8/699: write db/dc/d29/f47 [1880701,72352] 0 2024-04-24T22:45:41.743 INFO:tasks.workunit.client.1.smithi192.stdout:7/797: unlink d2/f91 0 2024-04-24T22:45:41.745 INFO:tasks.workunit.client.1.smithi192.stdout:2/809: rmdir d4/d9/d10 39 2024-04-24T22:45:41.747 INFO:tasks.workunit.client.0.smithi063.stdout:8/923: dwrite d1/d2/d3/db6/fbb [0,4194304] 0 2024-04-24T22:45:41.747 INFO:tasks.workunit.client.0.smithi063.stdout:0/814: mknod d6/d64/da0/d98/c114 0 2024-04-24T22:45:41.748 INFO:tasks.workunit.client.0.smithi063.stdout:0/815: readlink d6/d64/d24/lac 0 2024-04-24T22:45:41.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/900: getdents d6/d2f/dcd/d1f/d20 0 2024-04-24T22:45:41.750 INFO:tasks.workunit.client.1.smithi192.stdout:8/700: mknod db/d43/d76/d92/cdd 0 2024-04-24T22:45:41.752 INFO:tasks.workunit.client.1.smithi192.stdout:7/798: dread d2/db/d21/d42/d67/d9b/d6e/f9e [0,4194304] 0 2024-04-24T22:45:41.754 INFO:tasks.workunit.client.1.smithi192.stdout:2/810: chown d4/d9/d10/da7/fbb 1721613402 1 2024-04-24T22:45:41.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/851: mknod d2/d8/d3c/de9/c11c 0 2024-04-24T22:45:41.760 INFO:tasks.workunit.client.1.smithi192.stdout:7/799: dread d2/db/d25/f96 [0,4194304] 0 2024-04-24T22:45:41.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/701: symlink db/d43/d7c/d85/db4/lde 0 2024-04-24T22:45:41.766 INFO:tasks.workunit.client.0.smithi063.stdout:5/833: write d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f48 [4291903,102407] 0 2024-04-24T22:45:41.766 INFO:tasks.workunit.client.1.smithi192.stdout:5/606: getdents d0/d13/d19/d64/d5f/d2c 0 2024-04-24T22:45:41.769 INFO:tasks.workunit.client.1.smithi192.stdout:8/702: fdatasync db/f56 0 2024-04-24T22:45:41.770 INFO:tasks.workunit.client.1.smithi192.stdout:8/703: truncate db/d75/fc9 36728 0 2024-04-24T22:45:41.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/901: chown d6/d2f/dcd/db/d6b/d4f/d59/lf3 5571477 1 2024-04-24T22:45:41.775 INFO:tasks.workunit.client.0.smithi063.stdout:6/943: mknod db/d1d/d8d/c13e 0 2024-04-24T22:45:41.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/811: getdents d4/d9/d9f 0 2024-04-24T22:45:41.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/704: dwrite db/dc/d22/d31/d33/d8f/da6/fb2 [0,4194304] 0 2024-04-24T22:45:41.777 INFO:tasks.workunit.client.0.smithi063.stdout:1/924: creat d3/db5/f12a x:0 0 0 2024-04-24T22:45:41.778 INFO:tasks.workunit.client.1.smithi192.stdout:8/705: readlink db/dc/d22/l57 0 2024-04-24T22:45:41.780 INFO:tasks.workunit.client.0.smithi063.stdout:4/852: symlink d2/d8/d5c/l11d 0 2024-04-24T22:45:41.785 INFO:tasks.workunit.client.1.smithi192.stdout:5/607: write d0/d13/f43 [1326183,111400] 0 2024-04-24T22:45:41.787 INFO:tasks.workunit.client.1.smithi192.stdout:2/812: symlink d4/d9/d10/d6c/d8c/l107 0 2024-04-24T22:45:41.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/706: rmdir db/d43/d71 39 2024-04-24T22:45:41.790 INFO:tasks.workunit.client.0.smithi063.stdout:5/834: creat d5/d11/d23/d2d/d62/d81/f11e x:0 0 0 2024-04-24T22:45:41.791 INFO:tasks.workunit.client.0.smithi063.stdout:8/924: creat d1/d2/d3/d86/f125 x:0 0 0 2024-04-24T22:45:41.791 INFO:tasks.workunit.client.1.smithi192.stdout:7/800: link d2/db/f60 d2/db/d21/d42/d67/d9b/dc1/ffd 0 2024-04-24T22:45:41.792 INFO:tasks.workunit.client.0.smithi063.stdout:8/925: chown d1/d11/d32 103 1 2024-04-24T22:45:41.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/981: truncate d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa 675132 0 2024-04-24T22:45:41.796 INFO:tasks.workunit.client.0.smithi063.stdout:9/902: fsync d6/d2f/dcd/db/d6b/f5c 0 2024-04-24T22:45:41.797 INFO:tasks.workunit.client.0.smithi063.stdout:3/883: dwrite da/d18/d115/f12a [0,4194304] 0 2024-04-24T22:45:41.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/925: truncate d3/db5/f71 5028960 0 2024-04-24T22:45:41.799 INFO:tasks.workunit.client.0.smithi063.stdout:4/853: rename d2/d19/d82/d96/d11a to d2/d8/d36/d118/d11e 0 2024-04-24T22:45:41.800 INFO:tasks.workunit.client.1.smithi192.stdout:7/801: rmdir d2/db/d21/d42/d67/dae 39 2024-04-24T22:45:41.801 INFO:tasks.workunit.client.0.smithi063.stdout:0/816: truncate d6/d64/d21/d25/d31/f35 248750 0 2024-04-24T22:45:41.810 INFO:tasks.workunit.client.1.smithi192.stdout:4/718: sync 2024-04-24T22:45:41.812 INFO:tasks.workunit.client.1.smithi192.stdout:5/608: link d0/d13/d19/d64/d5f/f20 d0/d13/d19/d83/dad/fd2 0 2024-04-24T22:45:41.815 INFO:tasks.workunit.client.1.smithi192.stdout:8/707: write db/dc/d22/f35 [443278,34925] 0 2024-04-24T22:45:41.815 INFO:tasks.workunit.client.1.smithi192.stdout:2/813: creat d4/d9/d37/de5/df3/f108 x:0 0 0 2024-04-24T22:45:41.816 INFO:tasks.workunit.client.0.smithi063.stdout:8/926: chown d1/d11/d32/d38/db7/dc1 199911 1 2024-04-24T22:45:41.817 INFO:tasks.workunit.client.1.smithi192.stdout:2/814: truncate d4/d9/d10/d6c/fb5 1156985 0 2024-04-24T22:45:41.820 INFO:tasks.workunit.client.1.smithi192.stdout:7/802: truncate d2/db/d36/f73 815282 0 2024-04-24T22:45:41.821 INFO:tasks.workunit.client.1.smithi192.stdout:7/803: chown d2/db/d21/dd5/cf2 0 1 2024-04-24T22:45:41.824 INFO:tasks.workunit.client.1.smithi192.stdout:5/609: readlink d0/d13/d19/d71/d7f/d9a/lc8 0 2024-04-24T22:45:41.824 INFO:tasks.workunit.client.0.smithi063.stdout:5/835: write d5/d11/d23/d2d/d62/fe6 [578638,95588] 0 2024-04-24T22:45:41.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/610: fdatasync d0/d13/d19/d64/d4f/d6c/fae 0 2024-04-24T22:45:41.826 INFO:tasks.workunit.client.1.smithi192.stdout:8/708: fdatasync db/dc/d22/d31/d33/f6d 0 2024-04-24T22:45:41.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/903: read d6/d2f/dcd/f17 [1298497,129958] 0 2024-04-24T22:45:41.828 INFO:tasks.workunit.client.0.smithi063.stdout:3/884: creat da/d18/dfb/f134 x:0 0 0 2024-04-24T22:45:41.829 INFO:tasks.workunit.client.1.smithi192.stdout:2/815: mkdir d4/d9/d10/d6c/d95/db7/dbf/d109 0 2024-04-24T22:45:41.831 INFO:tasks.workunit.client.0.smithi063.stdout:2/982: write d0/dd/d72/d77/fce [641516,47655] 0 2024-04-24T22:45:41.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/719: mkdir d2/db/d5d/dfc 0 2024-04-24T22:45:41.834 INFO:tasks.workunit.client.0.smithi063.stdout:9/904: dread d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/d12e/f104 [0,4194304] 0 2024-04-24T22:45:41.834 INFO:tasks.workunit.client.0.smithi063.stdout:2/983: truncate d0/dd/d72/d77/fce 1594841 0 2024-04-24T22:45:41.834 INFO:tasks.workunit.client.1.smithi192.stdout:4/720: write d2/db/f7a [2294027,69424] 0 2024-04-24T22:45:41.834 INFO:tasks.workunit.client.1.smithi192.stdout:5/611: mknod d0/d13/d19/d71/d88/cd3 0 2024-04-24T22:45:41.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/854: write d2/d19/d33/dc1/d50/dc3/f101 [406421,101117] 0 2024-04-24T22:45:41.840 INFO:tasks.workunit.client.1.smithi192.stdout:2/816: rename d4/d9/d10/d6c/d95/cb1 to d4/d9/d37/c10a 0 2024-04-24T22:45:41.840 INFO:tasks.workunit.client.1.smithi192.stdout:7/804: dwrite d2/f2c [0,4194304] 0 2024-04-24T22:45:41.841 INFO:tasks.workunit.client.1.smithi192.stdout:7/805: stat d2/db/d21/dd5/cf2 0 2024-04-24T22:45:41.841 INFO:tasks.workunit.client.1.smithi192.stdout:7/806: chown d2/db/d49/cb7 322 1 2024-04-24T22:45:41.842 INFO:tasks.workunit.client.1.smithi192.stdout:7/807: chown d2/db/d21/d42/d5b/d63 3595 1 2024-04-24T22:45:41.847 INFO:tasks.workunit.client.1.smithi192.stdout:8/709: dwrite db/dc/d22/d31/f50 [0,4194304] 0 2024-04-24T22:45:41.849 INFO:tasks.workunit.client.0.smithi063.stdout:3/885: truncate da/d18/d47/d7b/fc5 1053157 0 2024-04-24T22:45:41.850 INFO:tasks.workunit.client.1.smithi192.stdout:7/808: mkdir d2/db/d21/d42/d67/da8/dc7/dfe 0 2024-04-24T22:45:41.852 INFO:tasks.workunit.client.0.smithi063.stdout:1/926: creat d3/db5/d21/d9f/dd5/d106/d39/d115/f12b x:0 0 0 2024-04-24T22:45:41.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/808: sync 2024-04-24T22:45:41.858 INFO:tasks.workunit.client.1.smithi192.stdout:5/612: creat d0/d13/d19/d64/d5f/fd4 x:0 0 0 2024-04-24T22:45:41.859 INFO:tasks.workunit.client.0.smithi063.stdout:2/984: creat d0/d2/d63/f132 x:0 0 0 2024-04-24T22:45:41.862 INFO:tasks.workunit.client.0.smithi063.stdout:4/855: dread d2/d8/f31 [0,4194304] 0 2024-04-24T22:45:41.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/817: symlink d6/d64/d21/d25/d62/d63/d89/d95/df6/l115 0 2024-04-24T22:45:41.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/818: fsync d6/d64/d24/d42/dd4/ff4 0 2024-04-24T22:45:41.864 INFO:tasks.workunit.client.1.smithi192.stdout:7/809: creat d2/db/d21/d42/d67/d9b/d6e/db5/db6/fff x:0 0 0 2024-04-24T22:45:41.866 INFO:tasks.workunit.client.1.smithi192.stdout:7/810: write d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb9 [3837675,14492] 0 2024-04-24T22:45:41.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/613: write d0/f18 [269255,115237] 0 2024-04-24T22:45:41.870 INFO:tasks.workunit.client.0.smithi063.stdout:6/944: sync 2024-04-24T22:45:41.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/614: read - d0/d1b/fc7 zero size 2024-04-24T22:45:41.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/721: rename d2/db/l12 to d2/d9/lfd 0 2024-04-24T22:45:41.875 INFO:tasks.workunit.client.1.smithi192.stdout:8/710: dwrite db/dc/d22/d31/d33/f6d [0,4194304] 0 2024-04-24T22:45:41.876 INFO:tasks.workunit.client.0.smithi063.stdout:9/905: mknod d6/d2f/d42/dda/c12f 0 2024-04-24T22:45:41.877 INFO:tasks.workunit.client.1.smithi192.stdout:7/811: chown d2/db/d21/d42/d67/d9b/d6e/d77/cf5 10573 1 2024-04-24T22:45:41.880 INFO:tasks.workunit.client.0.smithi063.stdout:2/985: fsync d0/dd/d72/d77/dda/d7c/fb9 0 2024-04-24T22:45:41.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/809: read d0/d4/de/d17/d28/f4a [1126766,95876] 0 2024-04-24T22:45:41.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/722: fdatasync d2/f1f 0 2024-04-24T22:45:41.882 INFO:tasks.workunit.client.0.smithi063.stdout:2/986: dread - d0/d2/d63/f128 zero size 2024-04-24T22:45:41.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/856: dread d2/d8/d40/f68 [0,4194304] 0 2024-04-24T22:45:41.883 INFO:tasks.workunit.client.1.smithi192.stdout:4/723: write d2/db/d10/fb6 [848687,16465] 0 2024-04-24T22:45:41.885 INFO:tasks.workunit.client.1.smithi192.stdout:2/817: rename d4/d9/d10/d6c/d8c/fab to d4/d9/d37/f10b 0 2024-04-24T22:45:41.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/927: link d1/d2/d17/dd1/c109 d1/d11/d8b/dd6/d11d/c126 0 2024-04-24T22:45:41.887 INFO:tasks.workunit.client.0.smithi063.stdout:1/927: dwrite d3/db5/d21/d9f/dd5/d106/d39/d5a/f8f [0,4194304] 0 2024-04-24T22:45:41.888 INFO:tasks.workunit.client.1.smithi192.stdout:8/711: unlink db/dc/d22/d31/f50 0 2024-04-24T22:45:41.889 INFO:tasks.workunit.client.0.smithi063.stdout:1/928: dread - d3/db5/d21/d9f/dd5/d106/d39/d5a/f97 zero size 2024-04-24T22:45:41.894 INFO:tasks.workunit.client.0.smithi063.stdout:8/928: dread d1/d2/d3/f14 [4194304,4194304] 0 2024-04-24T22:45:41.894 INFO:tasks.workunit.client.0.smithi063.stdout:3/886: symlink da/d39/d94/db3/df3/l135 0 2024-04-24T22:45:41.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/887: fsync da/d18/d9e/fed 0 2024-04-24T22:45:41.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/888: fsync da/d18/dfb/f134 0 2024-04-24T22:45:41.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/945: creat db/d1d/d20/d11e/f13f x:0 0 0 2024-04-24T22:45:41.897 INFO:tasks.workunit.client.1.smithi192.stdout:4/724: mkdir d2/db/d75/da1/d36/dfe 0 2024-04-24T22:45:41.899 INFO:tasks.workunit.client.0.smithi063.stdout:9/906: rmdir d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d11e 39 2024-04-24T22:45:41.905 INFO:tasks.workunit.client.0.smithi063.stdout:0/819: mkdir d6/d64/d21/d25/d62/d63/d89/d116 0 2024-04-24T22:45:41.907 INFO:tasks.workunit.client.0.smithi063.stdout:0/820: write d6/d64/d21/d25/d62/d63/d7c/f10b [371370,124203] 0 2024-04-24T22:45:41.908 INFO:tasks.workunit.client.0.smithi063.stdout:0/821: stat d6/d64/d21/d25/d62/f10f 0 2024-04-24T22:45:41.909 INFO:tasks.workunit.client.1.smithi192.stdout:5/615: dwrite d0/d13/d3b/d2f/f84 [0,4194304] 0 2024-04-24T22:45:41.910 INFO:tasks.workunit.client.0.smithi063.stdout:3/889: dread da/d39/f98 [0,4194304] 0 2024-04-24T22:45:41.911 INFO:tasks.workunit.client.0.smithi063.stdout:4/857: creat d2/d9f/d117/f11f x:0 0 0 2024-04-24T22:45:41.912 INFO:tasks.workunit.client.1.smithi192.stdout:8/712: fdatasync f5 0 2024-04-24T22:45:41.912 INFO:tasks.workunit.client.1.smithi192.stdout:8/713: chown db/l19 190 1 2024-04-24T22:45:41.914 INFO:tasks.workunit.client.0.smithi063.stdout:5/836: link d5/d11/d23/d2d/d62/d73/d118/d36/l90 d5/d11/d23/d2d/d62/d81/da4/da6/l11f 0 2024-04-24T22:45:41.917 INFO:tasks.workunit.client.0.smithi063.stdout:1/929: rename d3/db5/d21/d9f/dd5/d106/f8c to d3/db5/d21/d9f/dd5/d106/d39/d5a/d10e/f12c 0 2024-04-24T22:45:41.917 INFO:tasks.workunit.client.0.smithi063.stdout:5/837: read d5/d11/d23/d2d/fd1 [2632099,121689] 0 2024-04-24T22:45:41.919 INFO:tasks.workunit.client.0.smithi063.stdout:5/838: fdatasync d5/d11/d23/d2d/d62/d81/f11e 0 2024-04-24T22:45:41.919 INFO:tasks.workunit.client.0.smithi063.stdout:8/929: unlink d1/d2/fd 0 2024-04-24T22:45:41.919 INFO:tasks.workunit.client.0.smithi063.stdout:1/930: chown d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/fd0 0 1 2024-04-24T22:45:41.920 INFO:tasks.workunit.client.1.smithi192.stdout:2/818: link d4/d9/d10/da7/laf d4/d9/d10/d6c/d95/l10c 0 2024-04-24T22:45:41.920 INFO:tasks.workunit.client.1.smithi192.stdout:5/616: mkdir d0/d1b/d62/dc1/dd5 0 2024-04-24T22:45:41.920 INFO:tasks.workunit.client.1.smithi192.stdout:7/812: dread d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:41.922 INFO:tasks.workunit.client.1.smithi192.stdout:7/813: write d2/f2c [1650101,125238] 0 2024-04-24T22:45:41.926 INFO:tasks.workunit.client.0.smithi063.stdout:7/810: mkdir d0/d4/de/d6f/d2e/d55/d106/d10e 0 2024-04-24T22:45:41.927 INFO:tasks.workunit.client.0.smithi063.stdout:0/822: mknod d6/d64/d21/d25/d31/da3/c117 0 2024-04-24T22:45:41.930 INFO:tasks.workunit.client.0.smithi063.stdout:4/858: creat d2/d8/d40/d51/dd5/df6/f120 x:0 0 0 2024-04-24T22:45:41.931 INFO:tasks.workunit.client.0.smithi063.stdout:0/823: dread d6/dd/f6f [0,4194304] 0 2024-04-24T22:45:41.939 INFO:tasks.workunit.client.0.smithi063.stdout:6/946: mkdir db/d140 0 2024-04-24T22:45:41.940 INFO:tasks.workunit.client.0.smithi063.stdout:9/907: creat d6/d2f/dcd/db/d1c/d78/d9a/d129/f130 x:0 0 0 2024-04-24T22:45:41.941 INFO:tasks.workunit.client.0.smithi063.stdout:9/908: stat d6/d2f/dcd/db/d6b/d4f/dad/d36/c4e 0 2024-04-24T22:45:41.942 INFO:tasks.workunit.client.0.smithi063.stdout:2/987: rmdir d0/d2/d2f/ddc 0 2024-04-24T22:45:41.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/839: mkdir d5/d11/d23/d120 0 2024-04-24T22:45:41.955 INFO:tasks.workunit.client.1.smithi192.stdout:8/714: mkdir db/ddf 0 2024-04-24T22:45:41.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/811: dread d0/d4/de/d17/d35/d69/fb0 [0,4194304] 0 2024-04-24T22:45:41.956 INFO:tasks.workunit.client.0.smithi063.stdout:1/931: write d3/f6c [2012013,105903] 0 2024-04-24T22:45:41.957 INFO:tasks.workunit.client.0.smithi063.stdout:8/930: mknod d1/d21/df3/c127 0 2024-04-24T22:45:41.959 INFO:tasks.workunit.client.0.smithi063.stdout:6/947: chown db/d1d/d2d/d48/f51 18124786 1 2024-04-24T22:45:41.960 INFO:tasks.workunit.client.0.smithi063.stdout:9/909: mkdir d6/d2f/dcd/db/d1c/d78/d9a/ddd/d131 0 2024-04-24T22:45:41.962 INFO:tasks.workunit.client.0.smithi063.stdout:2/988: truncate d0/d2/d2d/d31/d89/f80 925974 0 2024-04-24T22:45:41.963 INFO:tasks.workunit.client.0.smithi063.stdout:8/931: dwrite d1/d2/d3/db6/fbb [0,4194304] 0 2024-04-24T22:45:41.964 INFO:tasks.workunit.client.0.smithi063.stdout:2/989: readlink d0/dd/d72/d77/dda/d7c/l102 0 2024-04-24T22:45:41.964 INFO:tasks.workunit.client.0.smithi063.stdout:8/932: dread - d1/d22/d33/da6/d54/d79/fda zero size 2024-04-24T22:45:41.967 INFO:tasks.workunit.client.0.smithi063.stdout:8/933: dread - d1/d22/d33/f108 zero size 2024-04-24T22:45:41.969 INFO:tasks.workunit.client.0.smithi063.stdout:0/824: write d6/d3a/fc5 [4956693,97349] 0 2024-04-24T22:45:41.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/819: getdents d4/d9 0 2024-04-24T22:45:41.972 INFO:tasks.workunit.client.1.smithi192.stdout:2/820: fdatasync d4/f5e 0 2024-04-24T22:45:41.973 INFO:tasks.workunit.client.0.smithi063.stdout:4/859: dwrite d2/d19/d33/dc1/f4b [0,4194304] 0 2024-04-24T22:45:41.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/860: dread - d2/d8/dd4/dda/f112 zero size 2024-04-24T22:45:41.975 INFO:tasks.workunit.client.1.smithi192.stdout:5/617: dwrite d0/d13/d19/d64/d4f/d6c/fb5 [0,4194304] 0 2024-04-24T22:45:41.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/861: chown d2/d19/d33/c119 16937 1 2024-04-24T22:45:41.976 INFO:tasks.workunit.client.1.smithi192.stdout:8/715: write db/dc/f1b [1094190,20222] 0 2024-04-24T22:45:41.982 INFO:tasks.workunit.client.0.smithi063.stdout:6/948: read db/d1c/f58 [431710,76403] 0 2024-04-24T22:45:41.983 INFO:tasks.workunit.client.0.smithi063.stdout:1/932: dwrite d3/db5/d21/d9f/dd5/d106/d11/f41 [4194304,4194304] 0 2024-04-24T22:45:41.985 INFO:tasks.workunit.client.0.smithi063.stdout:6/949: truncate db/d85/dd2/de6/f12b 654972 0 2024-04-24T22:45:41.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/585: sync 2024-04-24T22:45:41.988 INFO:tasks.workunit.client.1.smithi192.stdout:9/794: sync 2024-04-24T22:45:41.992 INFO:tasks.workunit.client.1.smithi192.stdout:7/814: write d2/db/f45 [768970,113482] 0 2024-04-24T22:45:41.992 INFO:tasks.workunit.client.1.smithi192.stdout:9/795: write d3/d59/ff5 [222937,90115] 0 2024-04-24T22:45:41.993 INFO:tasks.workunit.client.1.smithi192.stdout:8/716: dwrite db/dc/d22/d31/d33/f6d [0,4194304] 0 2024-04-24T22:45:41.997 INFO:tasks.workunit.client.0.smithi063.stdout:2/990: fsync d0/d2/d2d/d31/d89/d7/d86/da4/dd9/fe6 0 2024-04-24T22:45:41.999 INFO:tasks.workunit.client.0.smithi063.stdout:2/991: chown d0/d2/d2f/dba/d11b/d12e 310550 1 2024-04-24T22:45:41.999 INFO:tasks.workunit.client.1.smithi192.stdout:2/821: rename d4/d9/d10/d6c/d95/l10c to d4/d9/d10/d6c/d95/db7/l10d 0 2024-04-24T22:45:42.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/890: link da/d18/c44 da/d39/d94/de4/c136 0 2024-04-24T22:45:42.002 INFO:tasks.workunit.client.1.smithi192.stdout:7/815: symlink d2/db/d2a/d57/l100 0 2024-04-24T22:45:42.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/812: write d0/d4/fb [2458210,103772] 0 2024-04-24T22:45:42.003 INFO:tasks.workunit.client.0.smithi063.stdout:3/891: fsync da/d39/d94/f110 0 2024-04-24T22:45:42.004 INFO:tasks.workunit.client.1.smithi192.stdout:9/796: truncate d3/d59/dbf/fd9 1216448 0 2024-04-24T22:45:42.005 INFO:tasks.workunit.client.0.smithi063.stdout:3/892: chown da/d18/d9e/da3 24788959 1 2024-04-24T22:45:42.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/618: rename d0/d13/d19/d71/d7f/c97 to d0/d5b/cd6 0 2024-04-24T22:45:42.007 INFO:tasks.workunit.client.0.smithi063.stdout:5/840: dwrite d5/d11/d76/f9a [0,4194304] 0 2024-04-24T22:45:42.007 INFO:tasks.workunit.client.1.smithi192.stdout:2/822: symlink d4/d9/d10/d6c/d8c/l10e 0 2024-04-24T22:45:42.008 INFO:tasks.workunit.client.1.smithi192.stdout:1/586: fdatasync d9/d15/d2d/d31/f99 0 2024-04-24T22:45:42.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/797: dwrite d3/f21 [0,4194304] 0 2024-04-24T22:45:42.012 INFO:tasks.workunit.client.0.smithi063.stdout:1/933: creat d3/db5/d21/d9f/dd5/d106/d11/dc6/f12d x:0 0 0 2024-04-24T22:45:42.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/950: creat db/d1d/d20/d11e/f141 x:0 0 0 2024-04-24T22:45:42.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/951: stat db/d1d/d20/f71 0 2024-04-24T22:45:42.016 INFO:tasks.workunit.client.0.smithi063.stdout:6/952: fdatasync db/d1c/dc3/f115 0 2024-04-24T22:45:42.028 INFO:tasks.workunit.client.1.smithi192.stdout:5/619: mkdir d0/d13/d3b/dd7 0 2024-04-24T22:45:42.030 INFO:tasks.workunit.client.1.smithi192.stdout:8/717: truncate db/f65 925873 0 2024-04-24T22:45:42.031 INFO:tasks.workunit.client.0.smithi063.stdout:3/893: truncate da/f15 4868499 0 2024-04-24T22:45:42.032 INFO:tasks.workunit.client.1.smithi192.stdout:7/816: write d2/db/d21/d42/f4d [998204,100883] 0 2024-04-24T22:45:42.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/817: write d2/db/d21/d42/d67/d9b/d6e/fe4 [84329,94423] 0 2024-04-24T22:45:42.035 INFO:tasks.workunit.client.0.smithi063.stdout:1/934: chown d3/de0/d79/f127 5400 1 2024-04-24T22:45:42.036 INFO:tasks.workunit.client.1.smithi192.stdout:5/620: chown d0/d13/f39 21082 1 2024-04-24T22:45:42.037 INFO:tasks.workunit.client.0.smithi063.stdout:9/910: getdents d6/d2f/dcd/d1f/d20/d2a/d4b/d5e/d9d/d11e/d12a 0 2024-04-24T22:45:42.037 INFO:tasks.workunit.client.0.smithi063.stdout:0/825: truncate d6/da/f3d 2015639 0 2024-04-24T22:45:42.038 INFO:tasks.workunit.client.0.smithi063.stdout:6/953: truncate db/d1d/f4c 571395 0 2024-04-24T22:45:42.039 INFO:tasks.workunit.client.0.smithi063.stdout:0/826: write d6/dd/f4d [2729464,88961] 0 2024-04-24T22:45:42.039 INFO:tasks.workunit.client.1.smithi192.stdout:8/718: unlink db/dc/d29/cb5 0 2024-04-24T22:45:42.043 INFO:tasks.workunit.client.1.smithi192.stdout:1/587: rename d9/d15/d2d/c8f to d9/dd/cc3 0 2024-04-24T22:45:42.047 INFO:tasks.workunit.client.1.smithi192.stdout:9/798: write d3/d59/dbf/fd9 [466634,23295] 0 2024-04-24T22:45:42.048 INFO:tasks.workunit.client.1.smithi192.stdout:2/823: dwrite d4/d9/d10/da7/df5/ff9 [0,4194304] 0 2024-04-24T22:45:42.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/818: creat d2/db/d21/d42/d5b/d63/f101 x:0 0 0 2024-04-24T22:45:42.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/841: mkdir d5/d121 0 2024-04-24T22:45:42.052 INFO:tasks.workunit.client.0.smithi063.stdout:3/894: dwrite da/f2e [0,4194304] 0 2024-04-24T22:45:42.054 INFO:tasks.workunit.client.0.smithi063.stdout:4/862: sync 2024-04-24T22:45:42.055 INFO:tasks.workunit.client.1.smithi192.stdout:2/824: dwrite d4/d9/d9f/fd7 [0,4194304] 0 2024-04-24T22:45:42.056 INFO:tasks.workunit.client.0.smithi063.stdout:4/863: read d2/d19/d33/dc1/f6d [1088671,74291] 0 2024-04-24T22:45:42.061 INFO:tasks.workunit.client.0.smithi063.stdout:1/935: creat d3/de0/d79/d96/f12e x:0 0 0 2024-04-24T22:45:42.065 INFO:tasks.workunit.client.1.smithi192.stdout:2/825: dwrite d4/fe1 [0,4194304] 0 2024-04-24T22:45:42.071 INFO:tasks.workunit.client.0.smithi063.stdout:9/911: rename d6/d2f/dcd/d1f/d20/d2a/d4b to d6/d2f/dcd/d53/dff/d132 0 2024-04-24T22:45:42.076 INFO:tasks.workunit.client.0.smithi063.stdout:6/954: fsync db/d1d/d20/d3f/d5f/d93/f62 0 2024-04-24T22:45:42.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/955: chown db/d1d/d20/d3f/d7f/d68 715034 1 2024-04-24T22:45:42.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/819: unlink d2/db/d21/d42/d5b/d63/f6d 0 2024-04-24T22:45:42.080 INFO:tasks.workunit.client.0.smithi063.stdout:2/992: creat d0/d2/d27/f133 x:0 0 0 2024-04-24T22:45:42.080 INFO:tasks.workunit.client.0.smithi063.stdout:2/993: fdatasync d0/d2/d2d/d31/f5b 0 2024-04-24T22:45:42.081 INFO:tasks.workunit.client.1.smithi192.stdout:5/621: chown d0/d13/d53/f70 20949 1 2024-04-24T22:45:42.083 INFO:tasks.workunit.client.0.smithi063.stdout:8/934: getdents d1/d11/d3c 0 2024-04-24T22:45:42.083 INFO:tasks.workunit.client.1.smithi192.stdout:8/719: write db/dc/d22/d31/d33/d81/d8a/faa [538148,34522] 0 2024-04-24T22:45:42.083 INFO:tasks.workunit.client.0.smithi063.stdout:8/935: write d1/d22/d33/f108 [722219,29147] 0 2024-04-24T22:45:42.087 INFO:tasks.workunit.client.1.smithi192.stdout:1/588: mknod d9/cc4 0 2024-04-24T22:45:42.088 INFO:tasks.workunit.client.1.smithi192.stdout:2/826: symlink d4/d9/d10/d6c/d8c/da5/de2/l10f 0 2024-04-24T22:45:42.089 INFO:tasks.workunit.client.0.smithi063.stdout:7/813: link d0/d4/de/d17/d35/d6c/d7a/lbf d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/l10f 0 2024-04-24T22:45:42.089 INFO:tasks.workunit.client.1.smithi192.stdout:9/799: mkdir d3/dd/d53/dab/dcf/d10d 0 2024-04-24T22:45:42.090 INFO:tasks.workunit.client.0.smithi063.stdout:8/936: dwrite d1/d2/d3/d86/ff2 [0,4194304] 0 2024-04-24T22:45:42.097 INFO:tasks.workunit.client.0.smithi063.stdout:8/937: dwrite d1/d11/f15 [0,4194304] 0 2024-04-24T22:45:42.124 INFO:tasks.workunit.client.1.smithi192.stdout:2/827: dread d4/d9/d10/d6c/d6f/feb [4194304,4194304] 0 2024-04-24T22:45:42.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/828: dwrite d4/d9/d10/d6c/d6f/da8/f105 [0,4194304] 0 2024-04-24T22:45:42.129 INFO:tasks.workunit.client.1.smithi192.stdout:2/829: chown d4/d9/d37/l3a 2472779 1 2024-04-24T22:45:42.130 INFO:tasks.workunit.client.1.smithi192.stdout:2/830: stat d4/d9/d10/d6c/d95 0 2024-04-24T22:45:42.130 INFO:tasks.workunit.client.1.smithi192.stdout:2/831: stat d4/d9/d10/d6c/d95 0 2024-04-24T22:45:42.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/832: dwrite d4/f31 [0,4194304] 0 2024-04-24T22:45:42.187 INFO:tasks.workunit.client.1.smithi192.stdout:0/728: sync 2024-04-24T22:45:42.187 INFO:tasks.workunit.client.1.smithi192.stdout:0/729: chown d3/d27/d6a/dc4 69071 1 2024-04-24T22:45:42.188 INFO:tasks.workunit.client.0.smithi063.stdout:1/936: unlink d3/db5/d21/d9f/dd5/d106/d11/dc6/cdd 0 2024-04-24T22:45:42.192 INFO:tasks.workunit.client.1.smithi192.stdout:5/622: creat d0/d13/d19/d83/dad/fd8 x:0 0 0 2024-04-24T22:45:42.194 INFO:tasks.workunit.client.0.smithi063.stdout:9/912: rmdir d6/d2f/d42/d81/def 39 2024-04-24T22:45:42.194 INFO:tasks.workunit.client.0.smithi063.stdout:1/937: dwrite d3/de0/d79/d96/f12e [0,4194304] 0 2024-04-24T22:45:42.195 INFO:tasks.workunit.client.1.smithi192.stdout:8/720: symlink db/d5d/d74/le0 0 2024-04-24T22:45:42.202 INFO:tasks.workunit.client.1.smithi192.stdout:9/800: fdatasync d3/dd/d62/fb6 0 2024-04-24T22:45:42.204 INFO:tasks.workunit.client.1.smithi192.stdout:9/801: write d3/dd/de/dba/f42 [3449233,68429] 0 2024-04-24T22:45:42.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/956: unlink db/d1d/d20/d11e/f141 0 2024-04-24T22:45:42.207 INFO:tasks.workunit.client.0.smithi063.stdout:6/957: readlink db/d1d/d2d/lf8 0 2024-04-24T22:45:42.207 INFO:tasks.workunit.client.0.smithi063.stdout:6/958: read - db/d1d/d20/d29/f129 zero size 2024-04-24T22:45:42.208 INFO:tasks.workunit.client.1.smithi192.stdout:2/833: truncate d4/d9/d37/fa3 1348056 0 2024-04-24T22:45:42.209 INFO:tasks.workunit.client.0.smithi063.stdout:6/959: write db/d1d/d20/f38 [3798132,69424] 0 2024-04-24T22:45:42.209 INFO:tasks.workunit.client.0.smithi063.stdout:2/994: fdatasync d0/d2/d2d/d31/d89/d7/d86/d8e/ffb 0 2024-04-24T22:45:42.211 INFO:tasks.workunit.client.0.smithi063.stdout:2/995: readlink d0/d2/d39/d11a/de2/le5 0 2024-04-24T22:45:42.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/730: creat d3/d27/d6a/deb/ff1 x:0 0 0 2024-04-24T22:45:42.212 INFO:tasks.workunit.client.1.smithi192.stdout:8/721: dread db/dc/f24 [4194304,4194304] 0 2024-04-24T22:45:42.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/722: chown ca 0 1 2024-04-24T22:45:42.214 INFO:tasks.workunit.client.1.smithi192.stdout:5/623: symlink d0/d13/d19/d64/d5f/d66/ld9 0 2024-04-24T22:45:42.217 INFO:tasks.workunit.client.0.smithi063.stdout:3/895: truncate da/d18/d47/d4d/d62/f80 3971377 0 2024-04-24T22:45:42.218 INFO:tasks.workunit.client.1.smithi192.stdout:7/820: write d2/db/f60 [3993432,64663] 0 2024-04-24T22:45:42.226 INFO:tasks.workunit.client.1.smithi192.stdout:9/802: fsync d3/dd/d62/f92 0 2024-04-24T22:45:42.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/589: dwrite d9/d15/f1f [0,4194304] 0 2024-04-24T22:45:42.228 INFO:tasks.workunit.client.1.smithi192.stdout:1/590: fdatasync d9/d15/d5e/d67/fa2 0 2024-04-24T22:45:42.230 INFO:tasks.workunit.client.0.smithi063.stdout:8/938: chown d1/d22/ca3 0 1 2024-04-24T22:45:42.230 INFO:tasks.workunit.client.1.smithi192.stdout:0/731: creat d3/d6d/ff2 x:0 0 0 2024-04-24T22:45:42.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/960: dread db/d1d/d20/d3f/d7f/d68/f7e [4194304,4194304] 0 2024-04-24T22:45:42.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/746: sync 2024-04-24T22:45:42.236 INFO:tasks.workunit.client.1.smithi192.stdout:3/747: dread - d2/d4/d12/d1a/d1d/fd9 zero size 2024-04-24T22:45:42.237 INFO:tasks.workunit.client.1.smithi192.stdout:2/834: dread d4/d9/d10/f58 [0,4194304] 0 2024-04-24T22:45:42.238 INFO:tasks.workunit.client.0.smithi063.stdout:6/961: dwrite db/d1c/dc2/feb [0,4194304] 0 2024-04-24T22:45:42.239 INFO:tasks.workunit.client.0.smithi063.stdout:6/962: readlink db/d1d/d2d/l46 0 2024-04-24T22:45:42.239 INFO:tasks.workunit.client.0.smithi063.stdout:6/963: chown db/d1c/cf3 6522 1 2024-04-24T22:45:42.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/835: dwrite d4/d9/d10/d6c/d95/db7/fed [0,4194304] 0 2024-04-24T22:45:42.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/836: write d4/d9/d9f/fd7 [1672693,83474] 0 2024-04-24T22:45:42.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/939: dread d1/d11/f1b [0,4194304] 0 2024-04-24T22:45:42.279 INFO:tasks.workunit.client.0.smithi063.stdout:1/938: mkdir d3/db5/d21/d9f/dd5/d106/df5/d12f 0 2024-04-24T22:45:42.280 INFO:tasks.workunit.client.0.smithi063.stdout:1/939: write d3/db5/d21/d9f/dd5/d106/f100 [2219907,63031] 0 2024-04-24T22:45:42.282 INFO:tasks.workunit.client.0.smithi063.stdout:4/864: write d2/fb [10958607,125062] 0 2024-04-24T22:45:42.284 INFO:tasks.workunit.client.1.smithi192.stdout:1/591: mkdir d9/d15/d2d/d48/d93/dc5 0 2024-04-24T22:45:42.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/732: chown d3/d27/d28/dcc/d5c/d80/d83/dd9/cde 30 1 2024-04-24T22:45:42.286 INFO:tasks.workunit.client.1.smithi192.stdout:0/733: chown d3/d6d/d79/f7a 191800 1 2024-04-24T22:45:42.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/723: symlink db/d43/d7c/d85/dcd/le1 0 2024-04-24T22:45:42.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/724: chown db/d43/d7c/d85 959637 1 2024-04-24T22:45:42.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/725: fsync db/d43/f4e 0 2024-04-24T22:45:42.289 INFO:tasks.workunit.client.0.smithi063.stdout:7/814: dwrite d0/d4/de/d17/d35/d69/f7b [4194304,4194304] 0 2024-04-24T22:45:42.290 INFO:tasks.workunit.client.1.smithi192.stdout:3/748: mknod d2/d27/d28/dab/cf3 0 2024-04-24T22:45:42.296 INFO:tasks.workunit.client.1.smithi192.stdout:2/837: unlink d4/d9/d10/d6c/c100 0 2024-04-24T22:45:42.297 INFO:tasks.workunit.client.0.smithi063.stdout:5/842: creat d5/d11/d23/d2d/d62/d73/d118/d36/f122 x:0 0 0 2024-04-24T22:45:42.297 INFO:tasks.workunit.client.0.smithi063.stdout:3/896: creat da/d18/d3a/da1/f137 x:0 0 0 2024-04-24T22:45:42.298 INFO:tasks.workunit.client.1.smithi192.stdout:7/821: mkdir d2/db/d21/d42/d67/d9b/d102 0 2024-04-24T22:45:42.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/822: dwrite d2/db/d21/d42/d67/d9b/d6e/da2/fd1 [0,4194304] 0 2024-04-24T22:45:42.308 INFO:tasks.workunit.client.0.smithi063.stdout:8/940: creat d1/d22/d33/da6/dae/de7/f128 x:0 0 0 2024-04-24T22:45:42.308 INFO:tasks.workunit.client.0.smithi063.stdout:6/964: read db/d1d/d2d/d69/d82/d96/fb4 [3129602,30340] 0 2024-04-24T22:45:42.308 INFO:tasks.workunit.client.0.smithi063.stdout:8/941: chown d1/d2/d11b 11754 1 2024-04-24T22:45:42.309 INFO:tasks.workunit.client.1.smithi192.stdout:8/726: rmdir db/dc/d22/d31/d33/d8f 39 2024-04-24T22:45:42.313 INFO:tasks.workunit.client.1.smithi192.stdout:3/749: unlink d2/d4/d12/d39/d97/l82 0 2024-04-24T22:45:42.315 INFO:tasks.workunit.client.1.smithi192.stdout:5/624: creat d0/d13/d3b/fda x:0 0 0 2024-04-24T22:45:42.318 INFO:tasks.workunit.client.1.smithi192.stdout:2/838: rename d4/d29/l4d to d4/d29/d60/d9a/db3/l110 0 2024-04-24T22:45:42.318 INFO:tasks.workunit.client.0.smithi063.stdout:0/827: getdents d6/da/dc 0 2024-04-24T22:45:42.320 INFO:tasks.workunit.client.1.smithi192.stdout:1/592: unlink d9/d15/d2d/c2a 0 2024-04-24T22:45:42.321 INFO:tasks.workunit.client.1.smithi192.stdout:1/593: chown d9/d51/ca6 46 1 2024-04-24T22:45:42.322 INFO:tasks.workunit.client.0.smithi063.stdout:1/940: rename d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f123 to d3/da/d34/d99/f130 0 2024-04-24T22:45:42.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/839: dwrite d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:42.325 INFO:tasks.workunit.client.1.smithi192.stdout:2/840: chown d4/d9/d37/f3e 1285 1 2024-04-24T22:45:42.330 INFO:tasks.workunit.client.1.smithi192.stdout:7/823: truncate d2/db/d36/f3d 1485693 0 2024-04-24T22:45:42.336 INFO:tasks.workunit.client.0.smithi063.stdout:2/996: fdatasync d0/d2/d2d/d31/d89/d7/d86/da0/d94/ffa 0 2024-04-24T22:45:42.343 INFO:tasks.workunit.client.1.smithi192.stdout:5/625: dwrite d0/f8b [0,4194304] 0 2024-04-24T22:45:42.349 INFO:tasks.workunit.client.1.smithi192.stdout:8/727: dwrite f6 [0,4194304] 0 2024-04-24T22:45:42.351 INFO:tasks.workunit.client.1.smithi192.stdout:5/626: dread d0/d1b/f25 [0,4194304] 0 2024-04-24T22:45:42.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/734: getdents d3/d27/d28/dcc/da8 0 2024-04-24T22:45:42.361 INFO:tasks.workunit.client.1.smithi192.stdout:6/664: sync 2024-04-24T22:45:42.362 INFO:tasks.workunit.client.1.smithi192.stdout:4/725: sync 2024-04-24T22:45:42.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/665: readlink d0/d8/d29/d4c/d4d/lbc 0 2024-04-24T22:45:42.367 INFO:tasks.workunit.client.1.smithi192.stdout:8/728: chown db/d43/d71/f99 30434 1 2024-04-24T22:45:42.372 INFO:tasks.workunit.client.1.smithi192.stdout:1/594: creat d9/d15/d2d/d63/fc6 x:0 0 0 2024-04-24T22:45:42.373 INFO:tasks.workunit.client.1.smithi192.stdout:2/841: link d4/d9/f5b d4/d29/d60/d9a/f111 0 2024-04-24T22:45:42.380 INFO:tasks.workunit.client.1.smithi192.stdout:4/726: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:42.380 INFO:tasks.workunit.client.1.smithi192.stdout:3/750: getdents d2/d2d 0 2024-04-24T22:45:42.385 INFO:tasks.workunit.client.1.smithi192.stdout:1/595: symlink d9/d15/d2d/d48/d93/lc7 0 2024-04-24T22:45:42.386 INFO:tasks.workunit.client.1.smithi192.stdout:2/842: symlink d4/d9/dd4/l112 0 2024-04-24T22:45:42.387 INFO:tasks.workunit.client.1.smithi192.stdout:3/751: dwrite d2/d27/d28/dab/fc2 [0,4194304] 0 2024-04-24T22:45:42.388 INFO:tasks.workunit.client.1.smithi192.stdout:2/843: chown d4/d29/l3d 0 1 2024-04-24T22:45:42.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/666: creat d0/d8/d29/d4c/d4d/d83/dde/fe9 x:0 0 0 2024-04-24T22:45:42.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/667: readlink d0/d8/l61 0 2024-04-24T22:45:42.393 INFO:tasks.workunit.client.1.smithi192.stdout:8/729: creat db/dc/d22/d31/d33/d8f/fe2 x:0 0 0 2024-04-24T22:45:42.393 INFO:tasks.workunit.client.1.smithi192.stdout:8/730: stat db/d5d/f7e 0 2024-04-24T22:45:42.396 INFO:tasks.workunit.client.1.smithi192.stdout:4/727: symlink d2/db/d75/da1/d36/d5b/d28/d2f/lff 0 2024-04-24T22:45:42.397 INFO:tasks.workunit.client.1.smithi192.stdout:4/728: dread - d2/da5/fb9 zero size 2024-04-24T22:45:42.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/627: rmdir d0/d1b/d62/dc1/dd5 0 2024-04-24T22:45:42.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/596: rename d9/d15/d5e/d90/l9e to d9/d15/d2d/d48/d93/da4/dbb/lc8 0 2024-04-24T22:45:42.407 INFO:tasks.workunit.client.1.smithi192.stdout:3/752: truncate d2/d4/d12/fb4 319818 0 2024-04-24T22:45:42.410 INFO:tasks.workunit.client.1.smithi192.stdout:7/824: link d2/db/d21/d42/d67/d9b/l3e d2/l103 0 2024-04-24T22:45:42.411 INFO:tasks.workunit.client.1.smithi192.stdout:7/825: fdatasync d2/db/d21/d42/d67/da8/dc7/fbd 0 2024-04-24T22:45:42.412 INFO:tasks.workunit.client.0.smithi063.stdout:3/897: rmdir da/d18/d47/d4d/d62 39 2024-04-24T22:45:42.413 INFO:tasks.workunit.client.1.smithi192.stdout:7/826: write d2/db/d21/d42/d67/d9b/d6e/fe4 [21164,20879] 0 2024-04-24T22:45:42.414 INFO:tasks.workunit.client.0.smithi063.stdout:5/843: creat d5/d11/d76/d92/dea/f123 x:0 0 0 2024-04-24T22:45:42.419 INFO:tasks.workunit.client.0.smithi063.stdout:6/965: mknod db/d1d/d8d/dce/c142 0 2024-04-24T22:45:42.419 INFO:tasks.workunit.client.1.smithi192.stdout:1/597: dread d9/f14 [0,4194304] 0 2024-04-24T22:45:42.420 INFO:tasks.workunit.client.0.smithi063.stdout:6/966: dread - db/d1d/d20/d29/f129 zero size 2024-04-24T22:45:42.420 INFO:tasks.workunit.client.1.smithi192.stdout:0/735: getdents d3/d27/d28/d2e/dc7/dd4 0 2024-04-24T22:45:42.421 INFO:tasks.workunit.client.0.smithi063.stdout:8/942: symlink d1/d2/d3/d86/l129 0 2024-04-24T22:45:42.422 INFO:tasks.workunit.client.0.smithi063.stdout:8/943: chown d1/d22/d33/da6/dae/ceb 16680 1 2024-04-24T22:45:42.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/944: readlink d1/d11/d32/l7a 0 2024-04-24T22:45:42.423 INFO:tasks.workunit.client.0.smithi063.stdout:9/913: link d6/dbd/d12d/c116 d6/d2f/dcd/db/d6b/c133 0 2024-04-24T22:45:42.425 INFO:tasks.workunit.client.0.smithi063.stdout:9/914: stat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/d12e/f104 0 2024-04-24T22:45:42.425 INFO:tasks.workunit.client.1.smithi192.stdout:3/753: creat d2/d27/d28/dab/dca/ff4 x:0 0 0 2024-04-24T22:45:42.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/815: rename d0/d4/de/d17/d35/l83 to d0/d4/de/dfc/l110 0 2024-04-24T22:45:42.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/865: creat d2/d8/dd4/dda/da5/f121 x:0 0 0 2024-04-24T22:45:42.434 INFO:tasks.workunit.client.0.smithi063.stdout:1/941: truncate d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/fbe 303234 0 2024-04-24T22:45:42.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/997: rmdir d0/d2/d2d/d31/d89/d7/d86/da0/deb 39 2024-04-24T22:45:42.437 INFO:tasks.workunit.client.1.smithi192.stdout:1/598: creat d9/d15/d2d/d48/d93/da4/fc9 x:0 0 0 2024-04-24T22:45:42.438 INFO:tasks.workunit.client.0.smithi063.stdout:3/898: symlink da/d18/d47/d4d/d9b/dde/l138 0 2024-04-24T22:45:42.438 INFO:tasks.workunit.client.1.smithi192.stdout:1/599: fdatasync d9/d15/d2d/d4a/f8b 0 2024-04-24T22:45:42.439 INFO:tasks.workunit.client.0.smithi063.stdout:3/899: chown da/d18/d9e/da3/lb8 188941 1 2024-04-24T22:45:42.441 INFO:tasks.workunit.client.0.smithi063.stdout:6/967: unlink db/d1d/d8d/dfc/d11c/f125 0 2024-04-24T22:45:42.441 INFO:tasks.workunit.client.0.smithi063.stdout:6/968: fdatasync db/d1d/d2d/d69/d82/f9a 0 2024-04-24T22:45:42.442 INFO:tasks.workunit.client.1.smithi192.stdout:0/736: dread d3/d27/d3c/d3b/f65 [0,4194304] 0 2024-04-24T22:45:42.447 INFO:tasks.workunit.client.1.smithi192.stdout:2/844: creat d4/d9/d10/d6c/f113 x:0 0 0 2024-04-24T22:45:42.448 INFO:tasks.workunit.client.1.smithi192.stdout:2/845: read - d4/d9/f92 zero size 2024-04-24T22:45:42.448 INFO:tasks.workunit.client.1.smithi192.stdout:0/737: dwrite d3/f47 [0,4194304] 0 2024-04-24T22:45:42.449 INFO:tasks.workunit.client.1.smithi192.stdout:3/754: creat d2/d27/ff5 x:0 0 0 2024-04-24T22:45:42.451 INFO:tasks.workunit.client.0.smithi063.stdout:7/816: symlink d0/d4/de/d17/d35/d69/l111 0 2024-04-24T22:45:42.453 INFO:tasks.workunit.client.1.smithi192.stdout:6/668: creat d0/d8/d3d/d3e/fea x:0 0 0 2024-04-24T22:45:42.454 INFO:tasks.workunit.client.1.smithi192.stdout:4/729: write d2/f1f [3600660,22363] 0 2024-04-24T22:45:42.457 INFO:tasks.workunit.client.1.smithi192.stdout:1/600: mknod d9/dd/d74/cca 0 2024-04-24T22:45:42.459 INFO:tasks.workunit.client.1.smithi192.stdout:1/601: write d9/d15/d2d/d31/fab [742865,128641] 0 2024-04-24T22:45:42.460 INFO:tasks.workunit.client.0.smithi063.stdout:2/998: symlink d0/d2/d2d/d31/d89/d7/d86/d8e/l134 0 2024-04-24T22:45:42.461 INFO:tasks.workunit.client.0.smithi063.stdout:2/999: stat d0/d2/d2d/f4d 0 2024-04-24T22:45:42.463 INFO:tasks.workunit.client.1.smithi192.stdout:5/628: dwrite d0/d13/d19/d64/d5f/d2c/d3c/f77 [0,4194304] 0 2024-04-24T22:45:42.464 INFO:tasks.workunit.client.1.smithi192.stdout:5/629: fdatasync d0/f4a 0 2024-04-24T22:45:42.467 INFO:tasks.workunit.client.0.smithi063.stdout:0/828: dwrite d6/d64/d21/d25/d62/d63/d89/d95/da4/fc2 [0,4194304] 0 2024-04-24T22:45:42.469 INFO:tasks.workunit.client.1.smithi192.stdout:3/755: dread d2/d27/d28/da5/d93/faa [0,4194304] 0 2024-04-24T22:45:42.472 INFO:tasks.workunit.client.1.smithi192.stdout:8/731: getdents db/d43 0 2024-04-24T22:45:42.481 INFO:tasks.workunit.client.0.smithi063.stdout:7/817: creat d0/d4/de/d17/d28/f112 x:0 0 0 2024-04-24T22:45:42.482 INFO:tasks.workunit.client.0.smithi063.stdout:4/866: mkdir d2/d122 0 2024-04-24T22:45:42.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/915: dwrite d6/d2f/dcd/db/d93/fd8 [4194304,4194304] 0 2024-04-24T22:45:42.487 INFO:tasks.workunit.client.1.smithi192.stdout:0/738: mkdir d3/d27/d28/dcc/df3 0 2024-04-24T22:45:42.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/829: dread - d6/d64/da0/f84 zero size 2024-04-24T22:45:42.490 INFO:tasks.workunit.client.1.smithi192.stdout:5/630: symlink d0/d13/d19/d64/d5f/d66/ldb 0 2024-04-24T22:45:42.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/830: write d6/dd/f4d [1015876,89241] 0 2024-04-24T22:45:42.492 INFO:tasks.workunit.client.0.smithi063.stdout:6/969: unlink db/d1d/d20/c132 0 2024-04-24T22:45:42.494 INFO:tasks.workunit.client.0.smithi063.stdout:6/970: dread db/d1d/d20/d29/d5d/df5/f117 [0,4194304] 0 2024-04-24T22:45:42.497 INFO:tasks.workunit.client.1.smithi192.stdout:2/846: dwrite d4/d9/d37/f9c [0,4194304] 0 2024-04-24T22:45:42.501 INFO:tasks.workunit.client.1.smithi192.stdout:3/756: dread d2/d4/d12/f1b [4194304,4194304] 0 2024-04-24T22:45:42.502 INFO:tasks.workunit.client.1.smithi192.stdout:4/730: write d2/db/d75/da1/d36/d5b/d28/d2f/fb1 [107244,1013] 0 2024-04-24T22:45:42.503 INFO:tasks.workunit.client.0.smithi063.stdout:7/818: mkdir d0/d4/de/d17/d28/d31/d99/d113 0 2024-04-24T22:45:42.506 INFO:tasks.workunit.client.0.smithi063.stdout:4/867: mknod d2/d19/d33/dc1/d50/dfa/c123 0 2024-04-24T22:45:42.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/900: write da/d39/d94/db3/fc3 [657934,18682] 0 2024-04-24T22:45:42.507 INFO:tasks.workunit.client.0.smithi063.stdout:3/901: readlink da/d18/l35 0 2024-04-24T22:45:42.510 INFO:tasks.workunit.client.1.smithi192.stdout:1/602: mknod d9/d15/d2d/ccb 0 2024-04-24T22:45:42.512 INFO:tasks.workunit.client.1.smithi192.stdout:0/739: creat d3/d27/d6a/ff4 x:0 0 0 2024-04-24T22:45:42.512 INFO:tasks.workunit.client.1.smithi192.stdout:6/669: write d0/d8/d3d/d3e/d40/d44/f93 [852761,101332] 0 2024-04-24T22:45:42.515 INFO:tasks.workunit.client.0.smithi063.stdout:0/831: fsync d6/d64/d21/f86 0 2024-04-24T22:45:42.516 INFO:tasks.workunit.client.1.smithi192.stdout:2/847: mkdir d4/d9/d37/de5/d114 0 2024-04-24T22:45:42.517 INFO:tasks.workunit.client.0.smithi063.stdout:1/942: link d3/de0/d79/l59 d3/db5/l131 0 2024-04-24T22:45:42.520 INFO:tasks.workunit.client.1.smithi192.stdout:5/631: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:42.522 INFO:tasks.workunit.client.1.smithi192.stdout:4/731: creat d2/d5c/dbb/f100 x:0 0 0 2024-04-24T22:45:42.523 INFO:tasks.workunit.client.1.smithi192.stdout:5/632: dread - d0/d13/d19/d64/d4f/d6c/fae zero size 2024-04-24T22:45:42.523 INFO:tasks.workunit.client.0.smithi063.stdout:4/868: mkdir d2/d19/d82/d124 0 2024-04-24T22:45:42.523 INFO:tasks.workunit.client.1.smithi192.stdout:2/848: dwrite d4/d9/d37/f64 [4194304,4194304] 0 2024-04-24T22:45:42.524 INFO:tasks.workunit.client.1.smithi192.stdout:8/732: creat db/dc/d22/dca/fe3 x:0 0 0 2024-04-24T22:45:42.528 INFO:tasks.workunit.client.0.smithi063.stdout:9/916: mknod d6/d2f/dcd/db/d1c/d78/d9a/ddd/d131/c134 0 2024-04-24T22:45:42.528 INFO:tasks.workunit.client.1.smithi192.stdout:0/740: mknod d3/d27/d28/dba/cf5 0 2024-04-24T22:45:42.528 INFO:tasks.workunit.client.0.smithi063.stdout:9/917: chown d6/d2f/d42/dda/c12f 553652 1 2024-04-24T22:45:42.530 INFO:tasks.workunit.client.0.smithi063.stdout:5/844: sync 2024-04-24T22:45:42.530 INFO:tasks.workunit.client.0.smithi063.stdout:8/945: sync 2024-04-24T22:45:42.532 INFO:tasks.workunit.client.0.smithi063.stdout:0/832: creat d6/d64/d21/d80/f118 x:0 0 0 2024-04-24T22:45:42.533 INFO:tasks.workunit.client.1.smithi192.stdout:4/732: symlink d2/d9/d25/d2e/l101 0 2024-04-24T22:45:42.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/971: mkdir db/d1d/d20/d3f/d143 0 2024-04-24T22:45:42.534 INFO:tasks.workunit.client.0.smithi063.stdout:1/943: symlink d3/da/d34/d68/l132 0 2024-04-24T22:45:42.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/849: fdatasync d4/d29/fdc 0 2024-04-24T22:45:42.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/869: mkdir d2/d19/dce/d125 0 2024-04-24T22:45:42.538 INFO:tasks.workunit.client.0.smithi063.stdout:7/819: dwrite d0/d4/de/d6f/fcb [0,4194304] 0 2024-04-24T22:45:42.538 INFO:tasks.workunit.client.1.smithi192.stdout:8/733: mknod db/d5d/d74/ce4 0 2024-04-24T22:45:42.540 INFO:tasks.workunit.client.1.smithi192.stdout:1/603: symlink d9/d15/d2d/d48/d93/dc5/lcc 0 2024-04-24T22:45:42.542 INFO:tasks.workunit.client.0.smithi063.stdout:9/918: mkdir d6/d2f/dcd/d53/dff/d132/d5e/d9d/d100/d135 0 2024-04-24T22:45:42.544 INFO:tasks.workunit.client.1.smithi192.stdout:0/741: truncate d3/d27/d28/dcc/d5c/f9a 684230 0 2024-04-24T22:45:42.545 INFO:tasks.workunit.client.1.smithi192.stdout:9/803: sync 2024-04-24T22:45:42.546 INFO:tasks.workunit.client.1.smithi192.stdout:0/742: write d3/d27/d28/d2e/d3f/fc8 [4280663,89946] 0 2024-04-24T22:45:42.546 INFO:tasks.workunit.client.1.smithi192.stdout:4/733: unlink d2/db/d75/da1/d36/d5b/d28/d2f/lff 0 2024-04-24T22:45:42.548 INFO:tasks.workunit.client.1.smithi192.stdout:3/757: dread d2/d27/dc6/f2c [0,4194304] 0 2024-04-24T22:45:42.549 INFO:tasks.workunit.client.0.smithi063.stdout:0/833: creat d6/d74/f119 x:0 0 0 2024-04-24T22:45:42.550 INFO:tasks.workunit.client.0.smithi063.stdout:6/972: mkdir db/d1d/d8d/dfc/d144 0 2024-04-24T22:45:42.552 INFO:tasks.workunit.client.1.smithi192.stdout:8/734: rmdir db/dc/d29 39 2024-04-24T22:45:42.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/944: chown d3/db5/d21/d9f/dd5/d106/d11/lfb 24069 1 2024-04-24T22:45:42.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/804: dread - d3/dd/de/da7/dae/ff0 zero size 2024-04-24T22:45:42.560 INFO:tasks.workunit.client.0.smithi063.stdout:3/902: truncate da/d18/d47/d4d/d62/f6f 334184 0 2024-04-24T22:45:42.561 INFO:tasks.workunit.client.1.smithi192.stdout:4/734: symlink d2/db/d5d/d77/d82/l102 0 2024-04-24T22:45:42.563 INFO:tasks.workunit.client.1.smithi192.stdout:5/633: dwrite d0/d13/d19/f4b [0,4194304] 0 2024-04-24T22:45:42.564 INFO:tasks.workunit.client.0.smithi063.stdout:9/919: symlink d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/l136 0 2024-04-24T22:45:42.566 INFO:tasks.workunit.client.0.smithi063.stdout:9/920: dread - d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/f121 zero size 2024-04-24T22:45:42.568 INFO:tasks.workunit.client.1.smithi192.stdout:8/735: creat db/dc/d22/dca/fe5 x:0 0 0 2024-04-24T22:45:42.569 INFO:tasks.workunit.client.0.smithi063.stdout:8/946: write d1/d22/ffd [147275,107416] 0 2024-04-24T22:45:42.570 INFO:tasks.workunit.client.0.smithi063.stdout:8/947: chown d1/d2/d17/dd1/f103 155007 1 2024-04-24T22:45:42.572 INFO:tasks.workunit.client.1.smithi192.stdout:6/670: getdents d0/d8/d3d/d3e/d40 0 2024-04-24T22:45:42.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/834: unlink d6/d64/l2c 0 2024-04-24T22:45:42.574 INFO:tasks.workunit.client.0.smithi063.stdout:0/835: write d6/d74/fab [260707,12633] 0 2024-04-24T22:45:42.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/805: rename d3/d5d/da1/d85/cc0 to d3/d5d/da1/d5e/dfb/c10e 0 2024-04-24T22:45:42.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/973: symlink db/d85/dd2/de6/l145 0 2024-04-24T22:45:42.581 INFO:tasks.workunit.client.1.smithi192.stdout:0/743: dread d3/d27/d28/d31/f52 [0,4194304] 0 2024-04-24T22:45:42.582 INFO:tasks.workunit.client.1.smithi192.stdout:1/604: dwrite d9/d15/d2d/d31/d2f/f4c [0,4194304] 0 2024-04-24T22:45:42.582 INFO:tasks.workunit.client.0.smithi063.stdout:1/945: unlink d3/de0/d79/fc1 0 2024-04-24T22:45:42.587 INFO:tasks.workunit.client.0.smithi063.stdout:4/870: fdatasync d2/f26 0 2024-04-24T22:45:42.588 INFO:tasks.workunit.client.1.smithi192.stdout:5/634: chown d0/d13/f40 6 1 2024-04-24T22:45:42.589 INFO:tasks.workunit.client.0.smithi063.stdout:4/871: dread - d2/d19/d33/d9d/da8/dec/ded/ff5 zero size 2024-04-24T22:45:42.590 INFO:tasks.workunit.client.0.smithi063.stdout:5/845: truncate d5/de5/fed 172972 0 2024-04-24T22:45:42.591 INFO:tasks.workunit.client.1.smithi192.stdout:1/605: read d9/d15/d2d/d31/d2f/f85 [414469,12502] 0 2024-04-24T22:45:42.592 INFO:tasks.workunit.client.0.smithi063.stdout:3/903: creat da/d18/d47/f139 x:0 0 0 2024-04-24T22:45:42.592 INFO:tasks.workunit.client.1.smithi192.stdout:3/758: dwrite d2/f1e [0,4194304] 0 2024-04-24T22:45:42.593 INFO:tasks.workunit.client.0.smithi063.stdout:3/904: write da/d18/f30 [3076663,69728] 0 2024-04-24T22:45:42.595 INFO:tasks.workunit.client.1.smithi192.stdout:3/759: chown d2/d27/dc6/fd 6 1 2024-04-24T22:45:42.595 INFO:tasks.workunit.client.1.smithi192.stdout:1/606: truncate d9/dd/d74/d83/fb7 700107 0 2024-04-24T22:45:42.596 INFO:tasks.workunit.client.1.smithi192.stdout:1/607: stat d9/dd/l60 0 2024-04-24T22:45:42.597 INFO:tasks.workunit.client.1.smithi192.stdout:8/736: fdatasync db/dc/d22/d31/d33/f49 0 2024-04-24T22:45:42.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/635: dwrite d0/d13/d19/d64/d5f/fc4 [0,4194304] 0 2024-04-24T22:45:42.599 INFO:tasks.workunit.client.1.smithi192.stdout:4/735: write d2/db/d75/da1/f4a [4049735,82246] 0 2024-04-24T22:45:42.601 INFO:tasks.workunit.client.1.smithi192.stdout:6/671: symlink d0/d8/d29/d5a/d63/leb 0 2024-04-24T22:45:42.602 INFO:tasks.workunit.client.1.smithi192.stdout:5/636: write d0/d13/d19/d64/d5f/fc4 [1417488,96156] 0 2024-04-24T22:45:42.604 INFO:tasks.workunit.client.0.smithi063.stdout:6/974: mknod db/d1d/d20/d29/d5d/c146 0 2024-04-24T22:45:42.605 INFO:tasks.workunit.client.1.smithi192.stdout:9/806: rmdir d3/d59/d7d/d88 39 2024-04-24T22:45:42.608 INFO:tasks.workunit.client.1.smithi192.stdout:1/608: dwrite d9/d15/d2d/d63/fc6 [0,4194304] 0 2024-04-24T22:45:42.611 INFO:tasks.workunit.client.0.smithi063.stdout:0/836: dread d6/d64/d24/f6b [0,4194304] 0 2024-04-24T22:45:42.615 INFO:tasks.workunit.client.0.smithi063.stdout:8/948: dwrite d1/d11/d8b/fa9 [0,4194304] 0 2024-04-24T22:45:42.615 INFO:tasks.workunit.client.0.smithi063.stdout:4/872: mknod d2/d8/d36/d118/c126 0 2024-04-24T22:45:42.618 INFO:tasks.workunit.client.0.smithi063.stdout:5/846: mkdir d5/d11/d23/d2d/d62/d81/da4/da6/d124 0 2024-04-24T22:45:42.619 INFO:tasks.workunit.client.1.smithi192.stdout:8/737: symlink db/d43/d7c/le6 0 2024-04-24T22:45:42.620 INFO:tasks.workunit.client.0.smithi063.stdout:3/905: write da/d18/d47/d4d/d9b/fe1 [377810,82726] 0 2024-04-24T22:45:42.623 INFO:tasks.workunit.client.1.smithi192.stdout:4/736: symlink d2/db/d75/da1/d7d/d9c/l103 0 2024-04-24T22:45:42.627 INFO:tasks.workunit.client.0.smithi063.stdout:6/975: dread db/d1c/dc3/fc8 [0,4194304] 0 2024-04-24T22:45:42.628 INFO:tasks.workunit.client.1.smithi192.stdout:5/637: mknod d0/d13/d53/cdc 0 2024-04-24T22:45:42.628 INFO:tasks.workunit.client.1.smithi192.stdout:8/738: dwrite db/dc/d22/d31/d33/d81/d8a/faa [0,4194304] 0 2024-04-24T22:45:42.632 INFO:tasks.workunit.client.1.smithi192.stdout:9/807: creat d3/d5d/da1/d5e/dfb/f10f x:0 0 0 2024-04-24T22:45:42.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/820: getdents d0/d4/de/d6f 0 2024-04-24T22:45:42.636 INFO:tasks.workunit.client.0.smithi063.stdout:8/949: fdatasync d1/d11/d3c/f55 0 2024-04-24T22:45:42.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/821: dwrite d0/d4/de/d17/d35/d69/f6b [0,4194304] 0 2024-04-24T22:45:42.640 INFO:tasks.workunit.client.1.smithi192.stdout:3/760: mkdir d2/d4/d12/d1a/df6 0 2024-04-24T22:45:42.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/808: dread d3/d5d/fd7 [0,4194304] 0 2024-04-24T22:45:42.642 INFO:tasks.workunit.client.1.smithi192.stdout:3/761: chown d2/d27/dc6/da0/fce 8481177 1 2024-04-24T22:45:42.642 INFO:tasks.workunit.client.0.smithi063.stdout:5/847: chown d5/d11/d23/d2d/d62/d73/d118/f91 2188 1 2024-04-24T22:45:42.643 INFO:tasks.workunit.client.0.smithi063.stdout:5/848: readlink d5/d11/d23/d2d/lc4 0 2024-04-24T22:45:42.644 INFO:tasks.workunit.client.1.smithi192.stdout:4/737: dread d2/d9/f3f [0,4194304] 0 2024-04-24T22:45:42.644 INFO:tasks.workunit.client.0.smithi063.stdout:3/906: mknod da/d18/d115/c13a 0 2024-04-24T22:45:42.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/672: creat d0/d8/d3d/d3e/d7e/fec x:0 0 0 2024-04-24T22:45:42.646 INFO:tasks.workunit.client.0.smithi063.stdout:9/921: sync 2024-04-24T22:45:42.650 INFO:tasks.workunit.client.1.smithi192.stdout:8/739: creat db/dc/d22/d31/d33/d8f/fe7 x:0 0 0 2024-04-24T22:45:42.651 INFO:tasks.workunit.client.0.smithi063.stdout:9/922: read d6/d2f/d42/d81/fa9 [5988919,72781] 0 2024-04-24T22:45:42.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/946: dwrite d3/da/d34/fa1 [0,4194304] 0 2024-04-24T22:45:42.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/740: chown db/l60 40973 1 2024-04-24T22:45:42.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/947: chown d3/db5/d21/d9f/dd5/d106/df5 744034 1 2024-04-24T22:45:42.653 INFO:tasks.workunit.client.0.smithi063.stdout:9/923: dread d6/d2f/dcd/db/d1c/ffe [0,4194304] 0 2024-04-24T22:45:42.653 INFO:tasks.workunit.client.1.smithi192.stdout:7/827: sync 2024-04-24T22:45:42.655 INFO:tasks.workunit.client.1.smithi192.stdout:0/744: dwrite d3/d27/f71 [0,4194304] 0 2024-04-24T22:45:42.658 INFO:tasks.workunit.client.1.smithi192.stdout:5/638: dwrite d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:42.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/828: dread - d2/db/d21/d42/d67/d9b/d6e/db5/fdf zero size 2024-04-24T22:45:42.660 INFO:tasks.workunit.client.1.smithi192.stdout:8/741: read db/d43/d76/d92/f9b [2329174,94590] 0 2024-04-24T22:45:42.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/950: rmdir d1/d22/d33/da6/d54 39 2024-04-24T22:45:42.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/951: stat d1/d22/d33/da6/d91/c9b 0 2024-04-24T22:45:42.662 INFO:tasks.workunit.client.0.smithi063.stdout:8/952: write d1/d22/ffd [602689,6025] 0 2024-04-24T22:45:42.663 INFO:tasks.workunit.client.0.smithi063.stdout:8/953: chown d1/d2/d3/d86/c9d 25074100 1 2024-04-24T22:45:42.664 INFO:tasks.workunit.client.1.smithi192.stdout:3/762: rename d2/d27/dc6/d4b to d2/d4/d12/d1a/d1d/d7b/df7 0 2024-04-24T22:45:42.666 INFO:tasks.workunit.client.1.smithi192.stdout:3/763: dread d2/d27/d28/da5/d93/faa [0,4194304] 0 2024-04-24T22:45:42.668 INFO:tasks.workunit.client.1.smithi192.stdout:3/764: chown d2/d4/d12/d1a/d1d/d55/d5f 1 1 2024-04-24T22:45:42.669 INFO:tasks.workunit.client.1.smithi192.stdout:1/609: write d9/f14 [1431339,54892] 0 2024-04-24T22:45:42.670 INFO:tasks.workunit.client.1.smithi192.stdout:6/673: unlink d0/d8/d3d/d3e/d40/d44/c70 0 2024-04-24T22:45:42.671 INFO:tasks.workunit.client.1.smithi192.stdout:6/674: chown d0/d8/d3d/f49 1874299 1 2024-04-24T22:45:42.672 INFO:tasks.workunit.client.0.smithi063.stdout:3/907: chown da/f4c 15968994 1 2024-04-24T22:45:42.673 INFO:tasks.workunit.client.1.smithi192.stdout:0/745: symlink d3/d27/d6a/lf6 0 2024-04-24T22:45:42.675 INFO:tasks.workunit.client.0.smithi063.stdout:6/976: mkdir db/d1d/d8d/d147 0 2024-04-24T22:45:42.680 INFO:tasks.workunit.client.1.smithi192.stdout:8/742: mknod db/dc/d22/dca/ce8 0 2024-04-24T22:45:42.682 INFO:tasks.workunit.client.1.smithi192.stdout:8/743: write db/dc/fb0 [699728,83831] 0 2024-04-24T22:45:42.683 INFO:tasks.workunit.client.0.smithi063.stdout:4/873: dwrite d2/d80/d81/d90/fe1 [0,4194304] 0 2024-04-24T22:45:42.683 INFO:tasks.workunit.client.0.smithi063.stdout:1/948: rename d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/la9 to d3/db5/d21/d9f/dd5/d106/d39/l133 0 2024-04-24T22:45:42.684 INFO:tasks.workunit.client.0.smithi063.stdout:1/949: chown f1 4270 1 2024-04-24T22:45:42.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/837: link d6/dd/l85 d6/d64/d21/d25/d62/d75/l11a 0 2024-04-24T22:45:42.689 INFO:tasks.workunit.client.0.smithi063.stdout:8/954: symlink d1/d11/d32/d38/db1/l12a 0 2024-04-24T22:45:42.690 INFO:tasks.workunit.client.0.smithi063.stdout:7/822: symlink d0/d4/de/d17/d35/de4/l114 0 2024-04-24T22:45:42.692 INFO:tasks.workunit.client.1.smithi192.stdout:3/765: creat d2/d4/d12/d1a/d1d/d55/d5f/db1/ff8 x:0 0 0 2024-04-24T22:45:42.693 INFO:tasks.workunit.client.1.smithi192.stdout:3/766: dread - d2/d27/d28/dab/dca/fed zero size 2024-04-24T22:45:42.693 INFO:tasks.workunit.client.1.smithi192.stdout:1/610: mkdir d9/dd/d74/dcd 0 2024-04-24T22:45:42.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/908: stat da/c2d 0 2024-04-24T22:45:42.695 INFO:tasks.workunit.client.1.smithi192.stdout:2/850: sync 2024-04-24T22:45:42.698 INFO:tasks.workunit.client.1.smithi192.stdout:5/639: write d0/d5b/f75 [292976,52374] 0 2024-04-24T22:45:42.700 INFO:tasks.workunit.client.0.smithi063.stdout:9/924: write d6/d2f/dcd/d53/f99 [666031,125458] 0 2024-04-24T22:45:42.701 INFO:tasks.workunit.client.1.smithi192.stdout:7/829: fsync d2/db/d21/d42/d5b/f5d 0 2024-04-24T22:45:42.704 INFO:tasks.workunit.client.1.smithi192.stdout:7/830: stat d2/db/d21/d42/d67/dd4 0 2024-04-24T22:45:42.705 INFO:tasks.workunit.client.1.smithi192.stdout:9/809: link d3/dd/de/d17/c76 d3/dd/de/dba/dbc/c110 0 2024-04-24T22:45:42.710 INFO:tasks.workunit.client.1.smithi192.stdout:1/611: creat d9/d15/d2d/d4a/fce x:0 0 0 2024-04-24T22:45:42.711 INFO:tasks.workunit.client.0.smithi063.stdout:5/849: creat d5/d11/f125 x:0 0 0 2024-04-24T22:45:42.712 INFO:tasks.workunit.client.0.smithi063.stdout:5/850: readlink d5/d1d/d7d/d67/l11b 0 2024-04-24T22:45:42.713 INFO:tasks.workunit.client.1.smithi192.stdout:6/675: symlink d0/d8/d18/led 0 2024-04-24T22:45:42.716 INFO:tasks.workunit.client.1.smithi192.stdout:2/851: mknod d4/d9/d10/d6c/d8c/da5/c115 0 2024-04-24T22:45:42.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/612: dwrite d9/d15/d2d/d63/fb2 [0,4194304] 0 2024-04-24T22:45:42.719 INFO:tasks.workunit.client.1.smithi192.stdout:5/640: symlink d0/d13/d19/d64/d5f/d2c/ldd 0 2024-04-24T22:45:42.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/977: mkdir db/d1d/d8d/dce/d12a/d10f/d138/d148 0 2024-04-24T22:45:42.722 INFO:tasks.workunit.client.1.smithi192.stdout:0/746: write d3/d27/d3c/d3b/fa9 [594023,61834] 0 2024-04-24T22:45:42.723 INFO:tasks.workunit.client.1.smithi192.stdout:0/747: fdatasync d3/d27/d6a/ff4 0 2024-04-24T22:45:42.724 INFO:tasks.workunit.client.0.smithi063.stdout:9/925: symlink d6/d2f/d42/dda/l137 0 2024-04-24T22:45:42.724 INFO:tasks.workunit.client.0.smithi063.stdout:8/955: dread d1/d22/f77 [0,4194304] 0 2024-04-24T22:45:42.727 INFO:tasks.workunit.client.1.smithi192.stdout:7/831: mkdir d2/db/d49/dca/d104 0 2024-04-24T22:45:42.729 INFO:tasks.workunit.client.0.smithi063.stdout:1/950: mkdir d3/db5/d21/d9f/dd5/d106/d39/d5a/dff/d134 0 2024-04-24T22:45:42.729 INFO:tasks.workunit.client.1.smithi192.stdout:3/767: write d2/d4/f69 [1120493,56781] 0 2024-04-24T22:45:42.730 INFO:tasks.workunit.client.1.smithi192.stdout:8/744: fsync db/f65 0 2024-04-24T22:45:42.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/768: write d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/fe4 [71484,48995] 0 2024-04-24T22:45:42.734 INFO:tasks.workunit.client.1.smithi192.stdout:9/810: mknod d3/d5d/da1/d5e/c111 0 2024-04-24T22:45:42.734 INFO:tasks.workunit.client.1.smithi192.stdout:7/832: dread d2/db/d21/ff8 [0,4194304] 0 2024-04-24T22:45:42.734 INFO:tasks.workunit.client.0.smithi063.stdout:7/823: mknod d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/c115 0 2024-04-24T22:45:42.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/738: link d2/d5c/dbb/ld7 d2/db/d75/da1/d7d/l104 0 2024-04-24T22:45:42.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/739: fsync d2/d9/d6b/fac 0 2024-04-24T22:45:42.739 INFO:tasks.workunit.client.0.smithi063.stdout:6/978: truncate db/fc6 1009808 0 2024-04-24T22:45:42.740 INFO:tasks.workunit.client.0.smithi063.stdout:3/909: dwrite da/fa0 [0,4194304] 0 2024-04-24T22:45:42.740 INFO:tasks.workunit.client.1.smithi192.stdout:6/676: mknod d0/d8/d22/cee 0 2024-04-24T22:45:42.742 INFO:tasks.workunit.client.0.smithi063.stdout:9/926: mkdir d6/d2f/dcd/db/d6b/d4f/dad/d36/d138 0 2024-04-24T22:45:42.745 INFO:tasks.workunit.client.0.smithi063.stdout:4/874: dwrite d2/d19/d82/fb8 [0,4194304] 0 2024-04-24T22:45:42.747 INFO:tasks.workunit.client.1.smithi192.stdout:0/748: mknod d3/d27/d28/dcc/d5c/d80/d94/cf7 0 2024-04-24T22:45:42.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/851: truncate d5/de5/fed 1147096 0 2024-04-24T22:45:42.751 INFO:tasks.workunit.client.1.smithi192.stdout:8/745: symlink db/d43/d7c/d85/db4/le9 0 2024-04-24T22:45:42.752 INFO:tasks.workunit.client.1.smithi192.stdout:1/613: dread d9/d51/d70/f7f [0,4194304] 0 2024-04-24T22:45:42.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/910: write da/d18/d3a/f3f [1009622,39649] 0 2024-04-24T22:45:42.754 INFO:tasks.workunit.client.1.smithi192.stdout:8/746: dread db/d75/fc9 [0,4194304] 0 2024-04-24T22:45:42.754 INFO:tasks.workunit.client.1.smithi192.stdout:7/833: dread d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:42.756 INFO:tasks.workunit.client.1.smithi192.stdout:3/769: fdatasync d2/d62/f75 0 2024-04-24T22:45:42.760 INFO:tasks.workunit.client.0.smithi063.stdout:4/875: dwrite d2/d19/d33/d9d/da8/f115 [0,4194304] 0 2024-04-24T22:45:42.761 INFO:tasks.workunit.client.0.smithi063.stdout:9/927: creat d6/d2f/dcd/db/d1c/d78/f139 x:0 0 0 2024-04-24T22:45:42.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/740: creat d2/db/d5d/db3/f105 x:0 0 0 2024-04-24T22:45:42.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/741: readlink d2/d9/lfd 0 2024-04-24T22:45:42.764 INFO:tasks.workunit.client.0.smithi063.stdout:0/838: getdents d6/d64/d21/d80/dea 0 2024-04-24T22:45:42.765 INFO:tasks.workunit.client.1.smithi192.stdout:4/742: chown d2/d9/d25/f8e 35438199 1 2024-04-24T22:45:42.765 INFO:tasks.workunit.client.1.smithi192.stdout:4/743: chown d2/db/d5d/db3 994 1 2024-04-24T22:45:42.767 INFO:tasks.workunit.client.0.smithi063.stdout:7/824: mknod d0/d4/de/d6f/d2e/d55/d106/d10e/c116 0 2024-04-24T22:45:42.768 INFO:tasks.workunit.client.1.smithi192.stdout:2/852: symlink d4/d9/d10/d6c/d6f/l116 0 2024-04-24T22:45:42.768 INFO:tasks.workunit.client.0.smithi063.stdout:7/825: chown d0/d4/de/d17/d35/c71 40146 1 2024-04-24T22:45:42.769 INFO:tasks.workunit.client.0.smithi063.stdout:6/979: symlink db/d1c/l149 0 2024-04-24T22:45:42.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/826: stat d0/d4/de/d17/fc6 0 2024-04-24T22:45:42.770 INFO:tasks.workunit.client.0.smithi063.stdout:1/951: dwrite d3/db5/d21/f91 [0,4194304] 0 2024-04-24T22:45:42.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/827: readlink d0/l23 0 2024-04-24T22:45:42.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/828: chown d0/c27 2 1 2024-04-24T22:45:42.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/952: read d3/db5/d21/d9f/dd5/d106/d39/d5a/d10e/f12c [467633,109115] 0 2024-04-24T22:45:42.774 INFO:tasks.workunit.client.1.smithi192.stdout:0/749: symlink d3/d27/d28/dcc/d5c/d80/d83/lf8 0 2024-04-24T22:45:42.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/811: dwrite d3/f58 [0,4194304] 0 2024-04-24T22:45:42.783 INFO:tasks.workunit.client.0.smithi063.stdout:5/852: mknod d5/de5/c126 0 2024-04-24T22:45:42.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/747: creat db/d5d/d74/fea x:0 0 0 2024-04-24T22:45:42.789 INFO:tasks.workunit.client.0.smithi063.stdout:9/928: symlink d6/d2f/dcd/d53/dff/d132/d5e/d9d/d11e/l13a 0 2024-04-24T22:45:42.790 INFO:tasks.workunit.client.0.smithi063.stdout:0/839: dread - d6/d64/d21/d25/d62/fb7 zero size 2024-04-24T22:45:42.791 INFO:tasks.workunit.client.0.smithi063.stdout:3/911: dread da/f29 [0,4194304] 0 2024-04-24T22:45:42.791 INFO:tasks.workunit.client.1.smithi192.stdout:1/614: rename d9/d15/d2d/d31/d2f/f3f to d9/dd/d74/dcd/fcf 0 2024-04-24T22:45:42.791 INFO:tasks.workunit.client.0.smithi063.stdout:0/840: write d6/d64/d21/f48 [3139122,122364] 0 2024-04-24T22:45:42.795 INFO:tasks.workunit.client.0.smithi063.stdout:7/829: mkdir d0/d4/de/d6f/d2e/d55/d106/d117 0 2024-04-24T22:45:42.796 INFO:tasks.workunit.client.1.smithi192.stdout:1/615: dread d9/d15/d2d/d63/fb2 [0,4194304] 0 2024-04-24T22:45:42.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/953: mkdir d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/ddf/d135 0 2024-04-24T22:45:42.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/956: getdents d1/d11/d32/d38/db1 0 2024-04-24T22:45:42.801 INFO:tasks.workunit.client.1.smithi192.stdout:5/641: creat d0/d13/fde x:0 0 0 2024-04-24T22:45:42.803 INFO:tasks.workunit.client.1.smithi192.stdout:0/750: fdatasync d3/d27/f43 0 2024-04-24T22:45:42.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/929: creat d6/d2f/dcd/db/d1c/d78/d9a/ddd/d131/f13b x:0 0 0 2024-04-24T22:45:42.807 INFO:tasks.workunit.client.0.smithi063.stdout:9/930: dread - d6/d2f/dcd/db/d6b/d4f/dad/d48/f124 zero size 2024-04-24T22:45:42.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/748: write db/dc/d29/f2b [3334889,116354] 0 2024-04-24T22:45:42.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/980: mkdir db/d14a 0 2024-04-24T22:45:42.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/677: write d0/d8/d29/f95 [168733,35467] 0 2024-04-24T22:45:42.818 INFO:tasks.workunit.client.1.smithi192.stdout:3/770: symlink d2/d4/lf9 0 2024-04-24T22:45:42.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/744: dread d2/db/d75/da1/d36/d5b/f6e [0,4194304] 0 2024-04-24T22:45:42.822 INFO:tasks.workunit.client.0.smithi063.stdout:1/954: dread d3/de0/d79/d96/d112/fa0 [0,4194304] 0 2024-04-24T22:45:42.823 INFO:tasks.workunit.client.0.smithi063.stdout:1/955: chown d3/db5/d21/d9f/dd5/d106 8045869 1 2024-04-24T22:45:42.826 INFO:tasks.workunit.client.0.smithi063.stdout:4/876: truncate d2/d19/d33/dc1/f6d 218871 0 2024-04-24T22:45:42.841 INFO:tasks.workunit.client.1.smithi192.stdout:7/834: dwrite d2/db/d21/d42/d5b/d63/d80/fcc [0,4194304] 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:3/771: fsync d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/fe4 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/678: rename d0/d8/d3d/d9d/dd6 to d0/d8/d3d/d9d/dd6/ddc/def 22 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:7/835: write d2/db/d21/d42/d67/d9b/d6e/fe4 [1057561,113321] 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:5/642: stat d0/d13/d3b/d2f/f84 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/853: dwrite d4/d29/d60/d9a/f7b [0,4194304] 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:0/751: creat d3/d27/d28/dcc/d5c/d80/d83/ff9 x:0 0 0 2024-04-24T22:45:42.842 INFO:tasks.workunit.client.1.smithi192.stdout:0/752: read - d3/d27/d6a/deb/ff1 zero size 2024-04-24T22:45:42.843 INFO:tasks.workunit.client.0.smithi063.stdout:6/981: mknod db/d1d/d20/d29/dab/c14b 0 2024-04-24T22:45:42.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/830: mknod d0/d4/de/d17/d28/d31/d99/d113/c118 0 2024-04-24T22:45:42.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/956: truncate d3/db5/d21/d9f/dd5/d106/d39/f98 8739201 0 2024-04-24T22:45:42.843 INFO:tasks.workunit.client.1.smithi192.stdout:7/836: dwrite d2/db/d49/f70 [4194304,4194304] 0 2024-04-24T22:45:42.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/982: chown db/d1d/d20/d3e/f4f 8394 1 2024-04-24T22:45:42.845 INFO:tasks.workunit.client.0.smithi063.stdout:9/931: dread d6/d2f/dcd/d1f/d20/d22/f47 [0,4194304] 0 2024-04-24T22:45:42.846 INFO:tasks.workunit.client.0.smithi063.stdout:1/957: chown d3/de0/d79/d96/ld6 485936 1 2024-04-24T22:45:42.847 INFO:tasks.workunit.client.1.smithi192.stdout:6/679: mknod d0/d8/d29/d4c/cf0 0 2024-04-24T22:45:42.849 INFO:tasks.workunit.client.0.smithi063.stdout:8/957: link d1/d2/d3/f59 d1/d2/f12b 0 2024-04-24T22:45:42.849 INFO:tasks.workunit.client.0.smithi063.stdout:5/853: dwrite d5/f9 [0,4194304] 0 2024-04-24T22:45:42.851 INFO:tasks.workunit.client.1.smithi192.stdout:3/772: rmdir d2/d4/d12/d1a/d1d/d55/d5f/db1 39 2024-04-24T22:45:42.854 INFO:tasks.workunit.client.0.smithi063.stdout:3/912: getdents da/d18/d47/d4d/d9b/d125 0 2024-04-24T22:45:42.855 INFO:tasks.workunit.client.0.smithi063.stdout:6/983: mknod db/d1c/ddb/c14c 0 2024-04-24T22:45:42.856 INFO:tasks.workunit.client.0.smithi063.stdout:6/984: chown db/d85/dd2/dd7/c113 1336 1 2024-04-24T22:45:42.857 INFO:tasks.workunit.client.1.smithi192.stdout:5/643: rename d0/d1b/f45 to d0/d13/d3b/d79/fdf 0 2024-04-24T22:45:42.859 INFO:tasks.workunit.client.1.smithi192.stdout:2/854: fsync d4/d9/d10/da7/fbb 0 2024-04-24T22:45:42.867 INFO:tasks.workunit.client.1.smithi192.stdout:5/644: dwrite d0/d13/d3b/fbd [0,4194304] 0 2024-04-24T22:45:42.867 INFO:tasks.workunit.client.1.smithi192.stdout:8/749: link db/dc/f1b db/d43/d76/feb 0 2024-04-24T22:45:42.869 INFO:tasks.workunit.client.0.smithi063.stdout:0/841: dwrite d6/d64/d21/d25/d62/d63/f82 [0,4194304] 0 2024-04-24T22:45:42.872 INFO:tasks.workunit.client.0.smithi063.stdout:3/913: unlink da/d48/c7c 0 2024-04-24T22:45:42.875 INFO:tasks.workunit.client.0.smithi063.stdout:6/985: creat db/d1d/d20/d3f/d7f/d68/f14d x:0 0 0 2024-04-24T22:45:42.877 INFO:tasks.workunit.client.1.smithi192.stdout:1/616: write f3 [148367,127860] 0 2024-04-24T22:45:42.878 INFO:tasks.workunit.client.1.smithi192.stdout:6/680: dread d0/d8/d3d/f49 [0,4194304] 0 2024-04-24T22:45:42.878 INFO:tasks.workunit.client.0.smithi063.stdout:1/958: truncate d3/db5/d21/d9f/dd5/d106/fc4 2850881 0 2024-04-24T22:45:42.880 INFO:tasks.workunit.client.0.smithi063.stdout:7/831: dread d0/d4/fb [0,4194304] 0 2024-04-24T22:45:42.881 INFO:tasks.workunit.client.0.smithi063.stdout:9/932: creat d6/d2f/dcd/d53/dff/d132/d5e/db9/d112/f13c x:0 0 0 2024-04-24T22:45:42.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/773: rmdir d2/d4/d12/d1a/d1d/d55/d5f 39 2024-04-24T22:45:42.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/958: getdents d1/d2/d17/d124 0 2024-04-24T22:45:42.885 INFO:tasks.workunit.client.0.smithi063.stdout:8/959: write d1/f111 [1045000,86817] 0 2024-04-24T22:45:42.885 INFO:tasks.workunit.client.0.smithi063.stdout:4/877: dwrite d2/d8/d36/fb6 [0,4194304] 0 2024-04-24T22:45:42.886 INFO:tasks.workunit.client.0.smithi063.stdout:5/854: getdents d5/d11/d23/d2d/d62/d81/d10c 0 2024-04-24T22:45:42.889 INFO:tasks.workunit.client.1.smithi192.stdout:0/753: rename d3/d6d/led to d3/d27/d28/dcc/da8/lfa 0 2024-04-24T22:45:42.890 INFO:tasks.workunit.client.1.smithi192.stdout:7/837: rename d2/db/d21/d42/d67 to d2/db/d21/d42/d67/dae/dce/d105 22 2024-04-24T22:45:42.891 INFO:tasks.workunit.client.0.smithi063.stdout:0/842: mkdir d6/d64/da0/d98/d11b 0 2024-04-24T22:45:42.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/843: chown d6/d74/f97 371504 1 2024-04-24T22:45:42.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/914: rename da/d39/d94/db3/df3 to da/d18/d47/d4d/d9b/dde/df9/d13b 0 2024-04-24T22:45:42.897 INFO:tasks.workunit.client.1.smithi192.stdout:9/812: getdents d3/dd/de/da7/dae 0 2024-04-24T22:45:42.898 INFO:tasks.workunit.client.0.smithi063.stdout:3/915: fsync da/d18/d47/d4d/f8c 0 2024-04-24T22:45:42.898 INFO:tasks.workunit.client.0.smithi063.stdout:0/844: dwrite d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:42.898 INFO:tasks.workunit.client.1.smithi192.stdout:7/838: dwrite d2/db/d21/d42/f8c [4194304,4194304] 0 2024-04-24T22:45:42.902 INFO:tasks.workunit.client.0.smithi063.stdout:6/986: creat db/d1d/d20/d11e/f14e x:0 0 0 2024-04-24T22:45:42.903 INFO:tasks.workunit.client.0.smithi063.stdout:1/959: symlink d3/da/d34/l136 0 2024-04-24T22:45:42.906 INFO:tasks.workunit.client.1.smithi192.stdout:5/645: creat d0/d13/d3b/d79/fe0 x:0 0 0 2024-04-24T22:45:42.907 INFO:tasks.workunit.client.1.smithi192.stdout:8/750: rmdir db/dc/d22/d31/d33/db7 39 2024-04-24T22:45:42.910 INFO:tasks.workunit.client.0.smithi063.stdout:7/832: chown d0/d4/de/d17/d35/d6c/d7a/laf 1343432245 1 2024-04-24T22:45:42.911 INFO:tasks.workunit.client.1.smithi192.stdout:4/745: creat d2/db/d75/da1/d36/d5b/f106 x:0 0 0 2024-04-24T22:45:42.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/833: write d0/d4/de/d6f/d2e/d76/ff3 [4782721,67583] 0 2024-04-24T22:45:42.915 INFO:tasks.workunit.client.1.smithi192.stdout:6/681: stat d0/f82 0 2024-04-24T22:45:42.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/960: mkdir d1/d11/d32/d38/d12c 0 2024-04-24T22:45:42.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/878: truncate d2/f97 1638582 0 2024-04-24T22:45:42.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/774: rename d2/d4/d12/d1a/d1d/d7b/df7/f8d to d2/d27/dc6/d41/dad/ffa 0 2024-04-24T22:45:42.922 INFO:tasks.workunit.client.0.smithi063.stdout:8/961: dwrite d1/d11/fe4 [0,4194304] 0 2024-04-24T22:45:42.923 INFO:tasks.workunit.client.0.smithi063.stdout:3/916: symlink da/d18/d47/d4d/d9b/dde/df9/d13b/l13c 0 2024-04-24T22:45:42.926 INFO:tasks.workunit.client.1.smithi192.stdout:4/746: dread d2/fd0 [0,4194304] 0 2024-04-24T22:45:42.927 INFO:tasks.workunit.client.0.smithi063.stdout:1/960: creat d3/db5/d21/d9f/dd5/d106/d39/d115/f137 x:0 0 0 2024-04-24T22:45:42.931 INFO:tasks.workunit.client.1.smithi192.stdout:7/839: rename d2/db/d21/d42/d67/l6a to d2/db/d21/d42/d67/d9b/d6e/d77/d8e/l106 0 2024-04-24T22:45:42.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/879: fsync d2/d19/f20 0 2024-04-24T22:45:42.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/880: chown d2/d80/d81/d90 1 1 2024-04-24T22:45:42.934 INFO:tasks.workunit.client.0.smithi063.stdout:5/855: creat d5/d11/d23/d2d/d62/d81/d10c/f127 x:0 0 0 2024-04-24T22:45:42.945 INFO:tasks.workunit.client.0.smithi063.stdout:9/933: rmdir d6/d2f/dcd/d53/dff/d132/d5e/d123 0 2024-04-24T22:45:42.947 INFO:tasks.workunit.client.0.smithi063.stdout:7/834: unlink d0/d4/de/f12 0 2024-04-24T22:45:42.952 INFO:tasks.workunit.client.1.smithi192.stdout:1/617: dwrite d9/dd/ff [0,4194304] 0 2024-04-24T22:45:42.954 INFO:tasks.workunit.client.1.smithi192.stdout:0/754: write d3/d27/d28/d31/f52 [4569283,63224] 0 2024-04-24T22:45:42.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/881: rename d2/d8/d36/d118/c126 to d2/d8/d40/d51/dd5/df6/c127 0 2024-04-24T22:45:42.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/961: dread d3/db5/d21/d9f/dd5/d106/d39/f65 [0,4194304] 0 2024-04-24T22:45:42.961 INFO:tasks.workunit.client.0.smithi063.stdout:8/962: mknod d1/c12d 0 2024-04-24T22:45:42.966 INFO:tasks.workunit.client.0.smithi063.stdout:9/934: mknod d6/d2f/dcd/db/d6b/d4f/dad/c13d 0 2024-04-24T22:45:42.966 INFO:tasks.workunit.client.0.smithi063.stdout:6/987: dwrite db/d1d/d2d/f42 [0,4194304] 0 2024-04-24T22:45:42.968 INFO:tasks.workunit.client.0.smithi063.stdout:7/835: symlink d0/d4/de/d17/d28/d31/l119 0 2024-04-24T22:45:42.970 INFO:tasks.workunit.client.0.smithi063.stdout:5/856: fsync d5/d11/d23/fd5 0 2024-04-24T22:45:42.971 INFO:tasks.workunit.client.0.smithi063.stdout:5/857: fsync d5/d11/d76/fbc 0 2024-04-24T22:45:42.972 INFO:tasks.workunit.client.0.smithi063.stdout:0/845: getdents d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:42.978 INFO:tasks.workunit.client.0.smithi063.stdout:8/963: mkdir d1/d11/d32/d38/db7/dc1/d12e 0 2024-04-24T22:45:42.980 INFO:tasks.workunit.client.0.smithi063.stdout:0/846: dread d6/d64/d21/d25/f55 [0,4194304] 0 2024-04-24T22:45:42.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/935: dread d6/d2f/dcd/d1f/d20/d2a/f66 [0,4194304] 0 2024-04-24T22:45:42.982 INFO:tasks.workunit.client.0.smithi063.stdout:9/936: truncate d6/d2f/dcd/d53/f99 1820742 0 2024-04-24T22:45:42.982 INFO:tasks.workunit.client.0.smithi063.stdout:6/988: rename db/d1d/d20/d3f/d5f/lb3 to db/d1c/dc3/l14f 0 2024-04-24T22:45:42.983 INFO:tasks.workunit.client.1.smithi192.stdout:8/751: write db/dc/d22/f28 [1613086,111373] 0 2024-04-24T22:45:42.984 INFO:tasks.workunit.client.1.smithi192.stdout:9/813: dread d3/d24/f41 [0,4194304] 0 2024-04-24T22:45:42.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/646: mkdir d0/d13/dd1/de1 0 2024-04-24T22:45:42.997 INFO:tasks.workunit.client.1.smithi192.stdout:6/682: truncate d0/f7f 216717 0 2024-04-24T22:45:42.999 INFO:tasks.workunit.client.1.smithi192.stdout:7/840: rename d2/db/d21/d42/f4d to d2/db/d21/d42/d67/d9b/d6e/db5/db6/f107 0 2024-04-24T22:45:43.000 INFO:tasks.workunit.client.1.smithi192.stdout:7/841: dread - d2/db/d21/d42/d67/da8/dc7/feb zero size 2024-04-24T22:45:43.001 INFO:tasks.workunit.client.1.smithi192.stdout:7/842: chown d2/db/d21/d42/d5b/d63/d80/laf 406 1 2024-04-24T22:45:43.004 INFO:tasks.workunit.client.1.smithi192.stdout:2/855: getdents d4/d9/d10/d6c/d8c 0 2024-04-24T22:45:43.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/836: mkdir d0/d4/de/d17/d28/d11a 0 2024-04-24T22:45:43.007 INFO:tasks.workunit.client.0.smithi063.stdout:3/917: getdents da/d18/d47/d4d/d9b/dde/df9 0 2024-04-24T22:45:43.008 INFO:tasks.workunit.client.0.smithi063.stdout:8/964: creat d1/d11/d32/d6a/f12f x:0 0 0 2024-04-24T22:45:43.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/847: mknod d6/d64/d24/d42/dd2/c11c 0 2024-04-24T22:45:43.009 INFO:tasks.workunit.client.1.smithi192.stdout:1/618: rename d9/f6a to d9/d15/d2d/d4a/fd0 0 2024-04-24T22:45:43.011 INFO:tasks.workunit.client.0.smithi063.stdout:6/989: mkdir db/d1d/d8d/dfc/d150 0 2024-04-24T22:45:43.012 INFO:tasks.workunit.client.1.smithi192.stdout:3/775: write d2/d27/d28/f36 [550385,107894] 0 2024-04-24T22:45:43.014 INFO:tasks.workunit.client.1.smithi192.stdout:9/814: unlink d3/dd/de/d17/c76 0 2024-04-24T22:45:43.022 INFO:tasks.workunit.client.1.smithi192.stdout:6/683: mkdir d0/d8/d18/df1 0 2024-04-24T22:45:43.024 INFO:tasks.workunit.client.0.smithi063.stdout:8/965: mkdir d1/d11/d8b/d130 0 2024-04-24T22:45:43.025 INFO:tasks.workunit.client.1.smithi192.stdout:2/856: write d4/f41 [1834460,13238] 0 2024-04-24T22:45:43.026 INFO:tasks.workunit.client.1.smithi192.stdout:8/752: dwrite f8 [0,4194304] 0 2024-04-24T22:45:43.027 INFO:tasks.workunit.client.0.smithi063.stdout:0/848: rmdir d6/d64/d24/d42/dd4 39 2024-04-24T22:45:43.030 INFO:tasks.workunit.client.1.smithi192.stdout:1/619: rename d9/dd/l60 to d9/d15/d2d/d48/d93/da4/dbb/dac/ld1 0 2024-04-24T22:45:43.032 INFO:tasks.workunit.client.0.smithi063.stdout:7/837: dwrite d0/d4/de/d17/d35/d69/db1/fe2 [0,4194304] 0 2024-04-24T22:45:43.033 INFO:tasks.workunit.client.0.smithi063.stdout:7/838: fsync d0/d2f/f107 0 2024-04-24T22:45:43.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/990: rmdir db/d1d/d20/d84/d100 39 2024-04-24T22:45:43.035 INFO:tasks.workunit.client.0.smithi063.stdout:4/882: getdents d2/d19/d33/dc1/d50 0 2024-04-24T22:45:43.036 INFO:tasks.workunit.client.1.smithi192.stdout:9/815: dread d3/dd/d53/f5c [0,4194304] 0 2024-04-24T22:45:43.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/776: fsync d2/d27/dc6/fd 0 2024-04-24T22:45:43.037 INFO:tasks.workunit.client.0.smithi063.stdout:7/839: read d0/d4/de/d17/d35/d69/db1/fc1 [449329,120963] 0 2024-04-24T22:45:43.042 INFO:tasks.workunit.client.0.smithi063.stdout:3/918: mkdir da/d13d 0 2024-04-24T22:45:43.043 INFO:tasks.workunit.client.1.smithi192.stdout:7/843: mkdir d2/db/d21/d42/d67/d9b/d108 0 2024-04-24T22:45:43.045 INFO:tasks.workunit.client.0.smithi063.stdout:1/962: truncate d3/da/d34/d68/fa8 1494188 0 2024-04-24T22:45:43.052 INFO:tasks.workunit.client.0.smithi063.stdout:6/991: mknod db/d1d/d2d/d69/dd4/de4/c151 0 2024-04-24T22:45:43.054 INFO:tasks.workunit.client.1.smithi192.stdout:8/753: rename db/dc/d29/c9c to db/dc/d22/da3/cec 0 2024-04-24T22:45:43.055 INFO:tasks.workunit.client.0.smithi063.stdout:0/849: dread d6/d74/f97 [0,4194304] 0 2024-04-24T22:45:43.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/620: dread - d9/dd/d74/f7a zero size 2024-04-24T22:45:43.057 INFO:tasks.workunit.client.1.smithi192.stdout:5/647: write d0/d13/d19/d64/d4f/d52/f6a [513823,61693] 0 2024-04-24T22:45:43.063 INFO:tasks.workunit.client.1.smithi192.stdout:0/755: getdents d3/d27/d3c/d6c 0 2024-04-24T22:45:43.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/857: write d4/d9/f92 [266684,43804] 0 2024-04-24T22:45:43.065 INFO:tasks.workunit.client.1.smithi192.stdout:2/858: truncate d4/d9/d37/ff4 155668 0 2024-04-24T22:45:43.069 INFO:tasks.workunit.client.1.smithi192.stdout:9/816: creat d3/dd/de/dba/f112 x:0 0 0 2024-04-24T22:45:43.071 INFO:tasks.workunit.client.1.smithi192.stdout:9/817: chown d3/dd/de/d17/f2b 0 1 2024-04-24T22:45:43.071 INFO:tasks.workunit.client.1.smithi192.stdout:2/859: dwrite d4/d29/f2b [0,4194304] 0 2024-04-24T22:45:43.072 INFO:tasks.workunit.client.0.smithi063.stdout:9/937: getdents d6/d2f/dcd/db/d1c/d78/d9a/daa 0 2024-04-24T22:45:43.073 INFO:tasks.workunit.client.1.smithi192.stdout:9/818: chown d3/dd/de/d17/c87 43751 1 2024-04-24T22:45:43.077 INFO:tasks.workunit.client.1.smithi192.stdout:7/844: readlink d2/db/d25/l3b 0 2024-04-24T22:45:43.079 INFO:tasks.workunit.client.0.smithi063.stdout:8/966: dwrite d1/d11/d32/d38/fa0 [0,4194304] 0 2024-04-24T22:45:43.082 INFO:tasks.workunit.client.0.smithi063.stdout:6/992: write db/d1d/d20/d3f/d5f/d93/fcf [521650,75625] 0 2024-04-24T22:45:43.082 INFO:tasks.workunit.client.0.smithi063.stdout:0/850: symlink d6/d64/d21/d25/d62/d63/d89/d95/da4/l11d 0 2024-04-24T22:45:43.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/919: mkdir da/d39/d94/db3/dfa/d13e 0 2024-04-24T22:45:43.083 INFO:tasks.workunit.client.0.smithi063.stdout:1/963: dwrite d3/da/f5c [4194304,4194304] 0 2024-04-24T22:45:43.088 INFO:tasks.workunit.client.1.smithi192.stdout:1/621: rename d9/d15/d2d/d63/d68/da7 to d9/d15/d2d/d48/d93/da4/dd2 0 2024-04-24T22:45:43.090 INFO:tasks.workunit.client.1.smithi192.stdout:0/756: mkdir d3/d27/d28/dcc/d5c/d80/d83/dfb 0 2024-04-24T22:45:43.092 INFO:tasks.workunit.client.0.smithi063.stdout:4/883: write d2/d19/d33/dc1/d50/fe7 [709175,37822] 0 2024-04-24T22:45:43.092 INFO:tasks.workunit.client.0.smithi063.stdout:1/964: dwrite d3/de0/d79/ff0 [0,4194304] 0 2024-04-24T22:45:43.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/938: mknod d6/d2f/dcd/d1f/c13e 0 2024-04-24T22:45:43.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/939: write d6/d2f/dcd/d8a/f120 [19285,113531] 0 2024-04-24T22:45:43.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/819: read - d3/dd/d53/fd5 zero size 2024-04-24T22:45:43.101 INFO:tasks.workunit.client.0.smithi063.stdout:4/884: dwrite d2/d19/d33/dc1/d50/f5d [0,4194304] 0 2024-04-24T22:45:43.101 INFO:tasks.workunit.client.0.smithi063.stdout:5/858: sync 2024-04-24T22:45:43.102 INFO:tasks.workunit.client.1.smithi192.stdout:8/754: write db/dc/f1b [12672555,76077] 0 2024-04-24T22:45:43.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/993: truncate db/d1d/d2d/f109 1618365 0 2024-04-24T22:45:43.109 INFO:tasks.workunit.client.0.smithi063.stdout:4/885: dwrite d2/d80/d81/d90/fe1 [0,4194304] 0 2024-04-24T22:45:43.109 INFO:tasks.workunit.client.0.smithi063.stdout:6/994: write db/d1d/d20/d3f/d5f/d93/fcf [43692,20324] 0 2024-04-24T22:45:43.112 INFO:tasks.workunit.client.0.smithi063.stdout:0/851: write d6/d64/d21/d25/d62/d63/d89/d95/fdc [2787925,121511] 0 2024-04-24T22:45:43.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/820: dread d3/f8d [0,4194304] 0 2024-04-24T22:45:43.123 INFO:tasks.workunit.client.1.smithi192.stdout:6/684: getdents d0/d8/d3d 0 2024-04-24T22:45:43.123 INFO:tasks.workunit.client.1.smithi192.stdout:6/685: readlink d0/d8/d22/l6c 0 2024-04-24T22:45:43.124 INFO:tasks.workunit.client.0.smithi063.stdout:5/859: fdatasync d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f58 0 2024-04-24T22:45:43.129 INFO:tasks.workunit.client.1.smithi192.stdout:9/821: rmdir d3/d59 39 2024-04-24T22:45:43.130 INFO:tasks.workunit.client.0.smithi063.stdout:0/852: creat d6/d64/d21/d25/d62/d63/d89/d95/df6/f11e x:0 0 0 2024-04-24T22:45:43.131 INFO:tasks.workunit.client.1.smithi192.stdout:0/757: link d3/c5f d3/d27/d28/dcc/da8/def/cfc 0 2024-04-24T22:45:43.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/860: getdents d4/d9/d10/d6c 0 2024-04-24T22:45:43.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/861: stat d4/d9/d10/da7/df5 0 2024-04-24T22:45:43.134 INFO:tasks.workunit.client.1.smithi192.stdout:9/822: fsync d3/dd/dce/fec 0 2024-04-24T22:45:43.135 INFO:tasks.workunit.client.0.smithi063.stdout:4/886: symlink d2/d8/d36/l128 0 2024-04-24T22:45:43.139 INFO:tasks.workunit.client.0.smithi063.stdout:1/965: creat d3/de0/f138 x:0 0 0 2024-04-24T22:45:43.143 INFO:tasks.workunit.client.0.smithi063.stdout:9/940: rename d6/d2f/dcd/d53/dff/d132/d5e to d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f 0 2024-04-24T22:45:43.144 INFO:tasks.workunit.client.1.smithi192.stdout:2/862: mknod d4/d9/d37/c117 0 2024-04-24T22:45:43.144 INFO:tasks.workunit.client.1.smithi192.stdout:1/622: dwrite d9/d51/d70/f95 [0,4194304] 0 2024-04-24T22:45:43.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/920: dwrite da/d18/d9e/da3/dc1/ff6 [0,4194304] 0 2024-04-24T22:45:43.145 INFO:tasks.workunit.client.1.smithi192.stdout:1/623: chown d9/d15/d5e/d90 103 1 2024-04-24T22:45:43.146 INFO:tasks.workunit.client.0.smithi063.stdout:6/995: dread db/d1d/d20/d3f/d7f/d68/fe9 [0,4194304] 0 2024-04-24T22:45:43.146 INFO:tasks.workunit.client.0.smithi063.stdout:9/941: chown d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/d9d/d100/d11c 385 1 2024-04-24T22:45:43.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/840: sync 2024-04-24T22:45:43.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/996: write db/d1d/d20/d3f/d5f/d93/fcf [886107,12211] 0 2024-04-24T22:45:43.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/997: chown c2 1560446 1 2024-04-24T22:45:43.149 INFO:tasks.workunit.client.0.smithi063.stdout:5/860: creat d5/d11/d23/d2d/d62/d81/da4/dee/f128 x:0 0 0 2024-04-24T22:45:43.149 INFO:tasks.workunit.client.1.smithi192.stdout:9/823: fsync d3/dd/de/dba/f81 0 2024-04-24T22:45:43.150 INFO:tasks.workunit.client.1.smithi192.stdout:7/845: dwrite d2/db/d36/f3d [0,4194304] 0 2024-04-24T22:45:43.151 INFO:tasks.workunit.client.0.smithi063.stdout:4/887: readlink d2/d80/lcb 0 2024-04-24T22:45:43.152 INFO:tasks.workunit.client.1.smithi192.stdout:8/755: getdents db/dc/d22/dca 0 2024-04-24T22:45:43.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/758: symlink d3/d27/d28/dcc/lfd 0 2024-04-24T22:45:43.168 INFO:tasks.workunit.client.0.smithi063.stdout:9/942: fdatasync d6/d2f/d42/d81/fa8 0 2024-04-24T22:45:43.173 INFO:tasks.workunit.client.0.smithi063.stdout:5/861: fdatasync d5/d11/d23/d2d/d62/d73/d118/f3d 0 2024-04-24T22:45:43.175 INFO:tasks.workunit.client.1.smithi192.stdout:8/756: mkdir db/d43/d7c/ded 0 2024-04-24T22:45:43.176 INFO:tasks.workunit.client.1.smithi192.stdout:8/757: write db/d43/d76/feb [11839165,66647] 0 2024-04-24T22:45:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:42 smithi192 ceph-mon[33991]: pgmap v19: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 128 MiB/s rd, 296 MiB/s wr, 645 op/s 2024-04-24T22:45:43.177 INFO:tasks.workunit.client.0.smithi063.stdout:1/966: symlink d3/db5/d21/d9f/dd5/d106/d11/d5e/l139 0 2024-04-24T22:45:43.178 INFO:tasks.workunit.client.0.smithi063.stdout:0/853: link d6/d64/d21/d25/d62/cfe d6/d74/c11f 0 2024-04-24T22:45:43.179 INFO:tasks.workunit.client.1.smithi192.stdout:1/624: getdents d9/d15/d2d/d48/d93/da4/dd2/dbf 0 2024-04-24T22:45:43.179 INFO:tasks.workunit.client.0.smithi063.stdout:0/854: stat d6/d64/d21/d25/d62/fb7 0 2024-04-24T22:45:43.180 INFO:tasks.workunit.client.0.smithi063.stdout:8/967: sync 2024-04-24T22:45:43.181 INFO:tasks.workunit.client.1.smithi192.stdout:7/846: mknod d2/db/d21/d42/d67/dd4/c109 0 2024-04-24T22:45:43.185 INFO:tasks.workunit.client.0.smithi063.stdout:0/855: dwrite d6/d64/d21/d25/d62/d63/d89/d95/fdc [0,4194304] 0 2024-04-24T22:45:43.186 INFO:tasks.workunit.client.1.smithi192.stdout:2/863: rename d4/d9/d37/de5/df3 to d4/d9/d10/d118 0 2024-04-24T22:45:43.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/847: fsync d2/f1a 0 2024-04-24T22:45:43.191 INFO:tasks.workunit.client.0.smithi063.stdout:5/862: mknod d5/d11/d23/d2d/d62/d81/d10c/c129 0 2024-04-24T22:45:43.196 INFO:tasks.workunit.client.1.smithi192.stdout:0/759: creat d3/d27/d28/dcc/ffe x:0 0 0 2024-04-24T22:45:43.196 INFO:tasks.workunit.client.1.smithi192.stdout:1/625: creat d9/d15/d5e/d7c/fd3 x:0 0 0 2024-04-24T22:45:43.198 INFO:tasks.workunit.client.1.smithi192.stdout:1/626: chown d9/d15/d5e/d90 2877 1 2024-04-24T22:45:43.199 INFO:tasks.workunit.client.0.smithi063.stdout:1/967: rmdir d3/da/da2 39 2024-04-24T22:45:43.199 INFO:tasks.workunit.client.0.smithi063.stdout:6/998: write db/d1d/d20/d3f/d7f/fac [195592,25859] 0 2024-04-24T22:45:43.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/968: write d3/db5/d21/d9f/dd5/d106/d11/dc6/f12d [874119,5467] 0 2024-04-24T22:45:43.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/760: read d3/d27/d28/f34 [412649,69788] 0 2024-04-24T22:45:43.201 INFO:tasks.workunit.client.1.smithi192.stdout:2/864: rename d4/d9/d10/d6c/d6f/da8/f105 to d4/f119 0 2024-04-24T22:45:43.202 INFO:tasks.workunit.client.1.smithi192.stdout:0/761: stat d3/d27/d3c/fe3 0 2024-04-24T22:45:43.203 INFO:tasks.workunit.client.1.smithi192.stdout:0/762: write d3/d6d/d79/fd5 [280670,117653] 0 2024-04-24T22:45:43.203 INFO:tasks.workunit.client.0.smithi063.stdout:4/888: write d2/d8/f23 [448038,45905] 0 2024-04-24T22:45:43.204 INFO:tasks.workunit.client.1.smithi192.stdout:0/763: fsync d3/d27/d28/dcc/d5c/d8a/fad 0 2024-04-24T22:45:43.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/921: dread da/d40/fe0 [0,4194304] 0 2024-04-24T22:45:43.208 INFO:tasks.workunit.client.0.smithi063.stdout:7/841: rmdir d0/d4/d10b 0 2024-04-24T22:45:43.208 INFO:tasks.workunit.client.1.smithi192.stdout:9/824: truncate d3/dd/f6c 383264 0 2024-04-24T22:45:43.212 INFO:tasks.workunit.client.0.smithi063.stdout:0/856: creat d6/d64/d21/d25/d62/d63/d89/d95/da4/f120 x:0 0 0 2024-04-24T22:45:43.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/943: write d6/d2f/dcd/f71 [80492,64963] 0 2024-04-24T22:45:43.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/968: dread d1/d22/d33/da6/d54/f8e [0,4194304] 0 2024-04-24T22:45:43.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/865: rename d4/d9/d10/c43 to d4/d9/d10/d6c/d8c/c11a 0 2024-04-24T22:45:43.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/866: dread - d4/d9/d10/da7/fbb zero size 2024-04-24T22:45:43.221 INFO:tasks.workunit.client.0.smithi063.stdout:8/969: dwrite d1/d2/d17/dd1/f113 [0,4194304] 0 2024-04-24T22:45:43.222 INFO:tasks.workunit.client.0.smithi063.stdout:1/969: creat d3/de0/d79/d96/df8/f13a x:0 0 0 2024-04-24T22:45:43.223 INFO:tasks.workunit.client.1.smithi192.stdout:0/764: mknod d3/d27/d3c/d64/cff 0 2024-04-24T22:45:43.223 INFO:tasks.workunit.client.0.smithi063.stdout:8/970: dread d1/d11/d32/f100 [0,4194304] 0 2024-04-24T22:45:43.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/971: truncate d1/d11/d32/fb2 4497847 0 2024-04-24T22:45:43.225 INFO:tasks.workunit.client.1.smithi192.stdout:0/765: readlink d3/d27/d3c/d6c/l7e 0 2024-04-24T22:45:43.229 INFO:tasks.workunit.client.0.smithi063.stdout:4/889: creat d2/d8/d36/d118/f129 x:0 0 0 2024-04-24T22:45:43.235 INFO:tasks.workunit.client.0.smithi063.stdout:3/922: stat da/d18/d3a/fc7 0 2024-04-24T22:45:43.237 INFO:tasks.workunit.client.1.smithi192.stdout:1/627: symlink d9/d15/d2d/d48/d93/da4/dd2/dbf/ld4 0 2024-04-24T22:45:43.238 INFO:tasks.workunit.client.0.smithi063.stdout:5/863: write d5/d1d/dc0/dbb/f3e [1893045,101491] 0 2024-04-24T22:45:43.242 INFO:tasks.workunit.client.0.smithi063.stdout:7/842: dwrite d0/d4/de/f7c [0,4194304] 0 2024-04-24T22:45:43.243 INFO:tasks.workunit.client.1.smithi192.stdout:0/766: symlink d3/d6d/d79/l100 0 2024-04-24T22:45:43.247 INFO:tasks.workunit.client.1.smithi192.stdout:9/825: fsync d3/dd/f49 0 2024-04-24T22:45:43.247 INFO:tasks.workunit.client.0.smithi063.stdout:6/999: dwrite db/d1d/d2d/d48/f6e [4194304,4194304] 0 2024-04-24T22:45:43.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/867: dread d4/d29/d60/f89 [0,4194304] 0 2024-04-24T22:45:43.253 INFO:tasks.workunit.client.0.smithi063.stdout:0/857: dread d6/d3a/f7a [0,4194304] 0 2024-04-24T22:45:43.270 INFO:tasks.workunit.client.1.smithi192.stdout:7/848: dwrite d2/db/d36/f73 [0,4194304] 0 2024-04-24T22:45:43.283 INFO:tasks.workunit.client.0.smithi063.stdout:8/972: rename d1/d21/fb3 to d1/d11/d32/dd3/f131 0 2024-04-24T22:45:43.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/973: write d1/d11/d32/dd3/fe8 [4673131,108631] 0 2024-04-24T22:45:43.285 INFO:tasks.workunit.client.1.smithi192.stdout:7/849: fdatasync d2/db/d21/d42/d5b/f5d 0 2024-04-24T22:45:43.287 INFO:tasks.workunit.client.1.smithi192.stdout:7/850: chown d2/db/d21/d42/d67/d9b/d6e/db5/db6/lfc 57 1 2024-04-24T22:45:43.288 INFO:tasks.workunit.client.0.smithi063.stdout:7/843: creat d0/d4/de/d6f/d2e/d55/d106/d10e/f11b x:0 0 0 2024-04-24T22:45:43.289 INFO:tasks.workunit.client.1.smithi192.stdout:1/628: getdents d9/d15/d2d/d48/d93/da4/db5 0 2024-04-24T22:45:43.289 INFO:tasks.workunit.client.0.smithi063.stdout:7/844: chown d0/d4/de/d6f/l70 126642870 1 2024-04-24T22:45:43.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/629: fsync f3 0 2024-04-24T22:45:43.294 INFO:tasks.workunit.client.0.smithi063.stdout:1/970: truncate d3/db5/d21/d49/fe2 1125662 0 2024-04-24T22:45:43.295 INFO:tasks.workunit.client.1.smithi192.stdout:7/851: dwrite d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 [0,4194304] 0 2024-04-24T22:45:43.295 INFO:tasks.workunit.client.0.smithi063.stdout:1/971: write d3/db5/d21/d9f/dd5/d106/d39/d115/f119 [988590,35068] 0 2024-04-24T22:45:43.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/864: creat d5/d1d/d7d/d67/d115/f12a x:0 0 0 2024-04-24T22:45:43.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:42 smithi063 ceph-mon[26912]: pgmap v19: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 128 MiB/s rd, 296 MiB/s wr, 645 op/s 2024-04-24T22:45:43.303 INFO:tasks.workunit.client.1.smithi192.stdout:0/767: write d3/d27/d28/dcc/d5c/f59 [1289629,39699] 0 2024-04-24T22:45:43.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/768: chown d3/d27/d28/d31/dbc/dea 25303 1 2024-04-24T22:45:43.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/852: dwrite d2/db/d21/f28 [0,4194304] 0 2024-04-24T22:45:43.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/972: dread d3/db5/d21/d9f/dd5/d106/d11/d5e/d9c/dcb/f113 [0,4194304] 0 2024-04-24T22:45:43.308 INFO:tasks.workunit.client.1.smithi192.stdout:0/769: readlink d3/d27/d28/d2e/d3f/l77 0 2024-04-24T22:45:43.312 INFO:tasks.workunit.client.1.smithi192.stdout:7/853: write d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb9 [1795090,5521] 0 2024-04-24T22:45:43.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/944: write d6/d2f/dcd/db/d6b/d4f/dad/d36/f10e [677551,45784] 0 2024-04-24T22:45:43.316 INFO:tasks.workunit.client.1.smithi192.stdout:2/868: dwrite d4/fbd [0,4194304] 0 2024-04-24T22:45:43.317 INFO:tasks.workunit.client.0.smithi063.stdout:0/858: write d6/dd/f3f [1554196,442] 0 2024-04-24T22:45:43.318 INFO:tasks.workunit.client.1.smithi192.stdout:1/630: dread d9/d15/d2d/d31/d2f/f85 [0,4194304] 0 2024-04-24T22:45:43.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/826: truncate d3/d5d/fdc 2981653 0 2024-04-24T22:45:43.321 INFO:tasks.workunit.client.1.smithi192.stdout:7/854: creat d2/db/d21/d42/d5b/d63/f10a x:0 0 0 2024-04-24T22:45:43.327 INFO:tasks.workunit.client.1.smithi192.stdout:1/631: chown d9/c84 26299892 1 2024-04-24T22:45:43.331 INFO:tasks.workunit.client.1.smithi192.stdout:7/855: creat d2/db/d21/d42/d5b/d63/f10b x:0 0 0 2024-04-24T22:45:43.346 INFO:tasks.workunit.client.1.smithi192.stdout:9/827: link d3/dd/de/d17/d89/d8b/f82 d3/d59/f113 0 2024-04-24T22:45:43.349 INFO:tasks.workunit.client.1.smithi192.stdout:2/869: write d4/d9/d10/d6c/d6f/da8/fc1 [96627,106074] 0 2024-04-24T22:45:43.353 INFO:tasks.workunit.client.1.smithi192.stdout:2/870: creat d4/d29/d60/f11b x:0 0 0 2024-04-24T22:45:43.355 INFO:tasks.workunit.client.1.smithi192.stdout:1/632: write d9/f2c [2964956,17740] 0 2024-04-24T22:45:43.356 INFO:tasks.workunit.client.1.smithi192.stdout:2/871: unlink d4/d29/fdc 0 2024-04-24T22:45:43.358 INFO:tasks.workunit.client.1.smithi192.stdout:7/856: write d2/db/d21/d42/d5b/f5d [520230,4484] 0 2024-04-24T22:45:43.358 INFO:tasks.workunit.client.1.smithi192.stdout:1/633: mkdir d9/d15/d2d/d63/d68/dd5 0 2024-04-24T22:45:43.361 INFO:tasks.workunit.client.1.smithi192.stdout:2/872: creat d4/d29/d60/f11c x:0 0 0 2024-04-24T22:45:43.362 INFO:tasks.workunit.client.1.smithi192.stdout:2/873: chown d4/d9/d10/d6c/d8c/da5 2571 1 2024-04-24T22:45:43.366 INFO:tasks.workunit.client.1.smithi192.stdout:7/857: mknod d2/db/d21/d42/d5b/d63/d80/c10c 0 2024-04-24T22:45:43.368 INFO:tasks.workunit.client.0.smithi063.stdout:7/845: sync 2024-04-24T22:45:43.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/846: write d0/d4/de/d6f/d5c/d9c/ffa [88117,105716] 0 2024-04-24T22:45:43.370 INFO:tasks.workunit.client.1.smithi192.stdout:9/828: write d3/dd/de/d17/f4f [942574,88213] 0 2024-04-24T22:45:43.371 INFO:tasks.workunit.client.1.smithi192.stdout:7/858: chown d2/db/d2a/d57/l5f 6784 1 2024-04-24T22:45:43.371 INFO:tasks.workunit.client.1.smithi192.stdout:1/634: dread d9/f2c [0,4194304] 0 2024-04-24T22:45:43.372 INFO:tasks.workunit.client.1.smithi192.stdout:7/859: chown d2/db/c30 35367 1 2024-04-24T22:45:43.373 INFO:tasks.workunit.client.1.smithi192.stdout:1/635: write d9/d15/f1f [4942556,31125] 0 2024-04-24T22:45:43.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/874: creat d4/d29/f11d x:0 0 0 2024-04-24T22:45:43.378 INFO:tasks.workunit.client.1.smithi192.stdout:7/860: dread d2/db/d25/dda/fdb [0,4194304] 0 2024-04-24T22:45:43.379 INFO:tasks.workunit.client.1.smithi192.stdout:1/636: dread d9/f14 [0,4194304] 0 2024-04-24T22:45:43.386 INFO:tasks.workunit.client.1.smithi192.stdout:7/861: creat d2/db/d21/d42/d67/dd4/f10d x:0 0 0 2024-04-24T22:45:43.391 INFO:tasks.workunit.client.1.smithi192.stdout:2/875: rename d4/d9/d37/f8d to d4/d9/d10/d6c/d6f/da8/f11e 0 2024-04-24T22:45:43.393 INFO:tasks.workunit.client.1.smithi192.stdout:7/862: dwrite d2/db/d49/f70 [4194304,4194304] 0 2024-04-24T22:45:43.393 INFO:tasks.workunit.client.1.smithi192.stdout:2/876: write d4/d29/d60/d9a/db3/fd6 [4083561,108094] 0 2024-04-24T22:45:43.395 INFO:tasks.workunit.client.1.smithi192.stdout:7/863: chown d2/db/f20 4 1 2024-04-24T22:45:43.398 INFO:tasks.workunit.client.1.smithi192.stdout:7/864: write d2/db/d21/d42/d5b/d63/f101 [629181,22926] 0 2024-04-24T22:45:43.402 INFO:tasks.workunit.client.0.smithi063.stdout:3/923: rename da/c16 to da/d18/d9e/da3/dc1/c13f 0 2024-04-24T22:45:43.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/637: dread d9/d15/d2d/d31/d2f/f3a [0,4194304] 0 2024-04-24T22:45:43.406 INFO:tasks.workunit.client.0.smithi063.stdout:8/974: truncate d1/d22/d33/da6/f7c 3582857 0 2024-04-24T22:45:43.406 INFO:tasks.workunit.client.1.smithi192.stdout:9/829: write d3/dd/de/d17/f8f [355868,91499] 0 2024-04-24T22:45:43.408 INFO:tasks.workunit.client.1.smithi192.stdout:9/830: write d3/dd/de/f55 [1309173,20913] 0 2024-04-24T22:45:43.409 INFO:tasks.workunit.client.1.smithi192.stdout:9/831: stat d3/dd/de/f55 0 2024-04-24T22:45:43.414 INFO:tasks.workunit.client.0.smithi063.stdout:1/973: symlink d3/db5/d21/d9f/dd5/l13b 0 2024-04-24T22:45:43.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/974: write d3/db5/d21/d9f/f118 [1234597,71534] 0 2024-04-24T22:45:43.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/945: mknod d6/d2f/dcd/d1f/d20/d2a/c140 0 2024-04-24T22:45:43.417 INFO:tasks.workunit.client.0.smithi063.stdout:0/859: symlink d6/d64/d21/d25/d31/l121 0 2024-04-24T22:45:43.418 INFO:tasks.workunit.client.1.smithi192.stdout:7/865: creat d2/db/d25/f10e x:0 0 0 2024-04-24T22:45:43.419 INFO:tasks.workunit.client.1.smithi192.stdout:7/866: write d2/db/d36/ff0 [247346,52020] 0 2024-04-24T22:45:43.423 INFO:tasks.workunit.client.0.smithi063.stdout:4/890: getdents d2/d19/d33/dc1 0 2024-04-24T22:45:43.426 INFO:tasks.workunit.client.1.smithi192.stdout:7/867: rmdir d2/db/d21/dd5 39 2024-04-24T22:45:43.428 INFO:tasks.workunit.client.0.smithi063.stdout:8/975: truncate d1/d11/d32/d38/f8c 169441 0 2024-04-24T22:45:43.429 INFO:tasks.workunit.client.1.smithi192.stdout:9/832: creat d3/d59/f114 x:0 0 0 2024-04-24T22:45:43.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/877: write d4/d9/d37/f10b [1314447,113325] 0 2024-04-24T22:45:43.436 INFO:tasks.workunit.client.0.smithi063.stdout:5/865: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f58 [0,4194304] 0 2024-04-24T22:45:43.437 INFO:tasks.workunit.client.1.smithi192.stdout:2/878: unlink d4/d9/d10/da7/fbb 0 2024-04-24T22:45:43.437 INFO:tasks.workunit.client.1.smithi192.stdout:7/868: dwrite d2/db/d21/d42/d67/d9b/d6e/db5/db6/fa7 [0,4194304] 0 2024-04-24T22:45:43.440 INFO:tasks.workunit.client.1.smithi192.stdout:7/869: dread - d2/db/d21/d42/d67/d9b/d6e/db5/fea zero size 2024-04-24T22:45:43.441 INFO:tasks.workunit.client.0.smithi063.stdout:0/860: fdatasync d6/d64/d21/d25/d62/d63/d7c/f109 0 2024-04-24T22:45:43.445 INFO:tasks.workunit.client.1.smithi192.stdout:1/638: truncate d9/d51/d70/f94 3507877 0 2024-04-24T22:45:43.447 INFO:tasks.workunit.client.1.smithi192.stdout:2/879: unlink d4/d29/d60/f89 0 2024-04-24T22:45:43.449 INFO:tasks.workunit.client.0.smithi063.stdout:3/924: mknod da/d39/d94/db3/c140 0 2024-04-24T22:45:43.452 INFO:tasks.workunit.client.1.smithi192.stdout:9/833: link d3/d59/d7d/d9b/dd1/f105 d3/dd/f115 0 2024-04-24T22:45:43.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/891: mknod d2/d19/d33/d9d/da8/dec/c12a 0 2024-04-24T22:45:43.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/976: symlink d1/d2/d3/d86/l132 0 2024-04-24T22:45:43.456 INFO:tasks.workunit.client.0.smithi063.stdout:8/977: write d1/d11/f47 [4864057,40732] 0 2024-04-24T22:45:43.457 INFO:tasks.workunit.client.1.smithi192.stdout:2/880: unlink d4/d29/d60/d9a/l103 0 2024-04-24T22:45:43.458 INFO:tasks.workunit.client.1.smithi192.stdout:2/881: fsync d4/d9/d37/f91 0 2024-04-24T22:45:43.458 INFO:tasks.workunit.client.1.smithi192.stdout:1/639: truncate d9/d51/d70/f7f 1633518 0 2024-04-24T22:45:43.461 INFO:tasks.workunit.client.1.smithi192.stdout:9/834: unlink d3/dd/de/d17/d89/d8b/l93 0 2024-04-24T22:45:43.463 INFO:tasks.workunit.client.0.smithi063.stdout:0/861: write d6/d64/d21/d25/f7b [4297844,113584] 0 2024-04-24T22:45:43.464 INFO:tasks.workunit.client.0.smithi063.stdout:7/847: link d0/d4/de/d17/d35/de4/l114 d0/d4/de/d17/d35/de4/l11c 0 2024-04-24T22:45:43.472 INFO:tasks.workunit.client.1.smithi192.stdout:2/882: dread d4/d9/f8b [0,4194304] 0 2024-04-24T22:45:43.474 INFO:tasks.workunit.client.1.smithi192.stdout:9/835: rename d3/dd/de/dba/dbc to d3/dd/de/da7/d116 0 2024-04-24T22:45:43.476 INFO:tasks.workunit.client.1.smithi192.stdout:9/836: write d3/dd/de/d17/d89/de1/fe7 [2844946,65690] 0 2024-04-24T22:45:43.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/870: link d2/db/cbc d2/db/d25/c10f 0 2024-04-24T22:45:43.480 INFO:tasks.workunit.client.0.smithi063.stdout:5/866: dread d5/d11/d23/d2d/d62/d73/d118/f86 [4194304,4194304] 0 2024-04-24T22:45:43.483 INFO:tasks.workunit.client.0.smithi063.stdout:0/862: truncate d6/d3a/f7a 402721 0 2024-04-24T22:45:43.485 INFO:tasks.workunit.client.0.smithi063.stdout:7/848: unlink d0/d4/de/d17/d28/l4d 0 2024-04-24T22:45:43.487 INFO:tasks.workunit.client.1.smithi192.stdout:9/837: chown d3/dd/f6c 10969735 1 2024-04-24T22:45:43.491 INFO:tasks.workunit.client.0.smithi063.stdout:9/946: getdents d6/d2f/dcd/d53 0 2024-04-24T22:45:43.494 INFO:tasks.workunit.client.1.smithi192.stdout:1/640: dwrite d9/d15/d5e/d67/f71 [0,4194304] 0 2024-04-24T22:45:43.494 INFO:tasks.workunit.client.0.smithi063.stdout:5/867: rename d5/d11/d76/fbc to d5/de5/f12b 0 2024-04-24T22:45:43.498 INFO:tasks.workunit.client.0.smithi063.stdout:1/975: sync 2024-04-24T22:45:43.500 INFO:tasks.workunit.client.1.smithi192.stdout:2/883: write d4/d9/d10/fcf [180850,12942] 0 2024-04-24T22:45:43.501 INFO:tasks.workunit.client.1.smithi192.stdout:2/884: stat d4/d9/d37/f10b 0 2024-04-24T22:45:43.502 INFO:tasks.workunit.client.0.smithi063.stdout:7/849: dread - d0/d4/de/d17/d35/d69/da0/fe6 zero size 2024-04-24T22:45:43.502 INFO:tasks.workunit.client.0.smithi063.stdout:5/868: dwrite d5/d11/d23/d2d/d62/d73/f75 [0,4194304] 0 2024-04-24T22:45:43.502 INFO:tasks.workunit.client.1.smithi192.stdout:2/885: chown d4/f41 52 1 2024-04-24T22:45:43.506 INFO:tasks.workunit.client.0.smithi063.stdout:7/850: read - d0/d4/de/d17/d28/f112 zero size 2024-04-24T22:45:43.506 INFO:tasks.workunit.client.1.smithi192.stdout:1/641: truncate d9/d6f/f9b 669332 0 2024-04-24T22:45:43.507 INFO:tasks.workunit.client.0.smithi063.stdout:9/947: dread d6/d2f/dcd/d53/f7a [0,4194304] 0 2024-04-24T22:45:43.508 INFO:tasks.workunit.client.1.smithi192.stdout:9/838: rename d3/dd/d53/d60/le5 to d3/dd/de/d17/d89/de1/l117 0 2024-04-24T22:45:43.508 INFO:tasks.workunit.client.0.smithi063.stdout:4/892: write d2/f97 [1788681,28071] 0 2024-04-24T22:45:43.510 INFO:tasks.workunit.client.1.smithi192.stdout:2/886: creat d4/d9/d37/f11f x:0 0 0 2024-04-24T22:45:43.511 INFO:tasks.workunit.client.0.smithi063.stdout:4/893: dread - d2/d19/d33/d9d/da8/dec/ffc zero size 2024-04-24T22:45:43.512 INFO:tasks.workunit.client.0.smithi063.stdout:1/976: fdatasync d3/db5/fb2 0 2024-04-24T22:45:43.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/642: mkdir d9/d15/d2d/d48/d93/dc5/dd6 0 2024-04-24T22:45:43.515 INFO:tasks.workunit.client.0.smithi063.stdout:4/894: readlink d2/d8/d5c/l11d 0 2024-04-24T22:45:43.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/925: dwrite da/d18/d9e/f10a [4194304,4194304] 0 2024-04-24T22:45:43.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/926: stat da/f1d 0 2024-04-24T22:45:43.527 INFO:tasks.workunit.client.0.smithi063.stdout:1/977: creat d3/db5/d21/d9f/dd5/d106/d39/d115/f13c x:0 0 0 2024-04-24T22:45:43.529 INFO:tasks.workunit.client.0.smithi063.stdout:0/863: link d6/d3a/c8d d6/d64/d24/c122 0 2024-04-24T22:45:43.529 INFO:tasks.workunit.client.0.smithi063.stdout:1/978: dread - d3/db5/d21/d9f/dd5/d106/d39/d115/f13c zero size 2024-04-24T22:45:43.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/869: write d5/d11/d23/d2d/d62/f8b [318498,118102] 0 2024-04-24T22:45:43.533 INFO:tasks.workunit.client.0.smithi063.stdout:9/948: link d6/d2f/dcd/fe8 d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/d9d/d11e/d12a/f141 0 2024-04-24T22:45:43.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/851: rename d0/d4/de/d17/l1f to d0/l11d 0 2024-04-24T22:45:43.537 INFO:tasks.workunit.client.0.smithi063.stdout:7/852: write d0/d4/de/d17/d28/fb8 [1336332,18191] 0 2024-04-24T22:45:43.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/949: creat d6/d2f/dcd/d1f/d20/de6/f142 x:0 0 0 2024-04-24T22:45:43.540 INFO:tasks.workunit.client.0.smithi063.stdout:3/927: dread da/d39/d94/db3/fd2 [0,4194304] 0 2024-04-24T22:45:43.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/928: readlink da/d18/d47/d4d/d62/l90 0 2024-04-24T22:45:43.541 INFO:tasks.workunit.client.0.smithi063.stdout:0/864: rename d6/d64/d21/d9c/f108 to d6/d64/d24/dd1/dd5/f123 0 2024-04-24T22:45:43.542 INFO:tasks.workunit.client.0.smithi063.stdout:4/895: write d2/d80/d8d/f89 [2784001,27353] 0 2024-04-24T22:45:43.542 INFO:tasks.workunit.client.0.smithi063.stdout:5/870: symlink d5/d121/l12c 0 2024-04-24T22:45:43.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/979: write d3/da/d34/f93 [2012242,11059] 0 2024-04-24T22:45:43.547 INFO:tasks.workunit.client.0.smithi063.stdout:9/950: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/db9/d112/f143 x:0 0 0 2024-04-24T22:45:43.547 INFO:tasks.workunit.client.0.smithi063.stdout:3/929: mkdir da/d18/d47/d7b/d141 0 2024-04-24T22:45:43.548 INFO:tasks.workunit.client.0.smithi063.stdout:9/951: dread - d6/d2f/dcd/db/d6b/d4f/dad/f105 zero size 2024-04-24T22:45:43.549 INFO:tasks.workunit.client.0.smithi063.stdout:0/865: rename d6/d64/d21/d25/d62/cca to d6/d64/da0/d98/d11b/c124 0 2024-04-24T22:45:43.549 INFO:tasks.workunit.client.0.smithi063.stdout:5/871: symlink d5/d11/d23/d2d/d62/d81/l12d 0 2024-04-24T22:45:43.551 INFO:tasks.workunit.client.0.smithi063.stdout:4/896: mkdir d2/d8/d3c/d12b 0 2024-04-24T22:45:43.552 INFO:tasks.workunit.client.0.smithi063.stdout:4/897: stat d2/d19/d33/dc1/d50/dc3 0 2024-04-24T22:45:43.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/930: creat da/d39/d94/db3/dfa/f142 x:0 0 0 2024-04-24T22:45:43.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/931: stat da/d39/d94/faf 0 2024-04-24T22:45:43.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/952: creat d6/d2f/dcd/db/d1c/d78/d9a/f144 x:0 0 0 2024-04-24T22:45:43.555 INFO:tasks.workunit.client.0.smithi063.stdout:9/953: chown d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/d9d 4 1 2024-04-24T22:45:43.555 INFO:tasks.workunit.client.0.smithi063.stdout:7/853: link d0/d4/de/d6f/lba d0/d4/de/d17/d35/l11e 0 2024-04-24T22:45:43.558 INFO:tasks.workunit.client.0.smithi063.stdout:5/872: mknod d5/d11/d23/d2d/d62/d81/da4/da6/c12e 0 2024-04-24T22:45:43.561 INFO:tasks.workunit.client.0.smithi063.stdout:3/932: creat da/d18/d47/d4d/d9b/dde/df9/f143 x:0 0 0 2024-04-24T22:45:43.562 INFO:tasks.workunit.client.1.smithi192.stdout:2/887: creat d4/d29/d60/d9a/db3/f120 x:0 0 0 2024-04-24T22:45:43.564 INFO:tasks.workunit.client.1.smithi192.stdout:7/871: dwrite d2/db/d25/f27 [4194304,4194304] 0 2024-04-24T22:45:43.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/954: fsync d6/d2f/dcd/d53/f88 0 2024-04-24T22:45:43.566 INFO:tasks.workunit.client.1.smithi192.stdout:2/888: write d4/d29/f26 [1500638,2986] 0 2024-04-24T22:45:43.567 INFO:tasks.workunit.client.0.smithi063.stdout:1/980: write d3/db5/fb2 [2440768,64725] 0 2024-04-24T22:45:43.568 INFO:tasks.workunit.client.0.smithi063.stdout:0/866: creat d6/d64/d21/d25/d62/d63/d89/d116/f125 x:0 0 0 2024-04-24T22:45:43.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/643: dwrite d9/d15/d2d/d48/d93/da4/dd2/fbd [0,4194304] 0 2024-04-24T22:45:43.571 INFO:tasks.workunit.client.0.smithi063.stdout:4/898: creat d2/d122/f12c x:0 0 0 2024-04-24T22:45:43.573 INFO:tasks.workunit.client.0.smithi063.stdout:8/978: sync 2024-04-24T22:45:43.573 INFO:tasks.workunit.client.1.smithi192.stdout:9/839: dwrite d3/dd/de/f19 [0,4194304] 0 2024-04-24T22:45:43.574 INFO:tasks.workunit.client.1.smithi192.stdout:1/644: write d9/d15/d5e/d7c/fd3 [339586,94601] 0 2024-04-24T22:45:43.574 INFO:tasks.workunit.client.0.smithi063.stdout:3/933: creat da/d18/d3a/da1/f144 x:0 0 0 2024-04-24T22:45:43.575 INFO:tasks.workunit.client.0.smithi063.stdout:3/934: readlink da/d18/d3a/da1/l112 0 2024-04-24T22:45:43.575 INFO:tasks.workunit.client.1.smithi192.stdout:1/645: fsync d9/d51/f54 0 2024-04-24T22:45:43.576 INFO:tasks.workunit.client.0.smithi063.stdout:3/935: chown da/d18/d47/d4d/c6b 55 1 2024-04-24T22:45:43.582 INFO:tasks.workunit.client.0.smithi063.stdout:1/981: mkdir d3/db5/d21/d9f/dd5/d106/d11/d5e/d6f/d13d 0 2024-04-24T22:45:43.582 INFO:tasks.workunit.client.0.smithi063.stdout:0/867: mkdir d6/d64/d24/d42/dad/d126 0 2024-04-24T22:45:43.583 INFO:tasks.workunit.client.0.smithi063.stdout:5/873: symlink d5/d11/d23/d2d/d62/d73/d118/d36/l12f 0 2024-04-24T22:45:43.584 INFO:tasks.workunit.client.1.smithi192.stdout:9/840: symlink d3/dd/d62/d6d/l118 0 2024-04-24T22:45:43.586 INFO:tasks.workunit.client.0.smithi063.stdout:4/899: rename d2/d8/d36/d118/f129 to d2/d19/dce/f12d 0 2024-04-24T22:45:43.587 INFO:tasks.workunit.client.0.smithi063.stdout:7/854: dread d0/d4/de/d17/d35/d69/db1/fc1 [0,4194304] 0 2024-04-24T22:45:43.589 INFO:tasks.workunit.client.1.smithi192.stdout:1/646: creat d9/d15/d2d/d48/d93/da4/dd2/dbf/fd7 x:0 0 0 2024-04-24T22:45:43.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/979: mkdir d1/d22/d33/da6/d54/d7b/d133 0 2024-04-24T22:45:43.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/980: chown d1/f111 1754536 1 2024-04-24T22:45:43.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/981: chown d1/d11/d32/d38/db1/f121 3480 1 2024-04-24T22:45:43.594 INFO:tasks.workunit.client.1.smithi192.stdout:1/647: write d9/d15/d2d/d4a/fd0 [2867842,29156] 0 2024-04-24T22:45:43.595 INFO:tasks.workunit.client.0.smithi063.stdout:0/868: mkdir d6/d3a/dba/d127 0 2024-04-24T22:45:43.596 INFO:tasks.workunit.client.1.smithi192.stdout:1/648: dread - d9/d15/d2d/d31/d2f/fc0 zero size 2024-04-24T22:45:43.597 INFO:tasks.workunit.client.0.smithi063.stdout:5/874: creat d5/d11/d23/d2d/d62/d73/d118/d44/f130 x:0 0 0 2024-04-24T22:45:43.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/875: write d5/d11/d23/d2d/d62/d73/d118/db6/ff6 [761003,2704] 0 2024-04-24T22:45:43.599 INFO:tasks.workunit.client.0.smithi063.stdout:7/855: rmdir d0/d4/de/d6f/d2e/d55/d106 39 2024-04-24T22:45:43.602 INFO:tasks.workunit.client.1.smithi192.stdout:1/649: symlink d9/d15/d2d/d63/ld8 0 2024-04-24T22:45:43.605 INFO:tasks.workunit.client.0.smithi063.stdout:4/900: creat d2/d8/d3c/d12b/f12e x:0 0 0 2024-04-24T22:45:43.606 INFO:tasks.workunit.client.0.smithi063.stdout:4/901: read d2/d19/d33/dc1/d50/f5d [4174610,40647] 0 2024-04-24T22:45:43.606 INFO:tasks.workunit.client.1.smithi192.stdout:7/872: dwrite d2/db/f10 [0,4194304] 0 2024-04-24T22:45:43.607 INFO:tasks.workunit.client.1.smithi192.stdout:1/650: creat d9/d15/d2d/d31/d2f/dae/fd9 x:0 0 0 2024-04-24T22:45:43.607 INFO:tasks.workunit.client.0.smithi063.stdout:8/982: symlink d1/d11/l134 0 2024-04-24T22:45:43.609 INFO:tasks.workunit.client.1.smithi192.stdout:2/889: dread d4/f78 [0,4194304] 0 2024-04-24T22:45:43.609 INFO:tasks.workunit.client.0.smithi063.stdout:5/876: unlink d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f45 0 2024-04-24T22:45:43.610 INFO:tasks.workunit.client.0.smithi063.stdout:7/856: rmdir d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1 39 2024-04-24T22:45:43.613 INFO:tasks.workunit.client.1.smithi192.stdout:7/873: mknod d2/db/d25/dda/c110 0 2024-04-24T22:45:43.613 INFO:tasks.workunit.client.1.smithi192.stdout:9/841: dread d3/dd/d53/f7f [0,4194304] 0 2024-04-24T22:45:43.614 INFO:tasks.workunit.client.0.smithi063.stdout:1/982: creat d3/db5/d21/d9f/dd5/d106/d11/f13e x:0 0 0 2024-04-24T22:45:43.615 INFO:tasks.workunit.client.1.smithi192.stdout:7/874: write d2/db/d21/d42/d67/da8/dc7/fbd [2956288,115959] 0 2024-04-24T22:45:43.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/955: write d6/d2f/d42/d81/f95 [977948,123430] 0 2024-04-24T22:45:43.620 INFO:tasks.workunit.client.1.smithi192.stdout:9/842: unlink d3/dd/de/d17/l72 0 2024-04-24T22:45:43.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/956: stat d6/d2f/dcd/d53/dff/d132 0 2024-04-24T22:45:43.621 INFO:tasks.workunit.client.0.smithi063.stdout:4/902: truncate d2/d80/f8a 3052264 0 2024-04-24T22:45:43.622 INFO:tasks.workunit.client.0.smithi063.stdout:1/983: dwrite d3/db5/d21/d9f/dd5/d106/df5/f103 [0,4194304] 0 2024-04-24T22:45:43.625 INFO:tasks.workunit.client.0.smithi063.stdout:7/857: rmdir d0/d4/de/d17/d28/d31/d94 39 2024-04-24T22:45:43.627 INFO:tasks.workunit.client.1.smithi192.stdout:9/843: rmdir d3/dd/de/da7/d116/ddb 39 2024-04-24T22:45:43.627 INFO:tasks.workunit.client.1.smithi192.stdout:9/844: chown d3/dd/d53/dab/dcf 21063388 1 2024-04-24T22:45:43.630 INFO:tasks.workunit.client.1.smithi192.stdout:7/875: rename d2/db/d36/f73 to d2/db/d21/d42/d67/d9b/d6e/f111 0 2024-04-24T22:45:43.633 INFO:tasks.workunit.client.1.smithi192.stdout:4/747: sync 2024-04-24T22:45:43.634 INFO:tasks.workunit.client.1.smithi192.stdout:4/748: chown d2/db/d75/da1/d36/d5b/d28/d2f 1654 1 2024-04-24T22:45:43.639 INFO:tasks.workunit.client.0.smithi063.stdout:1/984: mkdir d3/db5/d21/d9f/dd5/d106/d39/d115/d13f 0 2024-04-24T22:45:43.640 INFO:tasks.workunit.client.1.smithi192.stdout:4/749: read d2/db/f52 [157710,92737] 0 2024-04-24T22:45:43.640 INFO:tasks.workunit.client.0.smithi063.stdout:3/936: write da/d18/d3a/f104 [1919993,113211] 0 2024-04-24T22:45:43.642 INFO:tasks.workunit.client.0.smithi063.stdout:5/877: truncate d5/d11/d23/d2d/d62/d73/d118/d44/f117 623703 0 2024-04-24T22:45:43.643 INFO:tasks.workunit.client.1.smithi192.stdout:4/750: mkdir d2/d5c/d107 0 2024-04-24T22:45:43.647 INFO:tasks.workunit.client.1.smithi192.stdout:7/876: creat d2/db/d21/d42/f112 x:0 0 0 2024-04-24T22:45:43.647 INFO:tasks.workunit.client.0.smithi063.stdout:9/957: dread d6/d2f/d42/f6f [0,4194304] 0 2024-04-24T22:45:43.650 INFO:tasks.workunit.client.1.smithi192.stdout:1/651: write d9/d15/d2d/d63/d68/f76 [49968,125368] 0 2024-04-24T22:45:43.653 INFO:tasks.workunit.client.0.smithi063.stdout:3/937: truncate da/f29 510416 0 2024-04-24T22:45:43.654 INFO:tasks.workunit.client.0.smithi063.stdout:3/938: chown da/d39/d94/de4/c136 31 1 2024-04-24T22:45:43.657 INFO:tasks.workunit.client.0.smithi063.stdout:3/939: chown da/d18/d47/d4d/l99 110732589 1 2024-04-24T22:45:43.657 INFO:tasks.workunit.client.0.smithi063.stdout:0/869: dwrite d6/da/d2b/f40 [0,4194304] 0 2024-04-24T22:45:43.657 INFO:tasks.workunit.client.1.smithi192.stdout:2/890: dwrite d4/d9/d37/f94 [0,4194304] 0 2024-04-24T22:45:43.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/751: rename d2/db/d75/da1/d7d/dbd to d2/db/d5d/d108 0 2024-04-24T22:45:43.664 INFO:tasks.workunit.client.0.smithi063.stdout:9/958: dread - d6/d2f/dcd/d53/dff/d132/de0/ffd zero size 2024-04-24T22:45:43.665 INFO:tasks.workunit.client.1.smithi192.stdout:2/891: creat d4/d9/d10/d6c/d95/db7/dbf/f121 x:0 0 0 2024-04-24T22:45:43.667 INFO:tasks.workunit.client.1.smithi192.stdout:9/845: write d3/d5d/fd7 [1182457,129482] 0 2024-04-24T22:45:43.667 INFO:tasks.workunit.client.0.smithi063.stdout:1/985: dread d3/db5/d21/d9f/dd5/d106/d11/d5e/d6f/fbb [0,4194304] 0 2024-04-24T22:45:43.668 INFO:tasks.workunit.client.0.smithi063.stdout:8/983: dwrite d1/f5 [0,4194304] 0 2024-04-24T22:45:43.668 INFO:tasks.workunit.client.1.smithi192.stdout:2/892: rmdir d4/d9/d10/d6c/d8c/da5 39 2024-04-24T22:45:43.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/984: stat d1/d2/d3/d86/l129 0 2024-04-24T22:45:43.670 INFO:tasks.workunit.client.0.smithi063.stdout:1/986: chown d3/db5/d21/d9f/dd5/d106/df5/f11d 0 1 2024-04-24T22:45:43.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/846: symlink d3/dd/de/da7/dae/l119 0 2024-04-24T22:45:43.672 INFO:tasks.workunit.client.1.smithi192.stdout:9/847: write d3/dd/de/d17/f8f [1366521,71134] 0 2024-04-24T22:45:43.672 INFO:tasks.workunit.client.0.smithi063.stdout:9/959: chown d6/d2f/dcd/d53/df6/lbf 1 1 2024-04-24T22:45:43.674 INFO:tasks.workunit.client.0.smithi063.stdout:9/960: chown d6/d2f/dcd/db/c1d 12 1 2024-04-24T22:45:43.674 INFO:tasks.workunit.client.1.smithi192.stdout:9/848: write d3/dd/de/f19 [2694583,30353] 0 2024-04-24T22:45:43.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/903: dwrite d2/d80/f8f [4194304,4194304] 0 2024-04-24T22:45:43.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/849: dread d3/f58 [0,4194304] 0 2024-04-24T22:45:43.679 INFO:tasks.workunit.client.0.smithi063.stdout:1/987: unlink d3/de0/d79/d96/ld6 0 2024-04-24T22:45:43.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/961: dwrite d6/d2f/dcd/d53/dff/d132/fdc [0,4194304] 0 2024-04-24T22:45:43.681 INFO:tasks.workunit.client.0.smithi063.stdout:1/988: chown d3/db5/d21/d9f/dd5/d106/f3a 471 1 2024-04-24T22:45:43.683 INFO:tasks.workunit.client.0.smithi063.stdout:0/870: link d6/d64/d21/d25/f55 d6/da/d2b/d60/f128 0 2024-04-24T22:45:43.684 INFO:tasks.workunit.client.0.smithi063.stdout:0/871: stat d6/da/d2b/l77 0 2024-04-24T22:45:43.684 INFO:tasks.workunit.client.1.smithi192.stdout:9/850: mknod d3/d59/d7d/d9b/c11a 0 2024-04-24T22:45:43.689 INFO:tasks.workunit.client.0.smithi063.stdout:0/872: creat d6/d64/d21/d80/f129 x:0 0 0 2024-04-24T22:45:43.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/752: dread d2/db/d75/da1/d36/d5b/f24 [0,4194304] 0 2024-04-24T22:45:43.695 INFO:tasks.workunit.client.0.smithi063.stdout:0/873: dwrite d6/d74/fab [0,4194304] 0 2024-04-24T22:45:43.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/753: dwrite d2/db/d10/f11 [4194304,4194304] 0 2024-04-24T22:45:43.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/686: sync 2024-04-24T22:45:43.715 INFO:tasks.workunit.client.1.smithi192.stdout:8/758: sync 2024-04-24T22:45:43.715 INFO:tasks.workunit.client.1.smithi192.stdout:3/777: sync 2024-04-24T22:45:43.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/648: sync 2024-04-24T22:45:43.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/770: sync 2024-04-24T22:45:43.716 INFO:tasks.workunit.client.1.smithi192.stdout:5/649: fsync d0/d13/d19/da9/fba 0 2024-04-24T22:45:43.721 INFO:tasks.workunit.client.1.smithi192.stdout:5/650: unlink d0/c4e 0 2024-04-24T22:45:43.721 INFO:tasks.workunit.client.1.smithi192.stdout:3/778: truncate d2/d27/dc6/d4f/f7a 366942 0 2024-04-24T22:45:43.722 INFO:tasks.workunit.client.1.smithi192.stdout:3/779: fsync d2/d27/d28/f36 0 2024-04-24T22:45:43.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/687: mkdir d0/d8/d18/df1/df2 0 2024-04-24T22:45:43.723 INFO:tasks.workunit.client.0.smithi063.stdout:9/962: dread d6/d2f/dcd/db/d93/da0/fdf [0,4194304] 0 2024-04-24T22:45:43.724 INFO:tasks.workunit.client.1.smithi192.stdout:0/771: creat d3/d27/d28/d31/dbc/dea/f101 x:0 0 0 2024-04-24T22:45:43.726 INFO:tasks.workunit.client.1.smithi192.stdout:0/772: fdatasync d3/d27/d28/d31/f85 0 2024-04-24T22:45:43.726 INFO:tasks.workunit.client.0.smithi063.stdout:9/963: mknod d6/d2f/dcd/db/d6b/d4f/dad/d36/c145 0 2024-04-24T22:45:43.728 INFO:tasks.workunit.client.0.smithi063.stdout:9/964: stat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/da1/d12e/fe1 0 2024-04-24T22:45:43.729 INFO:tasks.workunit.client.1.smithi192.stdout:3/780: fsync d2/d4/d12/d39/fd0 0 2024-04-24T22:45:43.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/651: dwrite d0/d13/d19/f4b [4194304,4194304] 0 2024-04-24T22:45:43.734 INFO:tasks.workunit.client.0.smithi063.stdout:9/965: truncate d6/d2f/dcd/db/d6b/d4f/dad/d36/fe4 310206 0 2024-04-24T22:45:43.736 INFO:tasks.workunit.client.1.smithi192.stdout:6/688: mkdir d0/d8/d3d/d3e/df3 0 2024-04-24T22:45:43.742 INFO:tasks.workunit.client.0.smithi063.stdout:9/966: readlink d6/l4a 0 2024-04-24T22:45:43.743 INFO:tasks.workunit.client.1.smithi192.stdout:3/781: rename d2/d4/d12/d1a/d1d/l40 to d2/d4/d12/dbc/db0/lfb 0 2024-04-24T22:45:43.743 INFO:tasks.workunit.client.1.smithi192.stdout:5/652: dwrite d0/d13/d19/d64/d5f/d2c/f5c [0,4194304] 0 2024-04-24T22:45:43.745 INFO:tasks.workunit.client.0.smithi063.stdout:3/940: sync 2024-04-24T22:45:43.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/941: readlink da/d18/d47/l75 0 2024-04-24T22:45:43.748 INFO:tasks.workunit.client.0.smithi063.stdout:5/878: rename d5/d11/d23/d2d/d62/d73/d118/d44 to d5/d11/d23/d2d/d62/d73/d118/d36/d131 0 2024-04-24T22:45:43.750 INFO:tasks.workunit.client.1.smithi192.stdout:5/653: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:43.754 INFO:tasks.workunit.client.1.smithi192.stdout:7/877: write d2/db/d21/d42/fb0 [1099222,48179] 0 2024-04-24T22:45:43.755 INFO:tasks.workunit.client.1.smithi192.stdout:3/782: chown d2/d27/d28/d38/ldd 1025 1 2024-04-24T22:45:43.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/878: write d2/db/f45 [122543,21772] 0 2024-04-24T22:45:43.759 INFO:tasks.workunit.client.0.smithi063.stdout:3/942: mkdir da/d18/d3a/d63/d145 0 2024-04-24T22:45:43.761 INFO:tasks.workunit.client.1.smithi192.stdout:6/689: symlink d0/d8/d3d/d9d/dd6/ddc/lf4 0 2024-04-24T22:45:43.761 INFO:tasks.workunit.client.0.smithi063.stdout:7/858: write d0/d4/de/d6f/d5c/d8b/dc0/fd0 [50578,88983] 0 2024-04-24T22:45:43.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/859: chown d0/d4/de/d17/d35/d6c/d7a/fa5 1 1 2024-04-24T22:45:43.763 INFO:tasks.workunit.client.1.smithi192.stdout:6/690: fsync d0/d8/d3d/d3e/d7e/f8e 0 2024-04-24T22:45:43.764 INFO:tasks.workunit.client.0.smithi063.stdout:7/860: chown d0/d4/de/d17/d35/d69/cd4 235600 1 2024-04-24T22:45:43.765 INFO:tasks.workunit.client.0.smithi063.stdout:1/989: sync 2024-04-24T22:45:43.766 INFO:tasks.workunit.client.1.smithi192.stdout:5/654: symlink d0/d13/d19/d71/d88/le2 0 2024-04-24T22:45:43.769 INFO:tasks.workunit.client.1.smithi192.stdout:2/893: truncate d4/d9/d37/f3b 3749962 0 2024-04-24T22:45:43.774 INFO:tasks.workunit.client.1.smithi192.stdout:5/655: dwrite d0/fc5 [0,4194304] 0 2024-04-24T22:45:43.775 INFO:tasks.workunit.client.0.smithi063.stdout:5/879: mknod d5/d1d/c132 0 2024-04-24T22:45:43.777 INFO:tasks.workunit.client.1.smithi192.stdout:4/754: dread d2/db/d75/da1/d36/d5b/f47 [0,4194304] 0 2024-04-24T22:45:43.781 INFO:tasks.workunit.client.1.smithi192.stdout:0/773: getdents d3/d27/d28/dcc/d5c/d80/d94 0 2024-04-24T22:45:43.782 INFO:tasks.workunit.client.0.smithi063.stdout:4/904: write d2/ff0 [397859,39669] 0 2024-04-24T22:45:43.782 INFO:tasks.workunit.client.1.smithi192.stdout:0/774: chown d3/d27/d3c/d3b/f8d 958482 1 2024-04-24T22:45:43.783 INFO:tasks.workunit.client.0.smithi063.stdout:4/905: chown d2/d19/dce 16 1 2024-04-24T22:45:43.784 INFO:tasks.workunit.client.0.smithi063.stdout:9/967: dread d6/d2f/d42/d81/f96 [0,4194304] 0 2024-04-24T22:45:43.784 INFO:tasks.workunit.client.0.smithi063.stdout:8/985: rename d1/d2/d17/lef to d1/d22/l135 0 2024-04-24T22:45:43.786 INFO:tasks.workunit.client.1.smithi192.stdout:9/851: write d3/d5d/fdc [3907772,114089] 0 2024-04-24T22:45:43.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/880: rmdir d5/d11/d23/d2d 39 2024-04-24T22:45:43.787 INFO:tasks.workunit.client.1.smithi192.stdout:9/852: chown d3/dd/de/d17/d89/db4/cda 429463 1 2024-04-24T22:45:43.790 INFO:tasks.workunit.client.0.smithi063.stdout:7/861: creat d0/d4/de/d17/d28/d31/d94/f11f x:0 0 0 2024-04-24T22:45:43.793 INFO:tasks.workunit.client.1.smithi192.stdout:6/691: dread d0/d8/d29/f30 [0,4194304] 0 2024-04-24T22:45:43.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/990: mknod d3/db5/d21/d9f/dd5/d106/d39/d115/d13f/c140 0 2024-04-24T22:45:43.795 INFO:tasks.workunit.client.0.smithi063.stdout:0/874: dwrite d6/d64/d24/fbc [0,4194304] 0 2024-04-24T22:45:43.796 INFO:tasks.workunit.client.1.smithi192.stdout:2/894: creat d4/d9/d9f/f122 x:0 0 0 2024-04-24T22:45:43.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/991: write d3/db5/d21/d9f/dd5/d106/d11/f5f [1024323,115061] 0 2024-04-24T22:45:43.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/895: chown d4/d9/d10/f1e 26307 1 2024-04-24T22:45:43.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/759: write db/dc/d22/d31/d33/db7/fc7 [312453,78852] 0 2024-04-24T22:45:43.801 INFO:tasks.workunit.client.0.smithi063.stdout:3/943: rename da/d18/d3a/f3f to da/d18/d47/d4d/dc2/d120/f146 0 2024-04-24T22:45:43.802 INFO:tasks.workunit.client.1.smithi192.stdout:7/879: mkdir d2/d113 0 2024-04-24T22:45:43.802 INFO:tasks.workunit.client.0.smithi063.stdout:4/906: dread d2/f26 [4194304,4194304] 0 2024-04-24T22:45:43.809 INFO:tasks.workunit.client.0.smithi063.stdout:8/986: dread d1/d22/d33/fee [0,4194304] 0 2024-04-24T22:45:43.811 INFO:tasks.workunit.client.0.smithi063.stdout:7/862: readlink d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/l10f 0 2024-04-24T22:45:43.813 INFO:tasks.workunit.client.0.smithi063.stdout:5/881: dread d5/d11/d23/d2d/d62/d73/f75 [0,4194304] 0 2024-04-24T22:45:43.816 INFO:tasks.workunit.client.0.smithi063.stdout:8/987: dwrite d1/d22/d33/da6/d54/d79/fda [0,4194304] 0 2024-04-24T22:45:43.819 INFO:tasks.workunit.client.1.smithi192.stdout:7/880: mkdir d2/db/d21/d42/d67/d9b/d6e/d77/d114 0 2024-04-24T22:45:43.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/907: dread d2/d19/d82/d96/fa9 [0,4194304] 0 2024-04-24T22:45:43.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/692: creat d0/d8/d29/d5a/ddf/ff5 x:0 0 0 2024-04-24T22:45:43.828 INFO:tasks.workunit.client.1.smithi192.stdout:2/896: creat d4/d9/d10/d102/f123 x:0 0 0 2024-04-24T22:45:43.829 INFO:tasks.workunit.client.1.smithi192.stdout:7/881: dwrite d2/db/d21/d42/f112 [0,4194304] 0 2024-04-24T22:45:43.830 INFO:tasks.workunit.client.0.smithi063.stdout:9/968: mknod d6/d2f/dcd/db/dca/d127/c146 0 2024-04-24T22:45:43.834 INFO:tasks.workunit.client.1.smithi192.stdout:6/693: write d0/fc2 [758598,14142] 0 2024-04-24T22:45:43.836 INFO:tasks.workunit.client.1.smithi192.stdout:3/783: dwrite d2/fa2 [0,4194304] 0 2024-04-24T22:45:43.855 INFO:tasks.workunit.client.1.smithi192.stdout:3/784: unlink d2/d4/d12/d1a/le9 0 2024-04-24T22:45:43.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/992: mkdir d3/db5/d21/d9f/dd5/d106/d11/d141 0 2024-04-24T22:45:43.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/993: fsync d3/db5/d21/f91 0 2024-04-24T22:45:43.856 INFO:tasks.workunit.client.0.smithi063.stdout:1/994: chown d3/db5/d21/d9f/dd5/cee 18074695 1 2024-04-24T22:45:43.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/882: dwrite d2/db/d21/ff8 [0,4194304] 0 2024-04-24T22:45:43.858 INFO:tasks.workunit.client.1.smithi192.stdout:0/775: dwrite d3/d27/d28/d2e/d3f/fc8 [0,4194304] 0 2024-04-24T22:45:43.859 INFO:tasks.workunit.client.0.smithi063.stdout:8/988: unlink d1/d22/d33/fee 0 2024-04-24T22:45:43.860 INFO:tasks.workunit.client.1.smithi192.stdout:9/853: dwrite d3/dd/f3a [4194304,4194304] 0 2024-04-24T22:45:43.860 INFO:tasks.workunit.client.1.smithi192.stdout:7/883: stat d2/db/d21/d42/d67 0 2024-04-24T22:45:43.864 INFO:tasks.workunit.client.0.smithi063.stdout:3/944: mkdir da/d18/d3a/d147 0 2024-04-24T22:45:43.865 INFO:tasks.workunit.client.1.smithi192.stdout:8/760: link db/dc/c25 db/dc/d22/d31/d33/cee 0 2024-04-24T22:45:43.867 INFO:tasks.workunit.client.0.smithi063.stdout:4/908: mkdir d2/d8/dd4/dda/dc6/d12f 0 2024-04-24T22:45:43.873 INFO:tasks.workunit.client.0.smithi063.stdout:9/969: fsync d6/f30 0 2024-04-24T22:45:43.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/909: read d2/d19/f37 [378805,126880] 0 2024-04-24T22:45:43.873 INFO:tasks.workunit.client.1.smithi192.stdout:7/884: mkdir d2/db/d49/d115 0 2024-04-24T22:45:43.877 INFO:tasks.workunit.client.0.smithi063.stdout:7/863: dwrite d0/d4/f3a [0,4194304] 0 2024-04-24T22:45:43.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/776: rename d3/d6d/ccb to d3/d6d/d79/c102 0 2024-04-24T22:45:43.878 INFO:tasks.workunit.client.1.smithi192.stdout:0/777: readlink d3/d27/d28/d2e/d3f/lb2 0 2024-04-24T22:45:43.882 INFO:tasks.workunit.client.1.smithi192.stdout:0/778: chown d3/d27/d28/d2e/dc7/fd0 142631 1 2024-04-24T22:45:43.882 INFO:tasks.workunit.client.0.smithi063.stdout:5/882: dwrite d5/d11/d23/d2d/d62/fad [0,4194304] 0 2024-04-24T22:45:43.883 INFO:tasks.workunit.client.1.smithi192.stdout:7/885: dwrite d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 [0,4194304] 0 2024-04-24T22:45:43.884 INFO:tasks.workunit.client.1.smithi192.stdout:2/897: read d4/d9/d37/f3b [647588,57381] 0 2024-04-24T22:45:43.884 INFO:tasks.workunit.client.0.smithi063.stdout:8/989: mknod d1/d21/de6/c136 0 2024-04-24T22:45:43.884 INFO:tasks.workunit.client.0.smithi063.stdout:5/883: chown d5/d11/d23/d2d/d62/d81/da4/dee 31533214 1 2024-04-24T22:45:43.885 INFO:tasks.workunit.client.0.smithi063.stdout:0/875: sync 2024-04-24T22:45:43.886 INFO:tasks.workunit.client.1.smithi192.stdout:2/898: write d4/fe1 [556911,91275] 0 2024-04-24T22:45:43.887 INFO:tasks.workunit.client.1.smithi192.stdout:2/899: chown d4/d9/d10/d6c/d95/dc0 0 1 2024-04-24T22:45:43.889 INFO:tasks.workunit.client.1.smithi192.stdout:0/779: mknod d3/d27/d3c/d3b/c103 0 2024-04-24T22:45:43.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/864: mkdir d0/d4/de/d17/d35/d69/dc9/d120 0 2024-04-24T22:45:43.896 INFO:tasks.workunit.client.0.smithi063.stdout:9/970: dread d6/d2f/dcd/db/d6b/f5c [0,4194304] 0 2024-04-24T22:45:43.897 INFO:tasks.workunit.client.1.smithi192.stdout:8/761: getdents db/d43/d76 0 2024-04-24T22:45:43.898 INFO:tasks.workunit.client.1.smithi192.stdout:8/762: stat db/d5d/d74/c93 0 2024-04-24T22:45:43.899 INFO:tasks.workunit.client.0.smithi063.stdout:3/945: symlink da/d18/d3a/d147/l148 0 2024-04-24T22:45:43.899 INFO:tasks.workunit.client.1.smithi192.stdout:7/886: getdents d2/db/d21/d42/d67/dae/dce 0 2024-04-24T22:45:43.902 INFO:tasks.workunit.client.1.smithi192.stdout:7/887: fsync d2/db/d21/d42/d67/d9b/d6e/db5/db6/fff 0 2024-04-24T22:45:43.903 INFO:tasks.workunit.client.0.smithi063.stdout:0/876: creat d6/dd/f12a x:0 0 0 2024-04-24T22:45:43.905 INFO:tasks.workunit.client.1.smithi192.stdout:9/854: dread d3/dd/de/d17/f2b [0,4194304] 0 2024-04-24T22:45:43.906 INFO:tasks.workunit.client.0.smithi063.stdout:7/865: dread - d0/d4/de/f60 zero size 2024-04-24T22:45:43.907 INFO:tasks.workunit.client.1.smithi192.stdout:2/900: link d4/ca d4/d9/d10/d6c/d8c/c124 0 2024-04-24T22:45:43.907 INFO:tasks.workunit.client.0.smithi063.stdout:7/866: fdatasync d0/d4/de/d17/d28/fb8 0 2024-04-24T22:45:43.909 INFO:tasks.workunit.client.0.smithi063.stdout:7/867: readlink d0/d4/de/d17/d28/d31/d99/le9 0 2024-04-24T22:45:43.910 INFO:tasks.workunit.client.0.smithi063.stdout:3/946: rename da/d18/d47/l75 to da/d39/d94/de4/l149 0 2024-04-24T22:45:43.911 INFO:tasks.workunit.client.1.smithi192.stdout:7/888: dread d2/db/d49/f6c [0,4194304] 0 2024-04-24T22:45:43.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/694: write d0/d8/d18/d2e/f51 [2759499,18396] 0 2024-04-24T22:45:43.913 INFO:tasks.workunit.client.1.smithi192.stdout:8/763: dwrite db/dc/d22/d31/d33/db7/fc7 [0,4194304] 0 2024-04-24T22:45:43.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/780: getdents d3/d27 0 2024-04-24T22:45:43.915 INFO:tasks.workunit.client.1.smithi192.stdout:0/781: chown d3/d27/d28/d31/dbc/fe4 46869 1 2024-04-24T22:45:43.918 INFO:tasks.workunit.client.1.smithi192.stdout:2/901: creat d4/f125 x:0 0 0 2024-04-24T22:45:43.918 INFO:tasks.workunit.client.1.smithi192.stdout:7/889: truncate d2/db/d21/d42/d67/d9b/d6e/db5/fdf 696701 0 2024-04-24T22:45:43.919 INFO:tasks.workunit.client.1.smithi192.stdout:2/902: fdatasync d4/f31 0 2024-04-24T22:45:43.921 INFO:tasks.workunit.client.1.smithi192.stdout:8/764: dwrite f6 [4194304,4194304] 0 2024-04-24T22:45:43.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/785: write d2/d27/dc6/d4f/f5a [453143,30373] 0 2024-04-24T22:45:43.927 INFO:tasks.workunit.client.0.smithi063.stdout:7/868: creat d0/d4/de/d6f/d5d/f121 x:0 0 0 2024-04-24T22:45:43.928 INFO:tasks.workunit.client.0.smithi063.stdout:7/869: chown d0/d4/de/d17/d28/d31/d99/c105 53214281 1 2024-04-24T22:45:43.931 INFO:tasks.workunit.client.1.smithi192.stdout:6/695: symlink d0/d8/d3d/d9d/dd6/ddc/lf6 0 2024-04-24T22:45:43.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/782: symlink d3/d27/d3c/d64/l104 0 2024-04-24T22:45:43.932 INFO:tasks.workunit.client.1.smithi192.stdout:3/786: stat d2/d27/dc6/f9 0 2024-04-24T22:45:43.933 INFO:tasks.workunit.client.0.smithi063.stdout:0/877: mknod d6/d64/d24/d42/dad/d126/c12b 0 2024-04-24T22:45:43.934 INFO:tasks.workunit.client.0.smithi063.stdout:1/995: write d3/de0/d79/d96/f9d [956850,71663] 0 2024-04-24T22:45:43.935 INFO:tasks.workunit.client.0.smithi063.stdout:8/990: dread d1/d2/d3/db6/ffc [0,4194304] 0 2024-04-24T22:45:43.936 INFO:tasks.workunit.client.0.smithi063.stdout:5/884: getdents d5/d11 0 2024-04-24T22:45:43.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/890: mknod d2/db/d21/d42/d67/dae/c116 0 2024-04-24T22:45:43.939 INFO:tasks.workunit.client.0.smithi063.stdout:4/910: write d2/d8/f31 [1116733,48048] 0 2024-04-24T22:45:43.939 INFO:tasks.workunit.client.1.smithi192.stdout:6/696: dread d0/d8/d3d/d3e/d40/d44/f93 [0,4194304] 0 2024-04-24T22:45:43.941 INFO:tasks.workunit.client.0.smithi063.stdout:1/996: dwrite d3/de0/d79/d96/f9d [0,4194304] 0 2024-04-24T22:45:43.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/878: rename d6/d64/da0/d98/dd6/fe7 to d6/d64/d21/d25/d62/d63/d7c/f12c 0 2024-04-24T22:45:43.954 INFO:tasks.workunit.client.0.smithi063.stdout:7/870: unlink d0/d4/de/d17/d28/d31/cf8 0 2024-04-24T22:45:43.956 INFO:tasks.workunit.client.0.smithi063.stdout:8/991: truncate d1/d2/ddf/f101 873894 0 2024-04-24T22:45:43.960 INFO:tasks.workunit.client.1.smithi192.stdout:2/903: dread d4/d9/d37/f40 [0,4194304] 0 2024-04-24T22:45:43.960 INFO:tasks.workunit.client.0.smithi063.stdout:5/885: symlink d5/l133 0 2024-04-24T22:45:43.962 INFO:tasks.workunit.client.0.smithi063.stdout:5/886: truncate d5/d11/d23/d2d/d62/d73/d118/d36/f122 801840 0 2024-04-24T22:45:43.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/911: rename d2/d19/d33/dc1/d50/dfa/c123 to d2/d8/d40/c130 0 2024-04-24T22:45:43.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/947: truncate da/d18/d47/d4d/d9b/fe1 82059 0 2024-04-24T22:45:43.971 INFO:tasks.workunit.client.0.smithi063.stdout:8/992: creat d1/d11/d32/d6a/f137 x:0 0 0 2024-04-24T22:45:43.971 INFO:tasks.workunit.client.0.smithi063.stdout:8/993: readlink d1/d11/d3c/l40 0 2024-04-24T22:45:43.972 INFO:tasks.workunit.client.1.smithi192.stdout:8/765: dwrite db/d43/f9e [0,4194304] 0 2024-04-24T22:45:43.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/997: write d3/da/f6a [322324,84835] 0 2024-04-24T22:45:43.979 INFO:tasks.workunit.client.0.smithi063.stdout:0/879: symlink d6/d64/da0/l12d 0 2024-04-24T22:45:43.980 INFO:tasks.workunit.client.0.smithi063.stdout:5/887: dread d5/de5/fed [0,4194304] 0 2024-04-24T22:45:43.984 INFO:tasks.workunit.client.0.smithi063.stdout:3/948: symlink da/d18/d47/d4d/d9b/d125/l14a 0 2024-04-24T22:45:43.985 INFO:tasks.workunit.client.0.smithi063.stdout:8/994: creat d1/d11/d32/d38/db1/f138 x:0 0 0 2024-04-24T22:45:43.987 INFO:tasks.workunit.client.0.smithi063.stdout:1/998: dwrite d3/db5/d21/d9f/dd5/d106/d39/d115/f119 [0,4194304] 0 2024-04-24T22:45:43.995 INFO:tasks.workunit.client.0.smithi063.stdout:0/880: unlink d6/dd/c30 0 2024-04-24T22:45:43.997 INFO:tasks.workunit.client.0.smithi063.stdout:0/881: chown d6/d64/d21/d25/d31/da3/f100 2209 1 2024-04-24T22:45:43.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/971: sync 2024-04-24T22:45:44.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/882: write d6/d64/d21/d9c/f9d [5180216,43430] 0 2024-04-24T22:45:44.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/912: dwrite d2/d19/f2b [0,4194304] 0 2024-04-24T22:45:44.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/871: rmdir d0/d4/de/d17/d35/d69/da0/dda 0 2024-04-24T22:45:44.004 INFO:tasks.workunit.client.0.smithi063.stdout:3/949: creat da/d18/d47/f14b x:0 0 0 2024-04-24T22:45:44.009 INFO:tasks.workunit.client.0.smithi063.stdout:1/999: creat d3/db5/d21/d9f/dd5/d106/d39/f142 x:0 0 0 2024-04-24T22:45:44.016 INFO:tasks.workunit.client.0.smithi063.stdout:0/883: stat d6/d64/lc0 0 2024-04-24T22:45:44.019 INFO:tasks.workunit.client.0.smithi063.stdout:4/913: readlink d2/d80/d81/d90/lff 0 2024-04-24T22:45:44.020 INFO:tasks.workunit.client.0.smithi063.stdout:4/914: chown d2/d80/d8d/l110 930801243 1 2024-04-24T22:45:44.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/872: truncate d0/d4/de/f60 232890 0 2024-04-24T22:45:44.020 INFO:tasks.workunit.client.1.smithi192.stdout:0/783: dread d3/d6d/d79/f7a [0,4194304] 0 2024-04-24T22:45:44.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/888: dwrite d5/d11/f6a [4194304,4194304] 0 2024-04-24T22:45:44.022 INFO:tasks.workunit.client.0.smithi063.stdout:3/950: fsync da/d18/f61 0 2024-04-24T22:45:44.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/951: chown da/d40/ca8 10116 1 2024-04-24T22:45:44.023 INFO:tasks.workunit.client.1.smithi192.stdout:9/855: link d3/dd/d53/f7f d3/dd/de/d17/d89/d8b/f11b 0 2024-04-24T22:45:44.024 INFO:tasks.workunit.client.0.smithi063.stdout:3/952: dread - da/d18/d3a/da1/f137 zero size 2024-04-24T22:45:44.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/856: dread d3/dd/d62/fb6 [0,4194304] 0 2024-04-24T22:45:44.029 INFO:tasks.workunit.client.0.smithi063.stdout:3/953: dwrite da/f1d [0,4194304] 0 2024-04-24T22:45:44.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/915: creat d2/d80/d8d/f131 x:0 0 0 2024-04-24T22:45:44.032 INFO:tasks.workunit.client.1.smithi192.stdout:2/904: mkdir d4/d9/d9f/d126 0 2024-04-24T22:45:44.034 INFO:tasks.workunit.client.1.smithi192.stdout:3/787: dread d2/d27/dc6/f60 [0,4194304] 0 2024-04-24T22:45:44.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/873: truncate d0/d4/de/d6f/d2e/d55/d106/d10e/f11b 688659 0 2024-04-24T22:45:44.038 INFO:tasks.workunit.client.0.smithi063.stdout:7/874: write d0/d4/de/d17/d35/d69/fbc [1929410,24397] 0 2024-04-24T22:45:44.041 INFO:tasks.workunit.client.1.smithi192.stdout:0/784: creat d3/d27/d6a/deb/f105 x:0 0 0 2024-04-24T22:45:44.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/884: symlink d6/d64/d24/d42/dd4/l12e 0 2024-04-24T22:45:44.048 INFO:tasks.workunit.client.0.smithi063.stdout:9/972: write d6/d2f/dcd/d1f/feb [957530,78639] 0 2024-04-24T22:45:44.053 INFO:tasks.workunit.client.0.smithi063.stdout:8/995: getdents d1/d11/d32/d6a 0 2024-04-24T22:45:44.054 INFO:tasks.workunit.client.1.smithi192.stdout:8/766: creat db/d5d/dcb/fef x:0 0 0 2024-04-24T22:45:44.055 INFO:tasks.workunit.client.0.smithi063.stdout:7/875: mknod d0/d4/de/d6f/d2e/d76/c122 0 2024-04-24T22:45:44.058 INFO:tasks.workunit.client.1.smithi192.stdout:7/891: creat d2/f117 x:0 0 0 2024-04-24T22:45:44.060 INFO:tasks.workunit.client.1.smithi192.stdout:6/697: dwrite d0/d8/d3d/f5f [0,4194304] 0 2024-04-24T22:45:44.060 INFO:tasks.workunit.client.1.smithi192.stdout:7/892: dread - d2/db/d2a/d57/fb3 zero size 2024-04-24T22:45:44.060 INFO:tasks.workunit.client.0.smithi063.stdout:5/889: write d5/d11/d23/d2d/d62/d73/d118/d36/d131/fa5 [969727,87267] 0 2024-04-24T22:45:44.061 INFO:tasks.workunit.client.1.smithi192.stdout:6/698: chown d0/d8/d18/df1 151562785 1 2024-04-24T22:45:44.062 INFO:tasks.workunit.client.1.smithi192.stdout:6/699: stat d0/d8/d3d/d9d/fb2 0 2024-04-24T22:45:44.064 INFO:tasks.workunit.client.1.smithi192.stdout:6/700: chown d0/d8/d3d/d9d 99759419 1 2024-04-24T22:45:44.066 INFO:tasks.workunit.client.1.smithi192.stdout:8/767: mkdir db/d5d/df0 0 2024-04-24T22:45:44.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/857: write d3/dd/fbe [2220841,3064] 0 2024-04-24T22:45:44.069 INFO:tasks.workunit.client.1.smithi192.stdout:8/768: read db/f88 [2528937,34774] 0 2024-04-24T22:45:44.070 INFO:tasks.workunit.client.1.smithi192.stdout:8/769: read db/fae [393890,76665] 0 2024-04-24T22:45:44.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/788: link d2/d4/f3e d2/d27/d28/ffc 0 2024-04-24T22:45:44.074 INFO:tasks.workunit.client.0.smithi063.stdout:0/885: write d6/d64/d21/d25/d62/d63/d89/f113 [987300,83694] 0 2024-04-24T22:45:44.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/916: dwrite d2/d8/f32 [4194304,4194304] 0 2024-04-24T22:45:44.075 INFO:tasks.workunit.client.0.smithi063.stdout:3/954: write da/db6/f101 [726612,117587] 0 2024-04-24T22:45:44.077 INFO:tasks.workunit.client.0.smithi063.stdout:8/996: dread d1/d22/f31 [0,4194304] 0 2024-04-24T22:45:44.079 INFO:tasks.workunit.client.1.smithi192.stdout:3/789: read d2/f1e [3118127,105946] 0 2024-04-24T22:45:44.080 INFO:tasks.workunit.client.1.smithi192.stdout:6/701: unlink d0/d8/d29/d5a/d63/l87 0 2024-04-24T22:45:44.081 INFO:tasks.workunit.client.0.smithi063.stdout:9/973: write d6/d2f/d42/d81/fa9 [477037,36863] 0 2024-04-24T22:45:44.084 INFO:tasks.workunit.client.0.smithi063.stdout:0/886: dwrite d6/d64/d21/d25/d62/d63/d89/d95/fdc [0,4194304] 0 2024-04-24T22:45:44.088 INFO:tasks.workunit.client.0.smithi063.stdout:5/890: rename d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/dda/l102 to d5/d11/d23/d2d/d62/d81/da4/l134 0 2024-04-24T22:45:44.091 INFO:tasks.workunit.client.0.smithi063.stdout:7/876: dread d0/d4/de/d17/d35/d69/db1/fd3 [0,4194304] 0 2024-04-24T22:45:44.091 INFO:tasks.workunit.client.1.smithi192.stdout:9/858: mknod d3/dd/de/d17/d89/d8b/ded/c11c 0 2024-04-24T22:45:44.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/917: symlink d2/d80/l132 0 2024-04-24T22:45:44.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/918: stat d2/d19/d82/c94 0 2024-04-24T22:45:44.097 INFO:tasks.workunit.client.0.smithi063.stdout:3/955: write da/d18/d47/d4d/dc2/d120/f146 [623168,113402] 0 2024-04-24T22:45:44.098 INFO:tasks.workunit.client.1.smithi192.stdout:3/790: mkdir d2/d27/d28/dfd 0 2024-04-24T22:45:44.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/974: chown d6/d2f/d42/d81/def/c118 445 1 2024-04-24T22:45:44.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/887: mkdir d6/d64/d24/d42/dad/d126/d12f 0 2024-04-24T22:45:44.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/785: getdents d3/d27/d28/dcc/d5c 0 2024-04-24T22:45:44.107 INFO:tasks.workunit.client.0.smithi063.stdout:8/997: rename d1/d22/d33/da6/dae/de7 to d1/d21/df3/d139 0 2024-04-24T22:45:44.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/770: mknod db/dc/d22/d31/cf1 0 2024-04-24T22:45:44.111 INFO:tasks.workunit.client.1.smithi192.stdout:7/893: creat d2/db/d21/d42/d67/f118 x:0 0 0 2024-04-24T22:45:44.114 INFO:tasks.workunit.client.0.smithi063.stdout:4/919: dread d2/d19/d33/dc1/f6d [0,4194304] 0 2024-04-24T22:45:44.117 INFO:tasks.workunit.client.1.smithi192.stdout:3/791: mkdir d2/d2d/dfe 0 2024-04-24T22:45:44.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/894: dwrite d2/db/d21/d42/d67/d9b/d6e/db5/db6/fff [0,4194304] 0 2024-04-24T22:45:44.121 INFO:tasks.workunit.client.1.smithi192.stdout:7/895: stat d2/db/d36/f3d 0 2024-04-24T22:45:44.121 INFO:tasks.workunit.client.1.smithi192.stdout:6/702: link d0/d8/d18/d2e/c36 d0/d8/d29/d5a/ddf/cf7 0 2024-04-24T22:45:44.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/896: write d2/db/d21/f22 [6079307,115917] 0 2024-04-24T22:45:44.123 INFO:tasks.workunit.client.0.smithi063.stdout:4/920: mknod d2/d19/d33/d9d/da8/dec/c133 0 2024-04-24T22:45:44.126 INFO:tasks.workunit.client.0.smithi063.stdout:9/975: mkdir d6/d2f/dcd/db/d6b/d4f/d147 0 2024-04-24T22:45:44.128 INFO:tasks.workunit.client.1.smithi192.stdout:6/703: read d0/d8/d29/d4c/db0/fcc [1905116,17090] 0 2024-04-24T22:45:44.128 INFO:tasks.workunit.client.0.smithi063.stdout:0/888: link d6/d64/d21/d25/d62/d63/d7c/f12c d6/d64/da0/d98/dd6/f130 0 2024-04-24T22:45:44.132 INFO:tasks.workunit.client.0.smithi063.stdout:8/998: mkdir d1/d11/d13a 0 2024-04-24T22:45:44.134 INFO:tasks.workunit.client.1.smithi192.stdout:3/792: mkdir d2/d4/d12/d39/dff 0 2024-04-24T22:45:44.134 INFO:tasks.workunit.client.1.smithi192.stdout:0/786: dread d3/d27/d28/d31/f52 [0,4194304] 0 2024-04-24T22:45:44.135 INFO:tasks.workunit.client.1.smithi192.stdout:3/793: write d2/d4/f69 [1284233,61790] 0 2024-04-24T22:45:44.136 INFO:tasks.workunit.client.1.smithi192.stdout:0/787: chown d3/d27/d28/dcc/d5c/c73 62101281 1 2024-04-24T22:45:44.141 INFO:tasks.workunit.client.0.smithi063.stdout:9/976: dread - d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/f10a zero size 2024-04-24T22:45:44.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/704: fdatasync d0/d8/d3d/d3e/d7e/fae 0 2024-04-24T22:45:44.145 INFO:tasks.workunit.client.1.smithi192.stdout:8/771: link db/dc/d22/d31/d33/l7f db/d5d/lf2 0 2024-04-24T22:45:44.145 INFO:tasks.workunit.client.1.smithi192.stdout:0/788: dwrite d3/d27/d28/dcc/d5c/d80/fe0 [4194304,4194304] 0 2024-04-24T22:45:44.145 INFO:tasks.workunit.client.1.smithi192.stdout:0/789: fdatasync d3/d27/d28/f34 0 2024-04-24T22:45:44.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/790: write d3/d27/d28/f20 [4295685,33226] 0 2024-04-24T22:45:44.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/977: dread d6/d2f/dcd/d53/f99 [0,4194304] 0 2024-04-24T22:45:44.148 INFO:tasks.workunit.client.0.smithi063.stdout:5/891: getdents d5/d11/d23/d2d/d62/d73/d118/d36/d131 0 2024-04-24T22:45:44.149 INFO:tasks.workunit.client.0.smithi063.stdout:9/978: stat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f 0 2024-04-24T22:45:44.150 INFO:tasks.workunit.client.0.smithi063.stdout:7/877: link d0/d4/de/f60 d0/d4/de/d17/d28/d11a/f123 0 2024-04-24T22:45:44.152 INFO:tasks.workunit.client.0.smithi063.stdout:4/921: creat d2/d8/d3c/d111/f134 x:0 0 0 2024-04-24T22:45:44.156 INFO:tasks.workunit.client.0.smithi063.stdout:8/999: mknod d1/d2/d17/d124/c13b 0 2024-04-24T22:45:44.159 INFO:tasks.workunit.client.0.smithi063.stdout:5/892: creat d5/d11/d76/d92/dea/f135 x:0 0 0 2024-04-24T22:45:44.163 INFO:tasks.workunit.client.1.smithi192.stdout:3/794: truncate d2/f1e 4194955 0 2024-04-24T22:45:44.164 INFO:tasks.workunit.client.0.smithi063.stdout:4/922: mknod d2/d19/d82/c135 0 2024-04-24T22:45:44.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/859: dwrite d3/dd/de/da7/fcb [0,4194304] 0 2024-04-24T22:45:44.164 INFO:tasks.workunit.client.0.smithi063.stdout:3/956: write da/d18/d47/d4d/f106 [443254,83550] 0 2024-04-24T22:45:44.165 INFO:tasks.workunit.client.0.smithi063.stdout:3/957: chown da/fb4 574012681 1 2024-04-24T22:45:44.170 INFO:tasks.workunit.client.1.smithi192.stdout:0/791: rename d3/d27/d28/dcc/f7f to d3/d27/d28/dcc/d5c/d80/d83/dfb/f106 0 2024-04-24T22:45:44.173 INFO:tasks.workunit.client.0.smithi063.stdout:3/958: dwrite da/d18/d9e/fed [0,4194304] 0 2024-04-24T22:45:44.174 INFO:tasks.workunit.client.1.smithi192.stdout:9/860: dread d3/dd/de/d17/f8f [0,4194304] 0 2024-04-24T22:45:44.175 INFO:tasks.workunit.client.0.smithi063.stdout:0/889: creat d6/d64/d21/d25/d62/d63/d89/f131 x:0 0 0 2024-04-24T22:45:44.178 INFO:tasks.workunit.client.0.smithi063.stdout:5/893: truncate d5/d1d/faa 5128296 0 2024-04-24T22:45:44.181 INFO:tasks.workunit.client.0.smithi063.stdout:7/878: mkdir d0/d4/de/d6f/d2e/d76/d9f/dbd/d124 0 2024-04-24T22:45:44.185 INFO:tasks.workunit.client.0.smithi063.stdout:4/923: mkdir d2/d8/d36/d118/d136 0 2024-04-24T22:45:44.191 INFO:tasks.workunit.client.0.smithi063.stdout:3/959: rmdir da/d18/d47/d4d/dc2 39 2024-04-24T22:45:44.192 INFO:tasks.workunit.client.1.smithi192.stdout:0/792: mknod d3/d27/d6a/c107 0 2024-04-24T22:45:44.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/890: symlink d6/da/dc/l132 0 2024-04-24T22:45:44.193 INFO:tasks.workunit.client.1.smithi192.stdout:7/897: getdents d2/db/d21/d42/d67/d9b/d6e/d77 0 2024-04-24T22:45:44.194 INFO:tasks.workunit.client.0.smithi063.stdout:5/894: mkdir d5/d11/d18/d136 0 2024-04-24T22:45:44.194 INFO:tasks.workunit.client.1.smithi192.stdout:7/898: chown d2/db/d21/d42/d67/dae 11373 1 2024-04-24T22:45:44.195 INFO:tasks.workunit.client.0.smithi063.stdout:5/895: dread - d5/d11/d76/d92/dea/f135 zero size 2024-04-24T22:45:44.195 INFO:tasks.workunit.client.1.smithi192.stdout:9/861: symlink d3/dd/de/d17/d89/db4/l11d 0 2024-04-24T22:45:44.196 INFO:tasks.workunit.client.0.smithi063.stdout:7/879: mkdir d0/d4/de/d17/d35/d6c/d7a/d125 0 2024-04-24T22:45:44.196 INFO:tasks.workunit.client.1.smithi192.stdout:6/705: creat d0/d8/d18/ff8 x:0 0 0 2024-04-24T22:45:44.198 INFO:tasks.workunit.client.1.smithi192.stdout:6/706: dread d0/d8/d3d/f62 [0,4194304] 0 2024-04-24T22:45:44.198 INFO:tasks.workunit.client.0.smithi063.stdout:9/979: link d6/d2f/dcd/d53/dff/d132/de0/ffd d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/d9d/d100/f148 0 2024-04-24T22:45:44.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/795: creat d2/d4/d12/d1a/d1d/d55/d5f/dbd/f100 x:0 0 0 2024-04-24T22:45:44.200 INFO:tasks.workunit.client.0.smithi063.stdout:3/960: truncate da/d39/d94/dbc/fd3 4115445 0 2024-04-24T22:45:44.201 INFO:tasks.workunit.client.1.smithi192.stdout:6/707: read d0/d8/d3d/d3e/d40/d44/f73 [1007860,123491] 0 2024-04-24T22:45:44.202 INFO:tasks.workunit.client.1.smithi192.stdout:0/793: creat d3/d6d/f108 x:0 0 0 2024-04-24T22:45:44.205 INFO:tasks.workunit.client.1.smithi192.stdout:8/772: getdents db/dc/d22/d31/d33/d81 0 2024-04-24T22:45:44.210 INFO:tasks.workunit.client.1.smithi192.stdout:5/656: sync 2024-04-24T22:45:44.210 INFO:tasks.workunit.client.1.smithi192.stdout:4/755: sync 2024-04-24T22:45:44.210 INFO:tasks.workunit.client.1.smithi192.stdout:1/652: sync 2024-04-24T22:45:44.211 INFO:tasks.workunit.client.1.smithi192.stdout:6/708: creat d0/d8/d29/d4c/d4d/d6a/ff9 x:0 0 0 2024-04-24T22:45:44.213 INFO:tasks.workunit.client.0.smithi063.stdout:7/880: dread d0/d4/de/d6f/d2e/d76/d9f/dbd/fcf [0,4194304] 0 2024-04-24T22:45:44.213 INFO:tasks.workunit.client.1.smithi192.stdout:7/899: symlink d2/db/d49/d115/l119 0 2024-04-24T22:45:44.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/794: mknod d3/d27/d6a/dc4/c109 0 2024-04-24T22:45:44.215 INFO:tasks.workunit.client.1.smithi192.stdout:5/657: chown d0/d13/d19/l9 3 1 2024-04-24T22:45:44.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/773: dwrite db/d5d/dcb/fef [0,4194304] 0 2024-04-24T22:45:44.218 INFO:tasks.workunit.client.0.smithi063.stdout:0/891: dread d6/d64/d24/f59 [4194304,4194304] 0 2024-04-24T22:45:44.218 INFO:tasks.workunit.client.1.smithi192.stdout:6/709: truncate d0/d8/d3d/d3e/d40/fcf 548649 0 2024-04-24T22:45:44.219 INFO:tasks.workunit.client.0.smithi063.stdout:7/881: symlink d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/l126 0 2024-04-24T22:45:44.219 INFO:tasks.workunit.client.1.smithi192.stdout:8/774: dread - db/d43/d76/d92/fd1 zero size 2024-04-24T22:45:44.220 INFO:tasks.workunit.client.0.smithi063.stdout:3/961: getdents da/d18/d47/d4d/dc2 0 2024-04-24T22:45:44.221 INFO:tasks.workunit.client.0.smithi063.stdout:3/962: chown da/d18/d47/d4d/d62 49 1 2024-04-24T22:45:44.222 INFO:tasks.workunit.client.0.smithi063.stdout:3/963: dread - da/d40/f128 zero size 2024-04-24T22:45:44.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/795: creat d3/d27/d28/dcc/d5c/f10a x:0 0 0 2024-04-24T22:45:44.229 INFO:tasks.workunit.client.0.smithi063.stdout:3/964: symlink da/d39/d94/de4/l14c 0 2024-04-24T22:45:44.229 INFO:tasks.workunit.client.0.smithi063.stdout:3/965: fsync da/fb4 0 2024-04-24T22:45:44.229 INFO:tasks.workunit.client.0.smithi063.stdout:9/980: dread d6/d2f/dcd/d1f/d20/d22/f3b [0,4194304] 0 2024-04-24T22:45:44.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/658: unlink d0/d13/d19/d64/d4f/d6c/f81 0 2024-04-24T22:45:44.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/775: readlink db/dc/ld 0 2024-04-24T22:45:44.239 INFO:tasks.workunit.client.0.smithi063.stdout:3/966: symlink da/d18/d47/d4d/d9b/dde/df9/d132/l14d 0 2024-04-24T22:45:44.241 INFO:tasks.workunit.client.1.smithi192.stdout:6/710: dread d0/d8/d18/d2e/f8d [0,4194304] 0 2024-04-24T22:45:44.242 INFO:tasks.workunit.client.1.smithi192.stdout:8/776: dwrite db/dc/d22/f35 [0,4194304] 0 2024-04-24T22:45:44.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/900: mknod d2/db/d49/dca/d104/c11a 0 2024-04-24T22:45:44.250 INFO:tasks.workunit.client.0.smithi063.stdout:3/967: mknod da/d18/d47/d4d/d9b/d125/d131/c14e 0 2024-04-24T22:45:44.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/924: dwrite d2/f99 [0,4194304] 0 2024-04-24T22:45:44.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/896: write d5/d1d/d7d/ffb [210002,122655] 0 2024-04-24T22:45:44.253 INFO:tasks.workunit.client.0.smithi063.stdout:4/925: stat d2/d19/d33/dc1/d109 0 2024-04-24T22:45:44.257 INFO:tasks.workunit.client.1.smithi192.stdout:9/862: dwrite d3/dd/f49 [0,4194304] 0 2024-04-24T22:45:44.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/968: symlink da/d18/d3a/d147/l14f 0 2024-04-24T22:45:44.260 INFO:tasks.workunit.client.1.smithi192.stdout:3/796: dwrite d2/d4/d12/d1a/d1d/d7b/df7/f50 [0,4194304] 0 2024-04-24T22:45:44.265 INFO:tasks.workunit.client.0.smithi063.stdout:5/897: rename d5/d11/d76/d92/d10d to d5/d11/d23/d2d/d62/d73/d95/d137 0 2024-04-24T22:45:44.266 INFO:tasks.workunit.client.1.smithi192.stdout:0/796: mknod d3/d40/dc2/c10b 0 2024-04-24T22:45:44.270 INFO:tasks.workunit.client.0.smithi063.stdout:0/892: dread d6/da/dc/f71 [0,4194304] 0 2024-04-24T22:45:44.271 INFO:tasks.workunit.client.0.smithi063.stdout:4/926: rename d2/d19/f37 to d2/d19/d33/dc1/d50/dfa/f137 0 2024-04-24T22:45:44.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/882: dwrite d0/d4/de/d6f/f47 [0,4194304] 0 2024-04-24T22:45:44.276 INFO:tasks.workunit.client.1.smithi192.stdout:1/653: write d9/dd/d74/dcd/fcf [573931,126446] 0 2024-04-24T22:45:44.281 INFO:tasks.workunit.client.0.smithi063.stdout:5/898: symlink d5/d11/d23/d2d/d62/d73/d118/d49/l138 0 2024-04-24T22:45:44.282 INFO:tasks.workunit.client.1.smithi192.stdout:9/863: dread d3/d59/f113 [0,4194304] 0 2024-04-24T22:45:44.282 INFO:tasks.workunit.client.0.smithi063.stdout:5/899: stat d5/d11/d23/d2d/d62/d81/da4 0 2024-04-24T22:45:44.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/900: chown d5/d1d 299 1 2024-04-24T22:45:44.284 INFO:tasks.workunit.client.0.smithi063.stdout:5/901: dread - d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/dda/f112 zero size 2024-04-24T22:45:44.286 INFO:tasks.workunit.client.1.smithi192.stdout:7/901: symlink d2/db/d21/d42/d67/d9b/l11b 0 2024-04-24T22:45:44.287 INFO:tasks.workunit.client.0.smithi063.stdout:5/902: read d5/d11/d76/d92/dea/ff0 [3896149,120908] 0 2024-04-24T22:45:44.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/797: unlink d3/d27/d28/dcc/d5c/d80/d83/dd9/cda 0 2024-04-24T22:45:44.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/711: dwrite d0/d8/d29/f7c [0,4194304] 0 2024-04-24T22:45:44.292 INFO:tasks.workunit.client.1.smithi192.stdout:8/777: dwrite db/dc/d29/f89 [0,4194304] 0 2024-04-24T22:45:44.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/864: dread - d3/d59/f5a zero size 2024-04-24T22:45:44.298 INFO:tasks.workunit.client.1.smithi192.stdout:7/902: creat d2/db/d49/d115/f11c x:0 0 0 2024-04-24T22:45:44.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/654: creat d9/d15/d2d/d63/d68/d9d/fda x:0 0 0 2024-04-24T22:45:44.301 INFO:tasks.workunit.client.1.smithi192.stdout:9/865: read d3/d24/f41 [1241821,13641] 0 2024-04-24T22:45:44.307 INFO:tasks.workunit.client.1.smithi192.stdout:6/712: mknod d0/d8/d3d/d9d/da5/cfa 0 2024-04-24T22:45:44.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/659: truncate d0/d13/d19/d64/d4f/d52/f67 1853487 0 2024-04-24T22:45:44.310 INFO:tasks.workunit.client.0.smithi063.stdout:4/927: creat d2/d8/f138 x:0 0 0 2024-04-24T22:45:44.311 INFO:tasks.workunit.client.0.smithi063.stdout:7/883: rmdir d0/d4/de/d17/dd7/d10c 0 2024-04-24T22:45:44.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/866: dread d3/dd/de/d17/f8f [0,4194304] 0 2024-04-24T22:45:44.322 INFO:tasks.workunit.client.1.smithi192.stdout:5/660: symlink d0/d13/d19/da9/le3 0 2024-04-24T22:45:44.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/797: getdents d2/d62/db2 0 2024-04-24T22:45:44.322 INFO:tasks.workunit.client.1.smithi192.stdout:3/798: stat d2/d4/d12/d1a/d1d/d55/d5f/d9b/lda 0 2024-04-24T22:45:44.323 INFO:tasks.workunit.client.1.smithi192.stdout:7/903: creat d2/db/d21/d42/d67/da8/dc7/dfe/f11d x:0 0 0 2024-04-24T22:45:44.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/778: creat db/ff3 x:0 0 0 2024-04-24T22:45:44.333 INFO:tasks.workunit.client.0.smithi063.stdout:0/893: write d6/d64/d21/d25/d62/d63/d7c/fb6 [282549,44920] 0 2024-04-24T22:45:44.333 INFO:tasks.workunit.client.0.smithi063.stdout:0/894: chown d6/dd 575908482 1 2024-04-24T22:45:44.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/799: fdatasync d2/d27/dc6/f87 0 2024-04-24T22:45:44.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/800: truncate d2/d27/d28/dab/fc2 4285246 0 2024-04-24T22:45:44.338 INFO:tasks.workunit.client.1.smithi192.stdout:7/904: rmdir d2/db/d21/d42/d67/da8 39 2024-04-24T22:45:44.339 INFO:tasks.workunit.client.1.smithi192.stdout:8/779: dwrite db/dc/d22/d31/d33/f6d [0,4194304] 0 2024-04-24T22:45:44.342 INFO:tasks.workunit.client.1.smithi192.stdout:7/905: read d2/db/d21/d42/d67/d9b/d6e/db5/db6/f107 [896849,12645] 0 2024-04-24T22:45:44.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/655: rmdir d9/d15/d2d/d63 39 2024-04-24T22:45:44.349 INFO:tasks.workunit.client.1.smithi192.stdout:0/798: dwrite d3/d27/d28/d2e/f63 [0,4194304] 0 2024-04-24T22:45:44.353 INFO:tasks.workunit.client.1.smithi192.stdout:8/780: mknod db/d43/d7c/d85/db4/cf4 0 2024-04-24T22:45:44.358 INFO:tasks.workunit.client.1.smithi192.stdout:5/661: rmdir d0/d13 39 2024-04-24T22:45:44.361 INFO:tasks.workunit.client.1.smithi192.stdout:3/801: read d2/d4/d12/d1a/d1d/d55/d5f/d9b/fe2 [124720,57211] 0 2024-04-24T22:45:44.364 INFO:tasks.workunit.client.1.smithi192.stdout:7/906: truncate d2/db/d2a/f3c 5542103 0 2024-04-24T22:45:44.369 INFO:tasks.workunit.client.1.smithi192.stdout:0/799: fsync d3/d27/d28/dcc/d5c/d80/fb7 0 2024-04-24T22:45:44.370 INFO:tasks.workunit.client.1.smithi192.stdout:0/800: chown d3/d27/d3c/fab 49880 1 2024-04-24T22:45:44.371 INFO:tasks.workunit.client.1.smithi192.stdout:6/713: dwrite d0/d8/f2b [0,4194304] 0 2024-04-24T22:45:44.371 INFO:tasks.workunit.client.1.smithi192.stdout:8/781: mkdir db/d5d/d74/d96/df5 0 2024-04-24T22:45:44.373 INFO:tasks.workunit.client.1.smithi192.stdout:0/801: truncate d3/d27/d28/f20 9303175 0 2024-04-24T22:45:44.374 INFO:tasks.workunit.client.0.smithi063.stdout:9/981: sync 2024-04-24T22:45:44.376 INFO:tasks.workunit.client.1.smithi192.stdout:3/802: mknod d2/d27/dc6/d41/c101 0 2024-04-24T22:45:44.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/656: mkdir d9/d15/d2d/d48/d93/da4/db5/ddb 0 2024-04-24T22:45:44.380 INFO:tasks.workunit.client.1.smithi192.stdout:7/907: creat d2/db/d21/d42/d67/d9b/d6e/db5/db6/f11e x:0 0 0 2024-04-24T22:45:44.385 INFO:tasks.workunit.client.1.smithi192.stdout:6/714: symlink d0/d8/d22/dcb/lfb 0 2024-04-24T22:45:44.387 INFO:tasks.workunit.client.1.smithi192.stdout:7/908: dwrite d2/db/d21/fc6 [0,4194304] 0 2024-04-24T22:45:44.387 INFO:tasks.workunit.client.1.smithi192.stdout:9/867: truncate d3/d59/d7d/d88/dc2/fd8 684408 0 2024-04-24T22:45:44.391 INFO:tasks.workunit.client.1.smithi192.stdout:0/802: rmdir d3/d27/d28/d2e/d3f 39 2024-04-24T22:45:44.403 INFO:tasks.workunit.client.0.smithi063.stdout:3/969: rename da/l32 to da/d39/d94/db3/l150 0 2024-04-24T22:45:44.405 INFO:tasks.workunit.client.1.smithi192.stdout:2/905: sync 2024-04-24T22:45:44.414 INFO:tasks.workunit.client.0.smithi063.stdout:7/884: mknod d0/d4/de/d17/c127 0 2024-04-24T22:45:44.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/895: sync 2024-04-24T22:45:44.417 INFO:tasks.workunit.client.0.smithi063.stdout:0/896: chown d6/d64/d21/d80/dde 406 1 2024-04-24T22:45:44.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/657: mknod d9/d15/d2d/d63/d68/cdc 0 2024-04-24T22:45:44.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/897: readlink d6/d64/d24/l5a 0 2024-04-24T22:45:44.418 INFO:tasks.workunit.client.1.smithi192.stdout:8/782: write db/dc/d22/d31/f45 [375763,102283] 0 2024-04-24T22:45:44.421 INFO:tasks.workunit.client.1.smithi192.stdout:5/662: write d0/d13/d3b/f55 [1384212,55007] 0 2024-04-24T22:45:44.422 INFO:tasks.workunit.client.0.smithi063.stdout:3/970: creat da/d18/d3a/d63/d145/f151 x:0 0 0 2024-04-24T22:45:44.428 INFO:tasks.workunit.client.1.smithi192.stdout:5/663: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:44.429 INFO:tasks.workunit.client.1.smithi192.stdout:5/664: readlink d0/d1b/d50/l6b 0 2024-04-24T22:45:44.429 INFO:tasks.workunit.client.1.smithi192.stdout:1/658: chown d9/dd/cc3 3 1 2024-04-24T22:45:44.430 INFO:tasks.workunit.client.1.smithi192.stdout:3/803: dwrite d2/d4/d12/f88 [0,4194304] 0 2024-04-24T22:45:44.430 INFO:tasks.workunit.client.1.smithi192.stdout:5/665: chown d0/f8b 45 1 2024-04-24T22:45:44.432 INFO:tasks.workunit.client.0.smithi063.stdout:3/971: creat da/d18/d47/d4d/d62/f152 x:0 0 0 2024-04-24T22:45:44.433 INFO:tasks.workunit.client.0.smithi063.stdout:5/903: write d5/d11/d23/d2d/d62/d73/d118/d36/f5a [109143,112448] 0 2024-04-24T22:45:44.437 INFO:tasks.workunit.client.0.smithi063.stdout:0/898: mknod d6/d64/da0/c133 0 2024-04-24T22:45:44.438 INFO:tasks.workunit.client.0.smithi063.stdout:4/928: dwrite d2/d80/f103 [0,4194304] 0 2024-04-24T22:45:44.441 INFO:tasks.workunit.client.0.smithi063.stdout:9/982: getdents d6/d2f/dcd/d1f/d20/d2a 0 2024-04-24T22:45:44.446 INFO:tasks.workunit.client.0.smithi063.stdout:3/972: chown da/cba 1746146006 1 2024-04-24T22:45:44.446 INFO:tasks.workunit.client.1.smithi192.stdout:7/909: rename d2/db/d25/f27 to d2/db/f11f 0 2024-04-24T22:45:44.447 INFO:tasks.workunit.client.1.smithi192.stdout:6/715: dwrite d0/d8/f76 [0,4194304] 0 2024-04-24T22:45:44.447 INFO:tasks.workunit.client.1.smithi192.stdout:1/659: dwrite d9/d15/d2d/d63/d68/f76 [0,4194304] 0 2024-04-24T22:45:44.453 INFO:tasks.workunit.client.0.smithi063.stdout:5/904: mknod d5/d11/d23/d2d/d62/d81/da4/dee/c139 0 2024-04-24T22:45:44.454 INFO:tasks.workunit.client.1.smithi192.stdout:3/804: dwrite d2/d27/f6f [0,4194304] 0 2024-04-24T22:45:44.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/899: creat d6/d64/d21/d25/d62/d63/d89/d95/da4/f134 x:0 0 0 2024-04-24T22:45:44.458 INFO:tasks.workunit.client.0.smithi063.stdout:4/929: creat d2/d8/d3c/d111/f139 x:0 0 0 2024-04-24T22:45:44.460 INFO:tasks.workunit.client.0.smithi063.stdout:5/905: dwrite d5/d1d/dc0/dbb/f3e [0,4194304] 0 2024-04-24T22:45:44.462 INFO:tasks.workunit.client.0.smithi063.stdout:9/983: creat d6/d2f/dcd/d8a/f149 x:0 0 0 2024-04-24T22:45:44.463 INFO:tasks.workunit.client.0.smithi063.stdout:7/885: link d0/d4/de/d17/d35/l100 d0/d4/de/d6f/d2e/d76/d9f/l128 0 2024-04-24T22:45:44.464 INFO:tasks.workunit.client.1.smithi192.stdout:2/906: dwrite d4/d9/f90 [0,4194304] 0 2024-04-24T22:45:44.467 INFO:tasks.workunit.client.1.smithi192.stdout:8/783: dread db/d5d/f7e [0,4194304] 0 2024-04-24T22:45:44.471 INFO:tasks.workunit.client.1.smithi192.stdout:0/803: dwrite d3/d27/d28/dcc/d5c/d80/d83/dfb/f106 [0,4194304] 0 2024-04-24T22:45:44.472 INFO:tasks.workunit.client.1.smithi192.stdout:7/910: unlink d2/db/d21/d42/d5b/d63/f101 0 2024-04-24T22:45:44.475 INFO:tasks.workunit.client.0.smithi063.stdout:4/930: dread - d2/d8/dd4/dda/ffe zero size 2024-04-24T22:45:44.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/906: mkdir d5/d11/d76/d92/d13a 0 2024-04-24T22:45:44.480 INFO:tasks.workunit.client.0.smithi063.stdout:5/907: truncate d5/d11/f125 271161 0 2024-04-24T22:45:44.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/908: stat d5/d1d/dc0/dbb/f55 0 2024-04-24T22:45:44.483 INFO:tasks.workunit.client.1.smithi192.stdout:6/716: dread d0/d8/d3d/d3e/d40/d44/fbd [0,4194304] 0 2024-04-24T22:45:44.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/973: creat da/d39/d94/db3/dfa/d13e/f153 x:0 0 0 2024-04-24T22:45:44.490 INFO:tasks.workunit.client.1.smithi192.stdout:2/907: unlink d4/d9/l4b 0 2024-04-24T22:45:44.491 INFO:tasks.workunit.client.1.smithi192.stdout:2/908: fsync d4/d29/d60/d9a/db3/fd6 0 2024-04-24T22:45:44.492 INFO:tasks.workunit.client.0.smithi063.stdout:9/984: creat d6/d2f/dcd/db/dca/d127/f14a x:0 0 0 2024-04-24T22:45:44.493 INFO:tasks.workunit.client.1.smithi192.stdout:1/660: mkdir d9/d15/d5e/d90/da9/ddd 0 2024-04-24T22:45:44.495 INFO:tasks.workunit.client.0.smithi063.stdout:7/886: readlink d0/l11d 0 2024-04-24T22:45:44.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/887: chown d0/d4/de/d6f/d2e/d55/l9d 52192 1 2024-04-24T22:45:44.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/888: read - d0/d4/de/d17/d28/d31/d94/f11f zero size 2024-04-24T22:45:44.500 INFO:tasks.workunit.client.1.smithi192.stdout:8/784: creat db/dc/d22/dca/ff6 x:0 0 0 2024-04-24T22:45:44.500 INFO:tasks.workunit.client.1.smithi192.stdout:8/785: fdatasync db/fae 0 2024-04-24T22:45:44.503 INFO:tasks.workunit.client.1.smithi192.stdout:0/804: creat d3/d40/dc2/f10c x:0 0 0 2024-04-24T22:45:44.506 INFO:tasks.workunit.client.0.smithi063.stdout:7/889: fsync d0/d4/de/f1a 0 2024-04-24T22:45:44.509 INFO:tasks.workunit.client.1.smithi192.stdout:8/786: dwrite db/dc/d22/d31/d33/fbc [0,4194304] 0 2024-04-24T22:45:44.511 INFO:tasks.workunit.client.0.smithi063.stdout:4/931: unlink d2/d8/c27 0 2024-04-24T22:45:44.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/974: mkdir da/d18/d47/d7b/d141/d154 0 2024-04-24T22:45:44.517 INFO:tasks.workunit.client.1.smithi192.stdout:1/661: creat d9/d15/d2d/d48/d93/da4/dbb/dac/fde x:0 0 0 2024-04-24T22:45:44.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/666: rename d0/d1b/d50/fcd to d0/d13/d19/fe4 0 2024-04-24T22:45:44.521 INFO:tasks.workunit.client.0.smithi063.stdout:0/900: dwrite d6/da/dc/f1e [0,4194304] 0 2024-04-24T22:45:44.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/911: creat d2/db/d21/d42/d67/dae/dce/f120 x:0 0 0 2024-04-24T22:45:44.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/912: readlink d2/db/l1b 0 2024-04-24T22:45:44.526 INFO:tasks.workunit.client.0.smithi063.stdout:7/890: symlink d0/d4/de/d17/d28/d31/d99/d113/l129 0 2024-04-24T22:45:44.526 INFO:tasks.workunit.client.0.smithi063.stdout:5/909: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/f84 [0,4194304] 0 2024-04-24T22:45:44.527 INFO:tasks.workunit.client.0.smithi063.stdout:4/932: mkdir d2/d8/dee/d13a 0 2024-04-24T22:45:44.527 INFO:tasks.workunit.client.0.smithi063.stdout:3/975: dread da/d18/d3a/f69 [0,4194304] 0 2024-04-24T22:45:44.528 INFO:tasks.workunit.client.1.smithi192.stdout:0/805: dread d3/d27/d28/dcc/d5c/f9a [0,4194304] 0 2024-04-24T22:45:44.530 INFO:tasks.workunit.client.1.smithi192.stdout:6/717: mknod d0/d8/d18/cfc 0 2024-04-24T22:45:44.530 INFO:tasks.workunit.client.1.smithi192.stdout:0/806: write d3/d27/d3c/fe3 [754084,115409] 0 2024-04-24T22:45:44.531 INFO:tasks.workunit.client.0.smithi063.stdout:9/985: dwrite d6/d2f/dcd/d1f/f28 [0,4194304] 0 2024-04-24T22:45:44.532 INFO:tasks.workunit.client.1.smithi192.stdout:0/807: dread - d3/d27/d6a/ff4 zero size 2024-04-24T22:45:44.535 INFO:tasks.workunit.client.0.smithi063.stdout:0/901: unlink d6/da/dc/l1c 0 2024-04-24T22:45:44.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/909: creat d4/d9/d10/d6c/d8c/da5/de2/f127 x:0 0 0 2024-04-24T22:45:44.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/910: mknod d5/d11/d23/d2d/d62/d73/d118/d36/d65/c13b 0 2024-04-24T22:45:44.536 INFO:tasks.workunit.client.1.smithi192.stdout:2/910: readlink d4/d9/d10/d6c/d6f/l7e 0 2024-04-24T22:45:44.538 INFO:tasks.workunit.client.1.smithi192.stdout:1/662: creat d9/d15/d5e/fdf x:0 0 0 2024-04-24T22:45:44.538 INFO:tasks.workunit.client.0.smithi063.stdout:5/911: write d5/f6f [1432149,47532] 0 2024-04-24T22:45:44.539 INFO:tasks.workunit.client.0.smithi063.stdout:4/933: dwrite d2/d80/f103 [0,4194304] 0 2024-04-24T22:45:44.541 INFO:tasks.workunit.client.1.smithi192.stdout:3/805: link d2/fa2 d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/f102 0 2024-04-24T22:45:44.542 INFO:tasks.workunit.client.1.smithi192.stdout:3/806: chown d2/d27/d28/d38/l64 1636312955 1 2024-04-24T22:45:44.546 INFO:tasks.workunit.client.0.smithi063.stdout:9/986: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/f14b x:0 0 0 2024-04-24T22:45:44.552 INFO:tasks.workunit.client.0.smithi063.stdout:5/912: symlink d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/l13c 0 2024-04-24T22:45:44.553 INFO:tasks.workunit.client.1.smithi192.stdout:6/718: truncate d0/d8/d3d/d3e/d40/d44/fbd 863810 0 2024-04-24T22:45:44.555 INFO:tasks.workunit.client.0.smithi063.stdout:0/902: mkdir d6/d64/d21/d80/dde/d105/d135 0 2024-04-24T22:45:44.556 INFO:tasks.workunit.client.0.smithi063.stdout:4/934: creat d2/d8/d36/dd9/f13b x:0 0 0 2024-04-24T22:45:44.557 INFO:tasks.workunit.client.0.smithi063.stdout:4/935: chown d2/d9f/d117/d10a 5 1 2024-04-24T22:45:44.557 INFO:tasks.workunit.client.1.smithi192.stdout:5/667: creat d0/d1b/dcb/fe5 x:0 0 0 2024-04-24T22:45:44.558 INFO:tasks.workunit.client.1.smithi192.stdout:5/668: truncate d0/d1b/dcb/fe5 975800 0 2024-04-24T22:45:44.558 INFO:tasks.workunit.client.0.smithi063.stdout:3/976: creat da/d39/d94/db3/f155 x:0 0 0 2024-04-24T22:45:44.559 INFO:tasks.workunit.client.1.smithi192.stdout:7/913: mknod d2/db/d21/d42/c121 0 2024-04-24T22:45:44.562 INFO:tasks.workunit.client.0.smithi063.stdout:0/903: rename d6/d64/da0/d33/f99 to d6/d74/f136 0 2024-04-24T22:45:44.563 INFO:tasks.workunit.client.0.smithi063.stdout:3/977: mkdir da/d18/d9e/da3/d156 0 2024-04-24T22:45:44.564 INFO:tasks.workunit.client.0.smithi063.stdout:0/904: dread - d6/d64/d21/d25/d62/d63/d89/d95/df6/f11e zero size 2024-04-24T22:45:44.565 INFO:tasks.workunit.client.0.smithi063.stdout:5/913: link d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f48 d5/d11/d23/d2d/d62/d81/d10c/f13d 0 2024-04-24T22:45:44.567 INFO:tasks.workunit.client.1.smithi192.stdout:1/663: creat d9/d15/d5e/d90/da9/ddd/fe0 x:0 0 0 2024-04-24T22:45:44.567 INFO:tasks.workunit.client.1.smithi192.stdout:1/664: chown d9/d15/d2d/d48/d93/dc5/dd6 964413 1 2024-04-24T22:45:44.568 INFO:tasks.workunit.client.0.smithi063.stdout:0/905: rmdir d6/d64/d24/dd1 39 2024-04-24T22:45:44.568 INFO:tasks.workunit.client.1.smithi192.stdout:4/756: sync 2024-04-24T22:45:44.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/808: creat d3/d27/d28/dcc/f10d x:0 0 0 2024-04-24T22:45:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:4/936: dread d2/ff [0,4194304] 0 2024-04-24T22:45:44.573 INFO:tasks.workunit.client.0.smithi063.stdout:5/914: mknod d5/d11/d23/d2d/d62/d81/da4/da6/c13e 0 2024-04-24T22:45:44.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/915: dread - d5/d11/d23/d2d/d62/d73/d118/d49/fb4 zero size 2024-04-24T22:45:44.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/787: dwrite db/dc/f8e [0,4194304] 0 2024-04-24T22:45:44.576 INFO:tasks.workunit.client.0.smithi063.stdout:5/916: dread - d5/d11/d23/d2d/d62/d73/d118/f11c zero size 2024-04-24T22:45:44.580 INFO:tasks.workunit.client.0.smithi063.stdout:4/937: creat d2/d8/d36/dd9/f13c x:0 0 0 2024-04-24T22:45:44.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/807: dread d2/f76 [0,4194304] 0 2024-04-24T22:45:44.583 INFO:tasks.workunit.client.1.smithi192.stdout:6/719: dread d0/d8/d29/f3b [0,4194304] 0 2024-04-24T22:45:44.583 INFO:tasks.workunit.client.1.smithi192.stdout:4/757: rename d2/db/d75/da1/d36/d5b/d28/f45 to d2/db/d75/da1/d7d/f109 0 2024-04-24T22:45:44.584 INFO:tasks.workunit.client.1.smithi192.stdout:4/758: dread - d2/d5c/dce/fd8 zero size 2024-04-24T22:45:44.588 INFO:tasks.workunit.client.0.smithi063.stdout:4/938: dwrite d2/ff0 [0,4194304] 0 2024-04-24T22:45:44.624 INFO:tasks.workunit.client.0.smithi063.stdout:0/906: truncate d6/d64/d21/fb3 1188457 0 2024-04-24T22:45:44.624 INFO:tasks.workunit.client.0.smithi063.stdout:0/907: dread - d6/d64/d21/d80/f129 zero size 2024-04-24T22:45:44.643 INFO:tasks.workunit.client.0.smithi063.stdout:0/908: dread d6/d64/d21/f48 [0,4194304] 0 2024-04-24T22:45:44.643 INFO:tasks.workunit.client.0.smithi063.stdout:7/891: sync 2024-04-24T22:45:44.645 INFO:tasks.workunit.client.0.smithi063.stdout:7/892: read d0/fa2 [1511479,126009] 0 2024-04-24T22:45:44.652 INFO:tasks.workunit.client.0.smithi063.stdout:7/893: dwrite d0/d4/de/d6f/d2e/d76/ff3 [0,4194304] 0 2024-04-24T22:45:44.664 INFO:tasks.workunit.client.1.smithi192.stdout:2/911: write d4/d9/fa4 [682252,42508] 0 2024-04-24T22:45:44.668 INFO:tasks.workunit.client.0.smithi063.stdout:4/939: symlink d2/d8/d40/d51/dd5/l13d 0 2024-04-24T22:45:44.669 INFO:tasks.workunit.client.0.smithi063.stdout:4/940: stat d2/d8/d36/d118/d11e 0 2024-04-24T22:45:44.670 INFO:tasks.workunit.client.1.smithi192.stdout:8/788: mkdir db/d43/d7c/d85/df7 0 2024-04-24T22:45:44.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/789: fsync db/dc/d22/d31/d33/d81/fb1 0 2024-04-24T22:45:44.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/790: stat db/dc/d22/l27 0 2024-04-24T22:45:44.675 INFO:tasks.workunit.client.0.smithi063.stdout:0/909: creat d6/d64/d21/d25/d31/da3/f137 x:0 0 0 2024-04-24T22:45:44.682 INFO:tasks.workunit.client.0.smithi063.stdout:3/978: write da/f3d [2785459,74640] 0 2024-04-24T22:45:44.683 INFO:tasks.workunit.client.0.smithi063.stdout:9/987: sync 2024-04-24T22:45:44.684 INFO:tasks.workunit.client.1.smithi192.stdout:5/669: truncate d0/d13/d19/d64/d4f/d52/f6a 303442 0 2024-04-24T22:45:44.685 INFO:tasks.workunit.client.1.smithi192.stdout:2/912: mknod d4/d9/dd4/c128 0 2024-04-24T22:45:44.685 INFO:tasks.workunit.client.1.smithi192.stdout:5/670: readlink d0/d13/d19/d64/d4f/d6c/l9e 0 2024-04-24T22:45:44.687 INFO:tasks.workunit.client.1.smithi192.stdout:1/665: write d9/f1a [3924855,1184] 0 2024-04-24T22:45:44.700 INFO:tasks.workunit.client.1.smithi192.stdout:5/671: truncate d0/d1b/f89 2341066 0 2024-04-24T22:45:44.701 INFO:tasks.workunit.client.1.smithi192.stdout:5/672: fdatasync d0/d13/d19/d64/d5f/f72 0 2024-04-24T22:45:44.702 INFO:tasks.workunit.client.1.smithi192.stdout:5/673: readlink d0/d13/d19/da9/le3 0 2024-04-24T22:45:44.702 INFO:tasks.workunit.client.1.smithi192.stdout:5/674: readlink d0/d1b/l26 0 2024-04-24T22:45:44.711 INFO:tasks.workunit.client.1.smithi192.stdout:2/913: getdents d4/d9/d10/da7/df5 0 2024-04-24T22:45:44.713 INFO:tasks.workunit.client.0.smithi063.stdout:5/917: getdents d5/d121 0 2024-04-24T22:45:44.715 INFO:tasks.workunit.client.0.smithi063.stdout:5/918: write d5/d11/d23/d2d/d62/d73/d118/db6/ff6 [859975,11381] 0 2024-04-24T22:45:44.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/809: getdents d3/d27/d28/dcc/d5c/d80/dac 0 2024-04-24T22:45:44.717 INFO:tasks.workunit.client.0.smithi063.stdout:9/988: mknod d6/d2f/c14c 0 2024-04-24T22:45:44.718 INFO:tasks.workunit.client.1.smithi192.stdout:9/868: sync 2024-04-24T22:45:44.721 INFO:tasks.workunit.client.1.smithi192.stdout:1/666: symlink d9/dd/le1 0 2024-04-24T22:45:44.722 INFO:tasks.workunit.client.0.smithi063.stdout:5/919: symlink d5/d11/dec/l13f 0 2024-04-24T22:45:44.727 INFO:tasks.workunit.client.0.smithi063.stdout:9/989: creat d6/d2f/dcd/db/d6b/d4f/dad/d48/d94/d13f/d9d/d100/d11c/f14d x:0 0 0 2024-04-24T22:45:44.727 INFO:tasks.workunit.client.1.smithi192.stdout:1/667: dwrite d9/d15/d5e/fdf [0,4194304] 0 2024-04-24T22:45:44.732 INFO:tasks.workunit.client.1.smithi192.stdout:0/810: read d3/d27/d6a/dc4/fd3 [4052585,73512] 0 2024-04-24T22:45:44.733 INFO:tasks.workunit.client.0.smithi063.stdout:5/920: dread d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/feb [0,4194304] 0 2024-04-24T22:45:44.734 INFO:tasks.workunit.client.1.smithi192.stdout:0/811: dread - d3/d27/d6a/ff4 zero size 2024-04-24T22:45:44.737 INFO:tasks.workunit.client.0.smithi063.stdout:9/990: mkdir d6/d2f/dcd/db/d6b/d4f/d14e 0 2024-04-24T22:45:44.740 INFO:tasks.workunit.client.1.smithi192.stdout:8/791: link db/dc/d22/d31/d33/d81/d8a/f8b db/dc/ff8 0 2024-04-24T22:45:44.742 INFO:tasks.workunit.client.0.smithi063.stdout:5/921: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/f11d [0,4194304] 0 2024-04-24T22:45:44.745 INFO:tasks.workunit.client.1.smithi192.stdout:0/812: fdatasync d3/d27/d28/d2e/d3f/fc8 0 2024-04-24T22:45:44.751 INFO:tasks.workunit.client.1.smithi192.stdout:0/813: creat d3/d27/d28/dcc/d5c/f10e x:0 0 0 2024-04-24T22:45:44.753 INFO:tasks.workunit.client.1.smithi192.stdout:8/792: creat db/dc/d22/d31/d33/d77/ff9 x:0 0 0 2024-04-24T22:45:44.754 INFO:tasks.workunit.client.1.smithi192.stdout:8/793: read db/d43/d76/f94 [343201,36528] 0 2024-04-24T22:45:44.757 INFO:tasks.workunit.client.0.smithi063.stdout:5/922: link d5/d11/d18/c53 d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/d10e/c140 0 2024-04-24T22:45:44.760 INFO:tasks.workunit.client.0.smithi063.stdout:5/923: symlink d5/d11/d23/d2d/d62/d81/d8f/l141 0 2024-04-24T22:45:44.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/808: symlink d2/d4/d12/d1a/d1d/d55/d5f/db1/l103 0 2024-04-24T22:45:44.762 INFO:tasks.workunit.client.0.smithi063.stdout:5/924: stat d5/d11/d23/d2d/d62/d73/d118/d36/d65/cd9 0 2024-04-24T22:45:44.763 INFO:tasks.workunit.client.0.smithi063.stdout:5/925: dread - d5/d11/dec/ffc zero size 2024-04-24T22:45:44.765 INFO:tasks.workunit.client.0.smithi063.stdout:5/926: symlink d5/d11/d18/l142 0 2024-04-24T22:45:44.767 INFO:tasks.workunit.client.1.smithi192.stdout:1/668: dread d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:44.770 INFO:tasks.workunit.client.1.smithi192.stdout:3/809: link d2/d27/dc6/d4f/f7a d2/d4/d12/d1a/d1d/d7b/ddb/f104 0 2024-04-24T22:45:44.771 INFO:tasks.workunit.client.0.smithi063.stdout:5/927: getdents d5/d11/d23/d2d/d62/d73/d118/d36/d131/dc9 0 2024-04-24T22:45:44.772 INFO:tasks.workunit.client.1.smithi192.stdout:8/794: dread db/f5a [0,4194304] 0 2024-04-24T22:45:44.803 INFO:tasks.workunit.client.1.smithi192.stdout:6/720: mknod d0/d8/d3d/d3e/d40/cfd 0 2024-04-24T22:45:44.805 INFO:tasks.workunit.client.1.smithi192.stdout:6/721: creat d0/d8/d29/d4c/d4d/d6a/ffe x:0 0 0 2024-04-24T22:45:44.806 INFO:tasks.workunit.client.1.smithi192.stdout:6/722: truncate d0/d8/d18/fca 566725 0 2024-04-24T22:45:44.808 INFO:tasks.workunit.client.1.smithi192.stdout:6/723: mkdir d0/d8/d18/dff 0 2024-04-24T22:45:44.811 INFO:tasks.workunit.client.1.smithi192.stdout:6/724: rmdir d0/d8/d3d/d9d/dd6/ddc 39 2024-04-24T22:45:44.813 INFO:tasks.workunit.client.1.smithi192.stdout:4/759: dwrite d2/d9/f8a [0,4194304] 0 2024-04-24T22:45:44.815 INFO:tasks.workunit.client.0.smithi063.stdout:7/894: dwrite d0/d4/de/f8d [0,4194304] 0 2024-04-24T22:45:44.815 INFO:tasks.workunit.client.1.smithi192.stdout:6/725: write d0/d8/d22/f27 [1267367,106918] 0 2024-04-24T22:45:44.817 INFO:tasks.workunit.client.1.smithi192.stdout:4/760: creat d2/db/d75/da1/d36/d5b/d64/f10a x:0 0 0 2024-04-24T22:45:44.821 INFO:tasks.workunit.client.1.smithi192.stdout:6/726: creat d0/d8/d22/dcb/de0/de7/f100 x:0 0 0 2024-04-24T22:45:44.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/941: dwrite d2/d80/d8d/f76 [0,4194304] 0 2024-04-24T22:45:44.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/942: stat d2/d8/lac 0 2024-04-24T22:45:44.824 INFO:tasks.workunit.client.0.smithi063.stdout:4/943: write d2/d9f/d117/f11f [568156,13696] 0 2024-04-24T22:45:44.830 INFO:tasks.workunit.client.0.smithi063.stdout:0/910: write d6/d64/d21/d25/d62/fb7 [351758,34277] 0 2024-04-24T22:45:44.831 INFO:tasks.workunit.client.1.smithi192.stdout:6/727: dwrite d0/d8/d3d/f5f [0,4194304] 0 2024-04-24T22:45:44.834 INFO:tasks.workunit.client.1.smithi192.stdout:4/761: dwrite d2/db/d75/da1/d7d/f109 [0,4194304] 0 2024-04-24T22:45:44.840 INFO:tasks.workunit.client.1.smithi192.stdout:6/728: symlink d0/d8/d3d/l101 0 2024-04-24T22:45:44.842 INFO:tasks.workunit.client.1.smithi192.stdout:4/762: truncate d2/db/d75/f8c 942708 0 2024-04-24T22:45:44.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/979: write da/f4c [443640,11443] 0 2024-04-24T22:45:44.849 INFO:tasks.workunit.client.1.smithi192.stdout:4/763: creat d2/db/d75/da1/d36/d5b/d28/d9f/f10b x:0 0 0 2024-04-24T22:45:44.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/764: unlink d2/db/d75/da1/d36/d5b/d28/lee 0 2024-04-24T22:45:44.854 INFO:tasks.workunit.client.1.smithi192.stdout:0/814: mknod d3/d27/c10f 0 2024-04-24T22:45:44.860 INFO:tasks.workunit.client.1.smithi192.stdout:8/795: fsync db/dc/d22/d31/d33/d81/d8a/f8b 0 2024-04-24T22:45:44.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/765: dread d2/db/f52 [0,4194304] 0 2024-04-24T22:45:44.862 INFO:tasks.workunit.client.1.smithi192.stdout:8/796: write db/dc/d29/f2b [6460325,59289] 0 2024-04-24T22:45:44.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/766: chown d2/db/d10/l35 4224181 1 2024-04-24T22:45:44.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/914: write d4/d9/d10/da7/fca [864060,88410] 0 2024-04-24T22:45:44.863 INFO:tasks.workunit.client.1.smithi192.stdout:8/797: write db/dc/f8e [2514956,128900] 0 2024-04-24T22:45:44.865 INFO:tasks.workunit.client.0.smithi063.stdout:4/944: dread d2/d8/dd4/dda/dc6/ff4 [0,4194304] 0 2024-04-24T22:45:44.866 INFO:tasks.workunit.client.1.smithi192.stdout:5/675: truncate d0/d13/d3b/d2f/f84 2419917 0 2024-04-24T22:45:44.868 INFO:tasks.workunit.client.1.smithi192.stdout:8/798: dread db/dc/f8e [0,4194304] 0 2024-04-24T22:45:44.869 INFO:tasks.workunit.client.1.smithi192.stdout:9/869: write d3/dd/fb8 [895274,15016] 0 2024-04-24T22:45:44.870 INFO:tasks.workunit.client.1.smithi192.stdout:8/799: dread db/fae [0,4194304] 0 2024-04-24T22:45:44.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/810: unlink d2/d4/d12/d1a/d1d/d55/cb 0 2024-04-24T22:45:44.872 INFO:tasks.workunit.client.0.smithi063.stdout:4/945: dwrite d2/d8/d40/f68 [0,4194304] 0 2024-04-24T22:45:44.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/800: fsync db/dc/f24 0 2024-04-24T22:45:44.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/870: dread d3/dd/de/d17/f35 [0,4194304] 0 2024-04-24T22:45:44.874 INFO:tasks.workunit.client.1.smithi192.stdout:8/801: chown db/d43/d7c/d85/db4/cf4 1 1 2024-04-24T22:45:44.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/946: write d2/d19/f25 [2894891,79095] 0 2024-04-24T22:45:44.877 INFO:tasks.workunit.client.0.smithi063.stdout:4/947: read d2/d80/f8f [5628417,118026] 0 2024-04-24T22:45:44.879 INFO:tasks.workunit.client.0.smithi063.stdout:4/948: write d2/d80/f103 [4352802,107715] 0 2024-04-24T22:45:44.879 INFO:tasks.workunit.client.1.smithi192.stdout:5/676: link d0/d13/d19/l63 d0/d13/d19/da9/le6 0 2024-04-24T22:45:44.885 INFO:tasks.workunit.client.1.smithi192.stdout:5/677: mkdir d0/d13/d19/d64/de7 0 2024-04-24T22:45:44.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/871: dwrite d3/dd/d53/f5c [4194304,4194304] 0 2024-04-24T22:45:44.887 INFO:tasks.workunit.client.0.smithi063.stdout:4/949: creat d2/d8/dd4/dda/da5/f13e x:0 0 0 2024-04-24T22:45:44.888 INFO:tasks.workunit.client.1.smithi192.stdout:9/872: stat d3/d5d/fb2 0 2024-04-24T22:45:44.894 INFO:tasks.workunit.client.1.smithi192.stdout:9/873: read d3/d24/f65 [359310,4488] 0 2024-04-24T22:45:44.898 INFO:tasks.workunit.client.1.smithi192.stdout:9/874: dread d3/dd/f49 [0,4194304] 0 2024-04-24T22:45:44.899 INFO:tasks.workunit.client.0.smithi063.stdout:5/928: write d5/d11/d23/f4e [5201481,25275] 0 2024-04-24T22:45:44.902 INFO:tasks.workunit.client.1.smithi192.stdout:9/875: mknod d3/d59/d7d/d88/c11e 0 2024-04-24T22:45:44.902 INFO:tasks.workunit.client.1.smithi192.stdout:9/876: fdatasync d3/d59/dbf/fd9 0 2024-04-24T22:45:44.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/929: read d5/d11/f15 [83788,104518] 0 2024-04-24T22:45:44.903 INFO:tasks.workunit.client.1.smithi192.stdout:3/811: rmdir d2/d27/dc6 39 2024-04-24T22:45:44.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/812: write d2/d27/fdc [235525,9437] 0 2024-04-24T22:45:44.906 INFO:tasks.workunit.client.1.smithi192.stdout:3/813: write d2/d27/d28/f30 [302588,124791] 0 2024-04-24T22:45:44.907 INFO:tasks.workunit.client.1.smithi192.stdout:3/814: fsync d2/d27/d28/dab/dca/fcf 0 2024-04-24T22:45:44.908 INFO:tasks.workunit.client.0.smithi063.stdout:5/930: dwrite d5/f6f [0,4194304] 0 2024-04-24T22:45:44.909 INFO:tasks.workunit.client.0.smithi063.stdout:9/991: sync 2024-04-24T22:45:44.912 INFO:tasks.workunit.client.1.smithi192.stdout:3/815: link d2/d27/d28/f36 d2/d27/d28/dab/dca/dee/f105 0 2024-04-24T22:45:44.914 INFO:tasks.workunit.client.1.smithi192.stdout:3/816: write d2/d4/d12/d1a/d1d/d7b/df7/f50 [2036282,89436] 0 2024-04-24T22:45:44.915 INFO:tasks.workunit.client.1.smithi192.stdout:5/678: dread d0/d1b/d50/f7d [0,4194304] 0 2024-04-24T22:45:44.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/817: write d2/d27/d28/da5/f66 [44827,4871] 0 2024-04-24T22:45:44.919 INFO:tasks.workunit.client.0.smithi063.stdout:9/992: dwrite d6/d2f/dcd/db/d1c/d78/d9a/f144 [0,4194304] 0 2024-04-24T22:45:44.920 INFO:tasks.workunit.client.0.smithi063.stdout:5/931: rename d5/d11/d23/d2d/d62/d73/d118/db6/fbd to d5/d11/d23/d2d/d62/f143 0 2024-04-24T22:45:44.921 INFO:tasks.workunit.client.0.smithi063.stdout:5/932: dread - d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/dda/f112 zero size 2024-04-24T22:45:44.922 INFO:tasks.workunit.client.1.smithi192.stdout:5/679: symlink d0/d13/d3b/d79/le8 0 2024-04-24T22:45:44.924 INFO:tasks.workunit.client.0.smithi063.stdout:9/993: mkdir d6/d2f/dcd/d1f/d14f 0 2024-04-24T22:45:44.929 INFO:tasks.workunit.client.0.smithi063.stdout:5/933: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/d3b/f58 [0,4194304] 0 2024-04-24T22:45:44.931 INFO:tasks.workunit.client.0.smithi063.stdout:7/895: write d0/d4/de/d17/d35/f54 [289343,94881] 0 2024-04-24T22:45:44.936 INFO:tasks.workunit.client.0.smithi063.stdout:5/934: read d5/d11/d23/d2d/d62/d73/d118/d36/fd2 [599890,76425] 0 2024-04-24T22:45:44.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/877: dread d3/dd/de/d17/f48 [0,4194304] 0 2024-04-24T22:45:44.939 INFO:tasks.workunit.client.0.smithi063.stdout:7/896: dwrite d0/d4/de/d17/d28/f112 [0,4194304] 0 2024-04-24T22:45:44.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/935: creat d5/d1d/dc7/f144 x:0 0 0 2024-04-24T22:45:44.943 INFO:tasks.workunit.client.0.smithi063.stdout:0/911: write d6/d3a/fcb [892731,104997] 0 2024-04-24T22:45:44.943 INFO:tasks.workunit.client.0.smithi063.stdout:9/994: link d6/d2f/dcd/d1f/d20/d22/l65 d6/d2f/dcd/d53/dff/d132/l150 0 2024-04-24T22:45:44.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/912: chown d6/d3a/fb0 65208 1 2024-04-24T22:45:44.946 INFO:tasks.workunit.client.0.smithi063.stdout:7/897: creat d0/d4/de/d6f/d2e/d76/f12a x:0 0 0 2024-04-24T22:45:44.950 INFO:tasks.workunit.client.0.smithi063.stdout:3/980: write da/f54 [347060,42902] 0 2024-04-24T22:45:44.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/981: truncate da/d39/d94/db3/fc3 4628526 0 2024-04-24T22:45:44.959 INFO:tasks.workunit.client.0.smithi063.stdout:7/898: unlink d0/d4/de/d6f/d2e/d76/d9f/l128 0 2024-04-24T22:45:44.973 INFO:tasks.workunit.client.1.smithi192.stdout:7/914: sync 2024-04-24T22:45:44.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/899: truncate d0/d4/f19 1419567 0 2024-04-24T22:45:44.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/950: rmdir d2/d8/dd4/dda 39 2024-04-24T22:45:44.980 INFO:tasks.workunit.client.0.smithi063.stdout:4/951: readlink d2/d8/d36/l38 0 2024-04-24T22:45:44.981 INFO:tasks.workunit.client.1.smithi192.stdout:8/802: dwrite db/dc/d22/d31/f4c [0,4194304] 0 2024-04-24T22:45:44.985 INFO:tasks.workunit.client.0.smithi063.stdout:4/952: dread - d2/d8/dd4/dda/f112 zero size 2024-04-24T22:45:44.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/953: chown d2/d8/d36/faf 115 1 2024-04-24T22:45:44.988 INFO:tasks.workunit.client.0.smithi063.stdout:4/954: fdatasync d2/d19/d33/d9d/da8/f115 0 2024-04-24T22:45:44.988 INFO:tasks.workunit.client.0.smithi063.stdout:7/900: link d0/l23 d0/d4/de/d17/d35/d69/dc9/l12b 0 2024-04-24T22:45:44.994 INFO:tasks.workunit.client.0.smithi063.stdout:4/955: creat d2/d8/d36/d118/f13f x:0 0 0 2024-04-24T22:45:44.995 INFO:tasks.workunit.client.0.smithi063.stdout:4/956: write d2/d19/f25 [4965671,85177] 0 2024-04-24T22:45:44.997 INFO:tasks.workunit.client.0.smithi063.stdout:7/901: mkdir d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/d12c 0 2024-04-24T22:45:45.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/957: mknod d2/d8/d36/d118/c140 0 2024-04-24T22:45:45.003 INFO:tasks.workunit.client.0.smithi063.stdout:0/913: dread d6/d64/d24/f73 [0,4194304] 0 2024-04-24T22:45:45.006 INFO:tasks.workunit.client.0.smithi063.stdout:7/902: creat d0/d4/de/d17/d35/d69/dc9/d120/f12d x:0 0 0 2024-04-24T22:45:45.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/903: chown d0/d4/de/d17/d35/d69/da0 380427 1 2024-04-24T22:45:45.007 INFO:tasks.workunit.client.1.smithi192.stdout:9/878: unlink d3/cc3 0 2024-04-24T22:45:45.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/914: mknod d6/dd/c138 0 2024-04-24T22:45:45.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/958: symlink d2/d8/dd4/l141 0 2024-04-24T22:45:45.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/959: fsync d2/d8/f138 0 2024-04-24T22:45:45.012 INFO:tasks.workunit.client.0.smithi063.stdout:4/960: chown d2/d8/dd4/dda/da5/f13e 14984620 1 2024-04-24T22:45:45.014 INFO:tasks.workunit.client.0.smithi063.stdout:4/961: chown d2/d8/d36/dd9/f13b 27813 1 2024-04-24T22:45:45.017 INFO:tasks.workunit.client.1.smithi192.stdout:7/915: fsync d2/db/f11f 0 2024-04-24T22:45:45.018 INFO:tasks.workunit.client.0.smithi063.stdout:5/936: dwrite d5/d11/d23/d2d/d62/d81/d10c/f13d [4194304,4194304] 0 2024-04-24T22:45:45.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/915: mkdir d6/d64/da0/d33/deb/d139 0 2024-04-24T22:45:45.019 INFO:tasks.workunit.client.1.smithi192.stdout:0/815: dwrite d3/d27/d28/dcc/d5c/d80/d83/dfb/f106 [4194304,4194304] 0 2024-04-24T22:45:45.020 INFO:tasks.workunit.client.0.smithi063.stdout:9/995: dwrite d6/d2f/dcd/db/d1c/d78/d9a/daa/f101 [0,4194304] 0 2024-04-24T22:45:45.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/982: write da/d18/d3a/da1/fbd [486590,65996] 0 2024-04-24T22:45:45.024 INFO:tasks.workunit.client.0.smithi063.stdout:9/996: write d6/d2f/dcd/db/d6b/d4f/dad/d36/fe5 [1213161,79186] 0 2024-04-24T22:45:45.028 INFO:tasks.workunit.client.1.smithi192.stdout:9/879: unlink d3/dd/f38 0 2024-04-24T22:45:45.029 INFO:tasks.workunit.client.0.smithi063.stdout:0/916: dwrite d6/d64/d24/d42/dd4/ff4 [0,4194304] 0 2024-04-24T22:45:45.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/916: creat d2/db/d25/dda/f122 x:0 0 0 2024-04-24T22:45:45.034 INFO:tasks.workunit.client.0.smithi063.stdout:3/983: rmdir da/d18/d115 39 2024-04-24T22:45:45.035 INFO:tasks.workunit.client.1.smithi192.stdout:7/917: write d2/db/d2a/ff7 [998786,86367] 0 2024-04-24T22:45:45.040 INFO:tasks.workunit.client.0.smithi063.stdout:9/997: mknod d6/d2f/dcd/db/d93/da0/c151 0 2024-04-24T22:45:45.041 INFO:tasks.workunit.client.1.smithi192.stdout:0/816: read d3/d27/d28/d2e/dc7/fd0 [272556,56376] 0 2024-04-24T22:45:45.049 INFO:tasks.workunit.client.1.smithi192.stdout:7/918: creat d2/db/d21/d42/d67/dae/dbb/f123 x:0 0 0 2024-04-24T22:45:45.051 INFO:tasks.workunit.client.0.smithi063.stdout:3/984: mknod da/d18/d3a/da1/c157 0 2024-04-24T22:45:45.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/919: creat d2/db/d25/dda/f124 x:0 0 0 2024-04-24T22:45:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:44 smithi063 ceph-mon[26912]: pgmap v20: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 189 MiB/s wr, 430 op/s 2024-04-24T22:45:45.055 INFO:tasks.workunit.client.0.smithi063.stdout:4/962: getdents d2/d8 0 2024-04-24T22:45:45.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/963: symlink d2/d8/d36/l142 0 2024-04-24T22:45:45.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/964: mkdir d2/d19/dce/d143 0 2024-04-24T22:45:45.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/904: dwrite d0/d4/f75 [0,4194304] 0 2024-04-24T22:45:45.070 INFO:tasks.workunit.client.0.smithi063.stdout:4/965: link d2/d8/d36/d118/f13f d2/d8/d36/d118/d136/f144 0 2024-04-24T22:45:45.070 INFO:tasks.workunit.client.1.smithi192.stdout:8/803: dwrite db/dc/f17 [0,4194304] 0 2024-04-24T22:45:45.074 INFO:tasks.workunit.client.0.smithi063.stdout:5/937: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/f6b [0,4194304] 0 2024-04-24T22:45:45.076 INFO:tasks.workunit.client.1.smithi192.stdout:9/880: write d3/dd/de/f1a [1109275,14807] 0 2024-04-24T22:45:45.081 INFO:tasks.workunit.client.1.smithi192.stdout:8/804: dwrite db/d43/d76/feb [4194304,4194304] 0 2024-04-24T22:45:45.082 INFO:tasks.workunit.client.1.smithi192.stdout:7/920: rmdir d2/db/d21 39 2024-04-24T22:45:45.084 INFO:tasks.workunit.client.0.smithi063.stdout:0/917: dwrite d6/d64/d24/d42/fbe [0,4194304] 0 2024-04-24T22:45:45.086 INFO:tasks.workunit.client.1.smithi192.stdout:9/881: mkdir d3/dd/de/da7/d11f 0 2024-04-24T22:45:45.092 INFO:tasks.workunit.client.0.smithi063.stdout:5/938: rename d5/d11/d23/daf/l11a to d5/d11/d23/d2d/d62/d81/l145 0 2024-04-24T22:45:45.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/998: dwrite d6/d2f/d42/d81/fc6 [0,4194304] 0 2024-04-24T22:45:45.096 INFO:tasks.workunit.client.1.smithi192.stdout:7/921: creat d2/db/d21/d42/d67/d9b/dc1/f125 x:0 0 0 2024-04-24T22:45:45.096 INFO:tasks.workunit.client.0.smithi063.stdout:3/985: dwrite da/d18/d47/d4d/d62/f6f [0,4194304] 0 2024-04-24T22:45:45.098 INFO:tasks.workunit.client.1.smithi192.stdout:7/922: readlink d2/db/d21/d42/d5b/d63/d80/laf 0 2024-04-24T22:45:45.103 INFO:tasks.workunit.client.0.smithi063.stdout:4/966: getdents d2/d19/d33/d9d/da8/dec/ded 0 2024-04-24T22:45:45.104 INFO:tasks.workunit.client.1.smithi192.stdout:9/882: getdents d3/dd/d53/ddd 0 2024-04-24T22:45:45.105 INFO:tasks.workunit.client.1.smithi192.stdout:7/923: dread d2/db/d36/ff0 [0,4194304] 0 2024-04-24T22:45:45.106 INFO:tasks.workunit.client.0.smithi063.stdout:9/999: mknod d6/d2f/dcd/db/d6b/c152 0 2024-04-24T22:45:45.106 INFO:tasks.workunit.client.1.smithi192.stdout:7/924: stat d2/db/d25/dda/c110 0 2024-04-24T22:45:45.108 INFO:tasks.workunit.client.0.smithi063.stdout:7/905: dwrite d0/d4/de/d17/d35/d69/db1/fca [0,4194304] 0 2024-04-24T22:45:45.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/986: rename da/d18/d47/d4d to da/d18/d3a/d147/d158 0 2024-04-24T22:45:45.112 INFO:tasks.workunit.client.0.smithi063.stdout:3/987: dread - da/d39/d94/db3/dfa/d13e/f153 zero size 2024-04-24T22:45:45.114 INFO:tasks.workunit.client.0.smithi063.stdout:5/939: symlink d5/d11/d23/l146 0 2024-04-24T22:45:45.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/883: dread d3/dd/de/f1a [0,4194304] 0 2024-04-24T22:45:45.117 INFO:tasks.workunit.client.1.smithi192.stdout:7/925: dwrite d2/db/d21/d42/d67/d9b/d6e/db5/fdf [0,4194304] 0 2024-04-24T22:45:45.118 INFO:tasks.workunit.client.0.smithi063.stdout:7/906: chown d0/d4/de/d17/d28/l2d 6838 1 2024-04-24T22:45:45.118 INFO:tasks.workunit.client.1.smithi192.stdout:7/926: readlink d2/db/d49/lad 0 2024-04-24T22:45:45.119 INFO:tasks.workunit.client.0.smithi063.stdout:0/918: getdents d6/d64/d21/d25/d31 0 2024-04-24T22:45:45.119 INFO:tasks.workunit.client.0.smithi063.stdout:7/907: chown d0/d4/de/d6f/d5c/d9c/f103 4747 1 2024-04-24T22:45:45.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/988: fsync da/d18/d9e/fc4 0 2024-04-24T22:45:45.123 INFO:tasks.workunit.client.0.smithi063.stdout:5/940: creat d5/d11/d76/d92/dea/f147 x:0 0 0 2024-04-24T22:45:45.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/927: fsync d2/db/d21/f84 0 2024-04-24T22:45:45.125 INFO:tasks.workunit.client.1.smithi192.stdout:9/884: mknod d3/dd/d62/c120 0 2024-04-24T22:45:45.126 INFO:tasks.workunit.client.1.smithi192.stdout:9/885: dread - d3/dd/de/da7/f108 zero size 2024-04-24T22:45:45.127 INFO:tasks.workunit.client.1.smithi192.stdout:9/886: dread - d3/dd/d62/f6b zero size 2024-04-24T22:45:45.130 INFO:tasks.workunit.client.0.smithi063.stdout:7/908: truncate d0/d4/de/d6f/d5c/d8b/fae 3565161 0 2024-04-24T22:45:45.132 INFO:tasks.workunit.client.1.smithi192.stdout:9/887: mknod d3/dd/df2/c121 0 2024-04-24T22:45:45.132 INFO:tasks.workunit.client.1.smithi192.stdout:9/888: stat d3/dd/dce/fec 0 2024-04-24T22:45:45.133 INFO:tasks.workunit.client.0.smithi063.stdout:3/989: creat da/d18/dfb/f159 x:0 0 0 2024-04-24T22:45:45.133 INFO:tasks.workunit.client.0.smithi063.stdout:0/919: dread d6/d74/f136 [0,4194304] 0 2024-04-24T22:45:45.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/889: truncate d3/d59/f5a 857772 0 2024-04-24T22:45:45.138 INFO:tasks.workunit.client.1.smithi192.stdout:9/890: chown d3/d59/d7d/d88/dc2 3674 1 2024-04-24T22:45:45.138 INFO:tasks.workunit.client.1.smithi192.stdout:9/891: chown d3/d59/d7d 14103 1 2024-04-24T22:45:45.139 INFO:tasks.workunit.client.1.smithi192.stdout:7/928: getdents d2/db/d21/d42/d67/dd4 0 2024-04-24T22:45:45.140 INFO:tasks.workunit.client.0.smithi063.stdout:4/967: write d2/d8/d40/d51/fc4 [526478,80307] 0 2024-04-24T22:45:45.145 INFO:tasks.workunit.client.0.smithi063.stdout:0/920: creat d6/d64/d21/d9c/f13a x:0 0 0 2024-04-24T22:45:45.146 INFO:tasks.workunit.client.1.smithi192.stdout:9/892: creat d3/dd/de/d17/d89/f122 x:0 0 0 2024-04-24T22:45:45.146 INFO:tasks.workunit.client.1.smithi192.stdout:7/929: getdents d2/db/d21/d42/d67/d9b/d102 0 2024-04-24T22:45:45.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/930: fsync d2/db/d21/d42/f112 0 2024-04-24T22:45:45.152 INFO:tasks.workunit.client.1.smithi192.stdout:7/931: chown d2/db/d21/d42/d67/d9b/l3e 1271 1 2024-04-24T22:45:45.153 INFO:tasks.workunit.client.0.smithi063.stdout:3/990: creat da/d18/d3a/d147/d158/f15a x:0 0 0 2024-04-24T22:45:45.156 INFO:tasks.workunit.client.0.smithi063.stdout:5/941: getdents d5/d11/d23/d2d/d62/d73/d95/d10a 0 2024-04-24T22:45:45.158 INFO:tasks.workunit.client.0.smithi063.stdout:0/921: mknod d6/d64/da0/d33/c13b 0 2024-04-24T22:45:45.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/932: dwrite d2/db/d21/d42/d67/dae/dce/f120 [0,4194304] 0 2024-04-24T22:45:45.159 INFO:tasks.workunit.client.0.smithi063.stdout:7/909: getdents d0/d4/de/d17/d35/d69/da0 0 2024-04-24T22:45:45.159 INFO:tasks.workunit.client.0.smithi063.stdout:7/910: readlink d0/d4/de/d17/dd7/l93 0 2024-04-24T22:45:45.163 INFO:tasks.workunit.client.1.smithi192.stdout:9/893: rmdir d3/dd/de/da7/d116/dca 0 2024-04-24T22:45:45.165 INFO:tasks.workunit.client.0.smithi063.stdout:7/911: dread - d0/d4/de/d6f/d5c/d8b/fe5 zero size 2024-04-24T22:45:45.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/933: dwrite d2/db/d21/d42/d67/d9b/d6e/d77/d8e/fa4 [0,4194304] 0 2024-04-24T22:45:45.166 INFO:tasks.workunit.client.1.smithi192.stdout:9/894: readlink d3/dd/de/da7/d116/lc8 0 2024-04-24T22:45:45.167 INFO:tasks.workunit.client.0.smithi063.stdout:3/991: mkdir da/d39/d15b 0 2024-04-24T22:45:45.169 INFO:tasks.workunit.client.0.smithi063.stdout:5/942: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/d3b/fa8 [4194304,4194304] 0 2024-04-24T22:45:45.169 INFO:tasks.workunit.client.0.smithi063.stdout:0/922: unlink d6/da/dc/f71 0 2024-04-24T22:45:45.173 INFO:tasks.workunit.client.1.smithi192.stdout:9/895: mknod d3/dd/d62/d6d/c123 0 2024-04-24T22:45:45.174 INFO:tasks.workunit.client.0.smithi063.stdout:7/912: chown d0/d4/de/d95/cf6 213032994 1 2024-04-24T22:45:45.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/992: mkdir da/db6/d15c 0 2024-04-24T22:45:45.177 INFO:tasks.workunit.client.1.smithi192.stdout:9/896: creat d3/dd/d62/d6d/d91/f124 x:0 0 0 2024-04-24T22:45:45.178 INFO:tasks.workunit.client.0.smithi063.stdout:4/968: dwrite d2/d8/f71 [0,4194304] 0 2024-04-24T22:45:45.179 INFO:tasks.workunit.client.0.smithi063.stdout:5/943: creat d5/d1d/da3/f148 x:0 0 0 2024-04-24T22:45:45.182 INFO:tasks.workunit.client.1.smithi192.stdout:9/897: creat d3/dd/df2/f125 x:0 0 0 2024-04-24T22:45:45.185 INFO:tasks.workunit.client.0.smithi063.stdout:7/913: mknod d0/d4/de/d17/d28/d31/c12e 0 2024-04-24T22:45:45.186 INFO:tasks.workunit.client.0.smithi063.stdout:5/944: symlink d5/d11/d76/l149 0 2024-04-24T22:45:45.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/923: getdents d6/d64/d21/d25/d31/da3 0 2024-04-24T22:45:45.189 INFO:tasks.workunit.client.0.smithi063.stdout:5/945: truncate d5/d11/d76/d92/dea/f147 388545 0 2024-04-24T22:45:45.210 INFO:tasks.workunit.client.1.smithi192.stdout:7/934: dwrite d2/db/d21/fc6 [4194304,4194304] 0 2024-04-24T22:45:45.225 INFO:tasks.workunit.client.0.smithi063.stdout:3/993: dwrite da/d40/f9c [0,4194304] 0 2024-04-24T22:45:45.225 INFO:tasks.workunit.client.1.smithi192.stdout:9/898: dwrite d3/dd/de/da7/dae/ff0 [0,4194304] 0 2024-04-24T22:45:45.228 INFO:tasks.workunit.client.1.smithi192.stdout:9/899: dread - d3/dd/f115 zero size 2024-04-24T22:45:45.228 INFO:tasks.workunit.client.1.smithi192.stdout:7/935: dread d2/db/d21/d42/d67/d9b/d6e/db5/db6/fa7 [0,4194304] 0 2024-04-24T22:45:45.228 INFO:tasks.workunit.client.1.smithi192.stdout:9/900: symlink d3/dd/d62/d6d/d91/l126 0 2024-04-24T22:45:45.232 INFO:tasks.workunit.client.0.smithi063.stdout:3/994: dread da/fb4 [0,4194304] 0 2024-04-24T22:45:45.236 INFO:tasks.workunit.client.0.smithi063.stdout:4/969: truncate d2/d19/d33/dc1/d50/dc3/ff9 300017 0 2024-04-24T22:45:45.237 INFO:tasks.workunit.client.0.smithi063.stdout:0/924: write d6/dd/f12 [1026065,113213] 0 2024-04-24T22:45:45.238 INFO:tasks.workunit.client.0.smithi063.stdout:7/914: write d0/d4/de/d17/d35/fd5 [599144,120711] 0 2024-04-24T22:45:45.239 INFO:tasks.workunit.client.0.smithi063.stdout:3/995: mkdir da/d18/d3a/d121/d15d 0 2024-04-24T22:45:45.241 INFO:tasks.workunit.client.1.smithi192.stdout:9/901: write d3/d5d/da1/f70 [1066918,80639] 0 2024-04-24T22:45:45.244 INFO:tasks.workunit.client.1.smithi192.stdout:7/936: read d2/db/d21/d42/d67/d9b/fcd [3561005,31792] 0 2024-04-24T22:45:45.244 INFO:tasks.workunit.client.0.smithi063.stdout:7/915: fsync d0/d4/de/d6f/d5c/d9c/fd8 0 2024-04-24T22:45:45.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/970: dwrite d2/d19/f25 [0,4194304] 0 2024-04-24T22:45:45.250 INFO:tasks.workunit.client.0.smithi063.stdout:5/946: dwrite d5/d11/d18/f3a [0,4194304] 0 2024-04-24T22:45:45.251 INFO:tasks.workunit.client.0.smithi063.stdout:0/925: rename d6/d3a/c8d to d6/d64/d24/dd1/dd5/c13c 0 2024-04-24T22:45:45.252 INFO:tasks.workunit.client.0.smithi063.stdout:3/996: getdents da/d18/d3a/d147/d158/d62 0 2024-04-24T22:45:45.255 INFO:tasks.workunit.client.0.smithi063.stdout:7/916: truncate d0/fa2 1695790 0 2024-04-24T22:45:45.256 INFO:tasks.workunit.client.0.smithi063.stdout:4/971: rename d2/d8/d40/da0/lc8 to d2/d8/dd4/dda/da5/l145 0 2024-04-24T22:45:45.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/972: readlink d2/d8/ld6 0 2024-04-24T22:45:45.260 INFO:tasks.workunit.client.0.smithi063.stdout:0/926: creat d6/d64/d21/d25/d62/d75/f13d x:0 0 0 2024-04-24T22:45:45.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/947: creat d5/d1d/f14a x:0 0 0 2024-04-24T22:45:45.262 INFO:tasks.workunit.client.0.smithi063.stdout:3/997: link da/d18/f123 da/db6/d15c/f15e 0 2024-04-24T22:45:45.264 INFO:tasks.workunit.client.0.smithi063.stdout:5/948: fdatasync d5/d11/d76/f9a 0 2024-04-24T22:45:45.265 INFO:tasks.workunit.client.0.smithi063.stdout:7/917: dwrite d0/d4/de/d6f/d5c/d9c/f103 [0,4194304] 0 2024-04-24T22:45:45.265 INFO:tasks.workunit.client.1.smithi192.stdout:9/902: write d3/dd/d62/f6b [1001366,37415] 0 2024-04-24T22:45:45.266 INFO:tasks.workunit.client.1.smithi192.stdout:9/903: readlink d3/dd/l2a 0 2024-04-24T22:45:45.266 INFO:tasks.workunit.client.1.smithi192.stdout:9/904: chown d3/d59/dbf/d106 172650 1 2024-04-24T22:45:45.267 INFO:tasks.workunit.client.0.smithi063.stdout:0/927: creat d6/d64/d24/d42/dd4/f13e x:0 0 0 2024-04-24T22:45:45.268 INFO:tasks.workunit.client.1.smithi192.stdout:7/937: write d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb1 [758836,57823] 0 2024-04-24T22:45:45.268 INFO:tasks.workunit.client.0.smithi063.stdout:5/949: truncate d5/d11/d23/d2d/ff9 1321255 0 2024-04-24T22:45:45.269 INFO:tasks.workunit.client.0.smithi063.stdout:3/998: mkdir da/db6/d15f 0 2024-04-24T22:45:45.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/905: creat d3/d5d/da1/d5e/f127 x:0 0 0 2024-04-24T22:45:45.271 INFO:tasks.workunit.client.0.smithi063.stdout:4/973: mknod d2/d8/dee/d13a/c146 0 2024-04-24T22:45:45.271 INFO:tasks.workunit.client.0.smithi063.stdout:3/999: read - da/d40/f128 zero size 2024-04-24T22:45:45.272 INFO:tasks.workunit.client.1.smithi192.stdout:9/906: fsync d3/dd/d53/ddd/ffd 0 2024-04-24T22:45:45.273 INFO:tasks.workunit.client.1.smithi192.stdout:7/938: fdatasync d2/db/f6f 0 2024-04-24T22:45:45.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/950: write d5/d11/d76/d92/dea/f135 [496723,31743] 0 2024-04-24T22:45:45.275 INFO:tasks.workunit.client.0.smithi063.stdout:0/928: dread d6/d64/d24/f73 [0,4194304] 0 2024-04-24T22:45:45.276 INFO:tasks.workunit.client.1.smithi192.stdout:7/939: creat d2/db/d49/d115/f126 x:0 0 0 2024-04-24T22:45:45.278 INFO:tasks.workunit.client.1.smithi192.stdout:7/940: dread - d2/db/d25/dda/f124 zero size 2024-04-24T22:45:45.283 INFO:tasks.workunit.client.0.smithi063.stdout:0/929: truncate d6/d64/da0/d33/fc3 4520279 0 2024-04-24T22:45:45.284 INFO:tasks.workunit.client.0.smithi063.stdout:5/951: mknod d5/d11/d23/d120/c14b 0 2024-04-24T22:45:45.289 INFO:tasks.workunit.client.0.smithi063.stdout:5/952: truncate d5/d11/d23/d2d/d62/d73/d118/d36/d131/f130 843588 0 2024-04-24T22:45:45.290 INFO:tasks.workunit.client.1.smithi192.stdout:7/941: getdents d2/db/d21/d42/d67/d9b 0 2024-04-24T22:45:45.291 INFO:tasks.workunit.client.0.smithi063.stdout:0/930: dwrite d6/d64/d24/d42/dd4/f13e [0,4194304] 0 2024-04-24T22:45:45.295 INFO:tasks.workunit.client.0.smithi063.stdout:7/918: dread d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:45.296 INFO:tasks.workunit.client.0.smithi063.stdout:5/953: symlink d5/d11/d18/d136/l14c 0 2024-04-24T22:45:45.297 INFO:tasks.workunit.client.0.smithi063.stdout:7/919: chown d0/d4/de/d17/d28/d31/d94/f11f 44 1 2024-04-24T22:45:45.301 INFO:tasks.workunit.client.1.smithi192.stdout:9/907: dwrite d3/d59/d7d/d88/dc2/fd8 [0,4194304] 0 2024-04-24T22:45:45.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/920: chown d0/d4/de/d6f/d2e/d55/d106/d10e/c116 6508 1 2024-04-24T22:45:45.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/974: dwrite d2/d8/dd4/dda/dc6/ff4 [0,4194304] 0 2024-04-24T22:45:45.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/954: dwrite d5/d11/d23/d2d/d62/d81/d10c/f127 [0,4194304] 0 2024-04-24T22:45:45.309 INFO:tasks.workunit.client.1.smithi192.stdout:7/942: dread d2/db/d25/f69 [0,4194304] 0 2024-04-24T22:45:45.309 INFO:tasks.workunit.client.1.smithi192.stdout:9/908: creat d3/dd/de/da7/d116/ddb/f128 x:0 0 0 2024-04-24T22:45:45.310 INFO:tasks.workunit.client.1.smithi192.stdout:7/943: chown d2/db/lc4 4622 1 2024-04-24T22:45:45.311 INFO:tasks.workunit.client.1.smithi192.stdout:7/944: stat d2/db/d21/f84 0 2024-04-24T22:45:45.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/921: rename d0/d4/de/d17/d28/l2d to d0/db5/l12f 0 2024-04-24T22:45:45.316 INFO:tasks.workunit.client.0.smithi063.stdout:0/931: write d6/da/d2b/d60/f128 [738131,3611] 0 2024-04-24T22:45:45.316 INFO:tasks.workunit.client.0.smithi063.stdout:4/975: rmdir d2/d19/d82/d96 39 2024-04-24T22:45:45.316 INFO:tasks.workunit.client.1.smithi192.stdout:7/945: creat d2/db/d2a/f127 x:0 0 0 2024-04-24T22:45:45.317 INFO:tasks.workunit.client.1.smithi192.stdout:9/909: mknod d3/dd/de/d17/c129 0 2024-04-24T22:45:45.318 INFO:tasks.workunit.client.0.smithi063.stdout:4/976: dread - d2/d19/d33/dc1/fd2 zero size 2024-04-24T22:45:45.318 INFO:tasks.workunit.client.0.smithi063.stdout:7/922: chown d0/d4/de/d6f/d5d/f121 1512 1 2024-04-24T22:45:45.319 INFO:tasks.workunit.client.0.smithi063.stdout:7/923: readlink d0/l41 0 2024-04-24T22:45:45.321 INFO:tasks.workunit.client.1.smithi192.stdout:7/946: mknod d2/db/d21/d42/d67/d9b/d6e/db5/c128 0 2024-04-24T22:45:45.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/910: write d3/d59/f5b [1284942,54998] 0 2024-04-24T22:45:45.344 INFO:tasks.workunit.client.1.smithi192.stdout:9/911: rename d3/dd/de/f55 to d3/dd/de/dba/f12a 0 2024-04-24T22:45:45.345 INFO:tasks.workunit.client.1.smithi192.stdout:7/947: write d2/db/d25/f7d [387162,49186] 0 2024-04-24T22:45:45.348 INFO:tasks.workunit.client.1.smithi192.stdout:9/912: rmdir d3/dd/de/da7 39 2024-04-24T22:45:45.352 INFO:tasks.workunit.client.0.smithi063.stdout:4/977: mkdir d2/d8/d3c/d12b/d147 0 2024-04-24T22:45:45.382 INFO:tasks.workunit.client.0.smithi063.stdout:0/932: symlink d6/d64/d21/d25/l13f 0 2024-04-24T22:45:45.383 INFO:tasks.workunit.client.0.smithi063.stdout:7/924: rename d0/fa2 to d0/d4/de/d17/d35/d69/da0/f130 0 2024-04-24T22:45:45.384 INFO:tasks.workunit.client.0.smithi063.stdout:5/955: getdents d5/d11/d76 0 2024-04-24T22:45:45.386 INFO:tasks.workunit.client.0.smithi063.stdout:5/956: chown d5/d11/d23/d2d/d62/d73/d118/l88 2 1 2024-04-24T22:45:45.386 INFO:tasks.workunit.client.0.smithi063.stdout:0/933: dread d6/d64/da0/f84 [0,4194304] 0 2024-04-24T22:45:45.387 INFO:tasks.workunit.client.0.smithi063.stdout:4/978: link d2/fb d2/d19/dce/f148 0 2024-04-24T22:45:45.390 INFO:tasks.workunit.client.0.smithi063.stdout:0/934: write d6/d64/d21/d25/d62/d63/d89/d95/fdc [1937998,105100] 0 2024-04-24T22:45:45.390 INFO:tasks.workunit.client.0.smithi063.stdout:4/979: creat d2/d19/d82/d96/f149 x:0 0 0 2024-04-24T22:45:45.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/980: mknod d2/d9f/d117/c14a 0 2024-04-24T22:45:45.394 INFO:tasks.workunit.client.0.smithi063.stdout:7/925: dread d0/d2f/f5a [0,4194304] 0 2024-04-24T22:45:45.397 INFO:tasks.workunit.client.0.smithi063.stdout:4/981: truncate d2/d9f/d117/d10a/f10d 22894 0 2024-04-24T22:45:45.398 INFO:tasks.workunit.client.0.smithi063.stdout:5/957: dread d5/d11/d23/d2d/d62/d73/d118/d36/d131/fa5 [0,4194304] 0 2024-04-24T22:45:45.398 INFO:tasks.workunit.client.1.smithi192.stdout:7/948: write d2/db/d21/d42/f93 [287611,32102] 0 2024-04-24T22:45:45.401 INFO:tasks.workunit.client.0.smithi063.stdout:4/982: unlink d2/d8/d40/c49 0 2024-04-24T22:45:45.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/926: dwrite d0/d4/de/d6f/f47 [4194304,4194304] 0 2024-04-24T22:45:45.401 INFO:tasks.workunit.client.0.smithi063.stdout:5/958: write d5/d11/d23/d2d/d62/d81/da4/dee/f128 [89738,10292] 0 2024-04-24T22:45:45.402 INFO:tasks.workunit.client.1.smithi192.stdout:7/949: rename d2/db/d21/d42/d5b/d63/l8f to d2/db/d21/d42/d67/d9b/dc1/l129 0 2024-04-24T22:45:45.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/927: dread d0/d4/de/d17/dd7/f84 [0,4194304] 0 2024-04-24T22:45:45.409 INFO:tasks.workunit.client.1.smithi192.stdout:7/950: dwrite d2/db/d21/d42/f112 [0,4194304] 0 2024-04-24T22:45:45.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/928: stat d0/d4/de/d17/d28/d31/lcc 0 2024-04-24T22:45:45.410 INFO:tasks.workunit.client.0.smithi063.stdout:7/929: write d0/d4/f11 [3934770,101185] 0 2024-04-24T22:45:45.411 INFO:tasks.workunit.client.0.smithi063.stdout:4/983: getdents d2/d8/dd4 0 2024-04-24T22:45:45.414 INFO:tasks.workunit.client.1.smithi192.stdout:7/951: unlink d2/db/d21/d42/d67/f7e 0 2024-04-24T22:45:45.415 INFO:tasks.workunit.client.1.smithi192.stdout:7/952: truncate d2/f117 149693 0 2024-04-24T22:45:45.416 INFO:tasks.workunit.client.1.smithi192.stdout:7/953: write d2/db/d49/f70 [6340810,29109] 0 2024-04-24T22:45:45.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/930: creat d0/db5/dbb/f131 x:0 0 0 2024-04-24T22:45:45.422 INFO:tasks.workunit.client.1.smithi192.stdout:7/954: mkdir d2/db/d49/d115/d12a 0 2024-04-24T22:45:45.423 INFO:tasks.workunit.client.1.smithi192.stdout:7/955: chown d2/db/d21/d42/d67/d9b/d6e/db5/db6/fb1 187 1 2024-04-24T22:45:45.423 INFO:tasks.workunit.client.1.smithi192.stdout:7/956: chown d2/lc0 59547 1 2024-04-24T22:45:45.424 INFO:tasks.workunit.client.0.smithi063.stdout:4/984: getdents d2/d8/d36/dd9 0 2024-04-24T22:45:45.426 INFO:tasks.workunit.client.0.smithi063.stdout:7/931: rename d0/d4/de/d17/d35/de4/l114 to d0/d4/de/dfc/dfe/l132 0 2024-04-24T22:45:45.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:44 smithi192 ceph-mon[33991]: pgmap v20: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 189 MiB/s wr, 430 op/s 2024-04-24T22:45:45.427 INFO:tasks.workunit.client.0.smithi063.stdout:7/932: stat d0/d4/de/d17/f8a 0 2024-04-24T22:45:45.431 INFO:tasks.workunit.client.0.smithi063.stdout:5/959: write d5/d11/d23/d2d/d62/d73/d118/f51 [1112644,9428] 0 2024-04-24T22:45:45.432 INFO:tasks.workunit.client.0.smithi063.stdout:5/960: chown d5/de5/f12b 48007136 1 2024-04-24T22:45:45.439 INFO:tasks.workunit.client.0.smithi063.stdout:5/961: write d5/d11/d23/d2d/d62/d81/da4/fd4 [594820,99050] 0 2024-04-24T22:45:45.444 INFO:tasks.workunit.client.0.smithi063.stdout:7/933: dwrite d0/d4/de/d17/d35/d69/db1/fc1 [0,4194304] 0 2024-04-24T22:45:45.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/957: dread d2/db/d49/f70 [4194304,4194304] 0 2024-04-24T22:45:45.444 INFO:tasks.workunit.client.0.smithi063.stdout:4/985: dread d2/f74 [0,4194304] 0 2024-04-24T22:45:45.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/986: stat d2/c2d 0 2024-04-24T22:45:45.446 INFO:tasks.workunit.client.1.smithi192.stdout:7/958: read d2/db/d21/f39 [3898325,51308] 0 2024-04-24T22:45:45.446 INFO:tasks.workunit.client.1.smithi192.stdout:7/959: readlink d2/db/d21/d42/d67/le5 0 2024-04-24T22:45:45.447 INFO:tasks.workunit.client.0.smithi063.stdout:5/962: link d5/d11/d23/d2d/d62/d73/d95/fac d5/d11/f14d 0 2024-04-24T22:45:45.451 INFO:tasks.workunit.client.0.smithi063.stdout:4/987: unlink d2/c78 0 2024-04-24T22:45:45.451 INFO:tasks.workunit.client.0.smithi063.stdout:7/934: link d0/d4/de/d6f/d2e/cb4 d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/c133 0 2024-04-24T22:45:45.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/988: readlink d2/d19/dce/lcf 0 2024-04-24T22:45:45.453 INFO:tasks.workunit.client.1.smithi192.stdout:7/960: link d2/db/d21/d42/d5b/ca5 d2/db/d21/d42/d67/da8/dc7/dec/c12b 0 2024-04-24T22:45:45.454 INFO:tasks.workunit.client.0.smithi063.stdout:4/989: readlink d2/d8/l39 0 2024-04-24T22:45:45.454 INFO:tasks.workunit.client.0.smithi063.stdout:5/963: dread d5/ff [0,4194304] 0 2024-04-24T22:45:45.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/935: mknod d0/d4/de/dfc/c134 0 2024-04-24T22:45:45.455 INFO:tasks.workunit.client.0.smithi063.stdout:4/990: creat d2/d9f/d117/f14b x:0 0 0 2024-04-24T22:45:45.456 INFO:tasks.workunit.client.1.smithi192.stdout:7/961: symlink d2/db/d21/dd5/l12c 0 2024-04-24T22:45:45.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/962: rmdir d2/db/d21/d42/d67/dae/dce 39 2024-04-24T22:45:45.459 INFO:tasks.workunit.client.1.smithi192.stdout:7/963: mknod d2/db/d25/dda/c12d 0 2024-04-24T22:45:45.459 INFO:tasks.workunit.client.0.smithi063.stdout:7/936: link d0/d4/de/d6f/d2e/l66 d0/d4/de/d6f/d5d/l135 0 2024-04-24T22:45:45.460 INFO:tasks.workunit.client.1.smithi192.stdout:7/964: chown d2/db/d25/c44 115721 1 2024-04-24T22:45:45.462 INFO:tasks.workunit.client.0.smithi063.stdout:4/991: dwrite d2/d8/d36/dd9/f13b [0,4194304] 0 2024-04-24T22:45:45.462 INFO:tasks.workunit.client.1.smithi192.stdout:7/965: rmdir d2/db/d21/d42/d67/da8/dc7/dfe 39 2024-04-24T22:45:45.463 INFO:tasks.workunit.client.1.smithi192.stdout:7/966: truncate d2/db/d21/d42/d5b/d63/f10b 1044748 0 2024-04-24T22:45:45.465 INFO:tasks.workunit.client.0.smithi063.stdout:4/992: mknod d2/d8/dd4/dda/dc6/d12f/c14c 0 2024-04-24T22:45:45.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/967: getdents d2/db/d21 0 2024-04-24T22:45:45.473 INFO:tasks.workunit.client.1.smithi192.stdout:7/968: link d2/db/d21/d42/d67/d9b/l68 d2/db/d21/d42/d67/da8/l12e 0 2024-04-24T22:45:45.474 INFO:tasks.workunit.client.1.smithi192.stdout:7/969: truncate d2/db/d49/f6c 681770 0 2024-04-24T22:45:45.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/970: dwrite d2/db/d21/d42/d67/d9b/d6e/f9e [0,4194304] 0 2024-04-24T22:45:45.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/964: dread d5/d11/d23/d2d/d62/d73/d118/d36/d131/f46 [0,4194304] 0 2024-04-24T22:45:45.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/965: dread d5/d11/d23/d2d/d62/d81/da4/dee/f128 [0,4194304] 0 2024-04-24T22:45:45.486 INFO:tasks.workunit.client.1.smithi192.stdout:7/971: dwrite d2/db/d21/fc6 [4194304,4194304] 0 2024-04-24T22:45:45.487 INFO:tasks.workunit.client.0.smithi063.stdout:4/993: dwrite d2/d19/d33/dc1/d50/f5d [0,4194304] 0 2024-04-24T22:45:45.488 INFO:tasks.workunit.client.0.smithi063.stdout:5/966: write d5/d11/d23/d2d/d62/d81/f11e [696777,36294] 0 2024-04-24T22:45:45.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/967: chown d5/d11/dec 14656 1 2024-04-24T22:45:45.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/937: dread d0/d4/de/d17/d35/d69/fb0 [0,4194304] 0 2024-04-24T22:45:45.492 INFO:tasks.workunit.client.0.smithi063.stdout:4/994: unlink d2/d8/dd4/c10f 0 2024-04-24T22:45:45.493 INFO:tasks.workunit.client.0.smithi063.stdout:5/968: mknod d5/d11/d76/d92/dea/c14e 0 2024-04-24T22:45:45.494 INFO:tasks.workunit.client.1.smithi192.stdout:7/972: dwrite d2/db/d21/d42/d67/dd4/f10d [0,4194304] 0 2024-04-24T22:45:45.499 INFO:tasks.workunit.client.1.smithi192.stdout:7/973: chown d2/db/d21/d42/d67/d9b/d6e/db5 1800782 1 2024-04-24T22:45:45.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/938: mknod d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/d12c/c136 0 2024-04-24T22:45:45.501 INFO:tasks.workunit.client.0.smithi063.stdout:5/969: mkdir d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/d10e/d14f 0 2024-04-24T22:45:45.503 INFO:tasks.workunit.client.1.smithi192.stdout:7/974: unlink d2/db/f10 0 2024-04-24T22:45:45.504 INFO:tasks.workunit.client.1.smithi192.stdout:7/975: readlink d2/db/d25/l3b 0 2024-04-24T22:45:45.505 INFO:tasks.workunit.client.0.smithi063.stdout:7/939: symlink d0/d4/de/d6f/d2e/d55/ddc/l137 0 2024-04-24T22:45:45.507 INFO:tasks.workunit.client.0.smithi063.stdout:5/970: creat d5/d121/f150 x:0 0 0 2024-04-24T22:45:45.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/976: dread - d2/db/d21/d42/d67/da8/dc7/dfe/f11d zero size 2024-04-24T22:45:45.519 INFO:tasks.workunit.client.0.smithi063.stdout:5/971: mknod d5/d11/d23/d2d/d62/d73/d95/d10a/c151 0 2024-04-24T22:45:45.523 INFO:tasks.workunit.client.0.smithi063.stdout:5/972: write d5/f87 [1876777,81518] 0 2024-04-24T22:45:45.524 INFO:tasks.workunit.client.0.smithi063.stdout:5/973: fsync d5/d11/d23/d2d/d62/d73/d118/db6/f111 0 2024-04-24T22:45:45.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/977: dwrite d2/db/d25/dda/fdb [0,4194304] 0 2024-04-24T22:45:45.527 INFO:tasks.workunit.client.0.smithi063.stdout:5/974: mknod d5/d11/d76/d92/d13a/c152 0 2024-04-24T22:45:45.527 INFO:tasks.workunit.client.0.smithi063.stdout:4/995: dread d2/d19/dce/f148 [0,4194304] 0 2024-04-24T22:45:45.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/978: chown d2/db/d21/d42/d67/d9b/d6e/db5/db6/fff 117486844 1 2024-04-24T22:45:45.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/979: fdatasync d2/db/d49/d115/f11c 0 2024-04-24T22:45:45.529 INFO:tasks.workunit.client.1.smithi192.stdout:7/980: chown d2/db/d49/dca/d104/c11a 0 1 2024-04-24T22:45:45.531 INFO:tasks.workunit.client.0.smithi063.stdout:4/996: symlink d2/d8/d3c/l14d 0 2024-04-24T22:45:45.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/975: link d5/d11/d23/d2d/l38 d5/d11/d23/d2d/d62/d73/d118/d49/d3b/l153 0 2024-04-24T22:45:45.535 INFO:tasks.workunit.client.0.smithi063.stdout:5/976: rename d5/d11/d76/d92/dea to d5/d11/d76/d92/dea/d154 22 2024-04-24T22:45:45.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/997: mkdir d2/d8/dd4/dda/d14e 0 2024-04-24T22:45:45.538 INFO:tasks.workunit.client.0.smithi063.stdout:5/977: symlink d5/d11/d76/d92/d13a/l155 0 2024-04-24T22:45:45.538 INFO:tasks.workunit.client.0.smithi063.stdout:4/998: dread - d2/d19/d33/dc1/d50/feb zero size 2024-04-24T22:45:45.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/978: rmdir d5/d1d/dc0 39 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:4/767: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:3/818: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:6/729: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:0/817: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/669: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:8/805: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:2/915: sync 2024-04-24T22:45:45.541 INFO:tasks.workunit.client.1.smithi192.stdout:5/680: sync 2024-04-24T22:45:45.542 INFO:tasks.workunit.client.0.smithi063.stdout:4/999: creat d2/d19/d33/dc1/d50/f14f x:0 0 0 2024-04-24T22:45:45.543 INFO:tasks.workunit.client.1.smithi192.stdout:3/819: chown d2/d4/d12/dbc/cec 79 1 2024-04-24T22:45:45.544 INFO:tasks.workunit.client.1.smithi192.stdout:6/730: dread - d0/d8/d29/d4c/d4d/f72 zero size 2024-04-24T22:45:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:5/979: rename d5/d11/d18/f19 to d5/d11/d18/f156 0 2024-04-24T22:45:45.547 INFO:tasks.workunit.client.1.smithi192.stdout:4/768: dread - d2/f76 zero size 2024-04-24T22:45:45.548 INFO:tasks.workunit.client.1.smithi192.stdout:6/731: chown d0/d8/d29/d4c/db0/fb5 55513 1 2024-04-24T22:45:45.549 INFO:tasks.workunit.client.0.smithi063.stdout:5/980: dread d5/d11/d23/d2d/d62/d81/f11e [0,4194304] 0 2024-04-24T22:45:45.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/681: unlink d0/d13/d19/d64/d4f/d52/f7e 0 2024-04-24T22:45:45.551 INFO:tasks.workunit.client.1.smithi192.stdout:2/916: dwrite d4/d9/f92 [0,4194304] 0 2024-04-24T22:45:45.552 INFO:tasks.workunit.client.0.smithi063.stdout:5/981: fdatasync d5/d11/d23/d2d/d62/d81/da4/da6/fc6 0 2024-04-24T22:45:45.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/818: dwrite d3/d27/d28/dcc/d5c/f3d [0,4194304] 0 2024-04-24T22:45:45.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/982: read d5/d11/d23/d2d/d62/d73/d118/d36/d65/fde [328025,108326] 0 2024-04-24T22:45:45.556 INFO:tasks.workunit.client.1.smithi192.stdout:3/820: mkdir d2/d106 0 2024-04-24T22:45:45.563 INFO:tasks.workunit.client.1.smithi192.stdout:5/682: dwrite d0/fc5 [0,4194304] 0 2024-04-24T22:45:45.564 INFO:tasks.workunit.client.0.smithi063.stdout:5/983: mknod d5/d11/d23/d2d/c157 0 2024-04-24T22:45:45.565 INFO:tasks.workunit.client.1.smithi192.stdout:5/683: fdatasync d0/d13/d19/d83/dad/fd8 0 2024-04-24T22:45:45.567 INFO:tasks.workunit.client.1.smithi192.stdout:1/670: rename d9/d15/c3d to d9/d15/d5e/d90/da9/ddd/ce2 0 2024-04-24T22:45:45.572 INFO:tasks.workunit.client.1.smithi192.stdout:2/917: creat d4/d29/d60/d9a/db3/f129 x:0 0 0 2024-04-24T22:45:45.576 INFO:tasks.workunit.client.1.smithi192.stdout:2/918: read d4/f27 [4035312,16114] 0 2024-04-24T22:45:45.576 INFO:tasks.workunit.client.1.smithi192.stdout:5/684: creat d0/d13/d19/d83/dad/fe9 x:0 0 0 2024-04-24T22:45:45.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/919: truncate d4/d9/d37/f3b 3832562 0 2024-04-24T22:45:45.578 INFO:tasks.workunit.client.1.smithi192.stdout:4/769: rename d2/d9/d6b/l9a to d2/db/d75/da1/d36/l10c 0 2024-04-24T22:45:45.579 INFO:tasks.workunit.client.1.smithi192.stdout:1/671: mknod d9/d15/d5e/d90/ce3 0 2024-04-24T22:45:45.580 INFO:tasks.workunit.client.1.smithi192.stdout:8/806: link db/d43/d76/d92/dad/cbe db/dc/d22/da3/cfa 0 2024-04-24T22:45:45.580 INFO:tasks.workunit.client.1.smithi192.stdout:1/672: fsync d9/d15/d5e/fdf 0 2024-04-24T22:45:45.582 INFO:tasks.workunit.client.1.smithi192.stdout:3/821: truncate d2/fa2 612226 0 2024-04-24T22:45:45.584 INFO:tasks.workunit.client.1.smithi192.stdout:5/685: creat d0/d13/d19/d71/d7f/d9a/fea x:0 0 0 2024-04-24T22:45:45.584 INFO:tasks.workunit.client.1.smithi192.stdout:0/819: rename d3 to d3/d110 22 2024-04-24T22:45:45.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/732: getdents d0/d8/d3d/d9d/da5/dc8/ddb 0 2024-04-24T22:45:45.605 INFO:tasks.workunit.client.0.smithi063.stdout:0/935: sync 2024-04-24T22:45:45.606 INFO:tasks.workunit.client.0.smithi063.stdout:0/936: dread - d6/d64/da0/f9b zero size 2024-04-24T22:45:45.610 INFO:tasks.workunit.client.0.smithi063.stdout:0/937: write d6/da/f10 [1205401,37909] 0 2024-04-24T22:45:45.613 INFO:tasks.workunit.client.0.smithi063.stdout:0/938: symlink d6/d64/da0/d98/dc7/l140 0 2024-04-24T22:45:45.615 INFO:tasks.workunit.client.0.smithi063.stdout:0/939: creat d6/d64/da0/d33/f141 x:0 0 0 2024-04-24T22:45:45.615 INFO:tasks.workunit.client.0.smithi063.stdout:0/940: dread - d6/d64/da0/ff1 zero size 2024-04-24T22:45:45.617 INFO:tasks.workunit.client.0.smithi063.stdout:0/941: creat d6/d64/d21/d80/dea/f142 x:0 0 0 2024-04-24T22:45:45.620 INFO:tasks.workunit.client.0.smithi063.stdout:0/942: fsync d6/d64/d21/d80/dea/f142 0 2024-04-24T22:45:45.622 INFO:tasks.workunit.client.0.smithi063.stdout:0/943: write d6/d64/d21/d25/d62/d63/d89/d116/f125 [855819,23370] 0 2024-04-24T22:45:45.623 INFO:tasks.workunit.client.0.smithi063.stdout:0/944: write d6/d64/d21/d25/d62/d63/f82 [4613077,104336] 0 2024-04-24T22:45:45.627 INFO:tasks.workunit.client.0.smithi063.stdout:0/945: getdents d6/d64/d24/d42 0 2024-04-24T22:45:45.628 INFO:tasks.workunit.client.0.smithi063.stdout:0/946: symlink d6/d64/da0/d98/d11b/l143 0 2024-04-24T22:45:45.629 INFO:tasks.workunit.client.0.smithi063.stdout:0/947: fdatasync d6/d64/d21/d25/d62/fb7 0 2024-04-24T22:45:45.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/948: write d6/d64/d24/d42/dd4/f96 [931693,73361] 0 2024-04-24T22:45:45.632 INFO:tasks.workunit.client.0.smithi063.stdout:0/949: chown d6/d64/d21/d25/d62/d63/d89/d95/fe8 117570625 1 2024-04-24T22:45:45.635 INFO:tasks.workunit.client.0.smithi063.stdout:0/950: link d6/d64/d21/d25/d31/da3/lc6 d6/d64/da0/d98/l144 0 2024-04-24T22:45:45.636 INFO:tasks.workunit.client.0.smithi063.stdout:0/951: read - d6/d64/d24/d42/dd2/ff5 zero size 2024-04-24T22:45:45.638 INFO:tasks.workunit.client.0.smithi063.stdout:0/952: fdatasync d6/d3a/fee 0 2024-04-24T22:45:45.641 INFO:tasks.workunit.client.0.smithi063.stdout:0/953: read d6/da/f14 [1877425,85633] 0 2024-04-24T22:45:45.676 INFO:tasks.workunit.client.0.smithi063.stdout:7/940: sync 2024-04-24T22:45:45.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/941: rename d0/d4/de/dee to d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/d12c/d138 0 2024-04-24T22:45:45.680 INFO:tasks.workunit.client.0.smithi063.stdout:7/942: creat d0/d4/de/d95/f139 x:0 0 0 2024-04-24T22:45:45.689 INFO:tasks.workunit.client.0.smithi063.stdout:7/943: dwrite d0/d4/de/d17/f8a [0,4194304] 0 2024-04-24T22:45:45.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/944: dwrite d0/d4/de/d17/d35/d69/db1/fc1 [0,4194304] 0 2024-04-24T22:45:45.699 INFO:tasks.workunit.client.0.smithi063.stdout:7/945: symlink d0/d4/de/d17/d35/l13a 0 2024-04-24T22:45:45.702 INFO:tasks.workunit.client.1.smithi192.stdout:2/920: rename d4/fbd to d4/d9/d10/d6c/d95/db7/dbf/f12a 0 2024-04-24T22:45:45.702 INFO:tasks.workunit.client.1.smithi192.stdout:0/820: mkdir d3/d27/d28/dcc/da8/d111 0 2024-04-24T22:45:45.704 INFO:tasks.workunit.client.1.smithi192.stdout:3/822: rmdir d2/d4/d12/d1a/d1d/d55 39 2024-04-24T22:45:45.704 INFO:tasks.workunit.client.0.smithi063.stdout:7/946: symlink d0/d4/de/d17/d35/d6c/d8e/l13b 0 2024-04-24T22:45:45.705 INFO:tasks.workunit.client.1.smithi192.stdout:8/807: rmdir db/d5d/d74/d96/df5 0 2024-04-24T22:45:45.706 INFO:tasks.workunit.client.1.smithi192.stdout:2/921: rmdir d4/d9/d10/d6c 39 2024-04-24T22:45:45.709 INFO:tasks.workunit.client.1.smithi192.stdout:0/821: fdatasync d3/d27/d28/f17 0 2024-04-24T22:45:45.711 INFO:tasks.workunit.client.0.smithi063.stdout:0/954: sync 2024-04-24T22:45:45.711 INFO:tasks.workunit.client.1.smithi192.stdout:6/733: dread d0/d8/d22/fda [0,4194304] 0 2024-04-24T22:45:45.712 INFO:tasks.workunit.client.1.smithi192.stdout:2/922: mkdir d4/d9/d37/de5/d12b 0 2024-04-24T22:45:45.712 INFO:tasks.workunit.client.1.smithi192.stdout:6/734: chown d0/d8/f76 27440 1 2024-04-24T22:45:45.713 INFO:tasks.workunit.client.1.smithi192.stdout:6/735: readlink d0/d8/d29/d4c/d4d/lbc 0 2024-04-24T22:45:45.713 INFO:tasks.workunit.client.1.smithi192.stdout:5/686: mknod d0/d1b/d50/ceb 0 2024-04-24T22:45:45.714 INFO:tasks.workunit.client.0.smithi063.stdout:0/955: write d6/d64/d24/f73 [481080,20118] 0 2024-04-24T22:45:45.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/822: readlink d3/d27/d3c/d6c/l70 0 2024-04-24T22:45:45.716 INFO:tasks.workunit.client.0.smithi063.stdout:0/956: write d6/d64/d21/d25/d62/d63/d89/d95/da4/f8f [1198668,110931] 0 2024-04-24T22:45:45.716 INFO:tasks.workunit.client.0.smithi063.stdout:0/957: stat d6/d64/d24/dd1/c101 0 2024-04-24T22:45:45.717 INFO:tasks.workunit.client.1.smithi192.stdout:0/823: write d3/d27/d28/dcc/d5c/d80/d94/fbe [3533025,42932] 0 2024-04-24T22:45:45.717 INFO:tasks.workunit.client.1.smithi192.stdout:2/923: fsync d4/d9/f4c 0 2024-04-24T22:45:45.718 INFO:tasks.workunit.client.1.smithi192.stdout:6/736: symlink d0/d8/d18/d2e/l102 0 2024-04-24T22:45:45.723 INFO:tasks.workunit.client.1.smithi192.stdout:5/687: creat d0/d13/d19/d71/dbf/fec x:0 0 0 2024-04-24T22:45:45.725 INFO:tasks.workunit.client.0.smithi063.stdout:5/984: sync 2024-04-24T22:45:45.725 INFO:tasks.workunit.client.1.smithi192.stdout:6/737: dread d0/f7f [0,4194304] 0 2024-04-24T22:45:45.725 INFO:tasks.workunit.client.1.smithi192.stdout:2/924: mkdir d4/d9/d10/d6c/d8c/da5/de2/d12c 0 2024-04-24T22:45:45.726 INFO:tasks.workunit.client.1.smithi192.stdout:3/823: dwrite d2/d4/d12/d1a/d1d/d55/d95/f92 [0,4194304] 0 2024-04-24T22:45:45.726 INFO:tasks.workunit.client.1.smithi192.stdout:5/688: mknod d0/d13/dd1/ced 0 2024-04-24T22:45:45.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/824: rmdir d2/d4/d12/d1a/d1d/d7b/ddb 39 2024-04-24T22:45:45.734 INFO:tasks.workunit.client.0.smithi063.stdout:5/985: dread d5/d11/d23/d2d/d62/d73/f74 [0,4194304] 0 2024-04-24T22:45:45.735 INFO:tasks.workunit.client.0.smithi063.stdout:5/986: stat d5/d11/d18/c7c 0 2024-04-24T22:45:45.736 INFO:tasks.workunit.client.1.smithi192.stdout:8/808: write db/dc/f5b [4513944,2085] 0 2024-04-24T22:45:45.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/809: dread - db/d5d/d74/fea zero size 2024-04-24T22:45:45.739 INFO:tasks.workunit.client.1.smithi192.stdout:3/825: chown d2/d27/dc6/d4f/f7a 6369 1 2024-04-24T22:45:45.739 INFO:tasks.workunit.client.1.smithi192.stdout:5/689: dwrite d0/d13/d3b/d79/fdf [0,4194304] 0 2024-04-24T22:45:45.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/947: dwrite d0/d4/de/d17/d28/f4a [0,4194304] 0 2024-04-24T22:45:45.742 INFO:tasks.workunit.client.0.smithi063.stdout:7/948: dread - d0/d4/de/d17/d28/d31/d94/f11f zero size 2024-04-24T22:45:45.743 INFO:tasks.workunit.client.1.smithi192.stdout:2/925: dread d4/d9/d10/d6c/d95/dc0/fd8 [0,4194304] 0 2024-04-24T22:45:45.743 INFO:tasks.workunit.client.1.smithi192.stdout:2/926: chown d4/d9/d10/d102 0 1 2024-04-24T22:45:45.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/810: unlink db/dc/d22/d31/d33/d81/faf 0 2024-04-24T22:45:45.745 INFO:tasks.workunit.client.0.smithi063.stdout:5/987: creat d5/d11/d23/d2d/d62/d73/d118/f158 x:0 0 0 2024-04-24T22:45:45.746 INFO:tasks.workunit.client.1.smithi192.stdout:0/824: write d3/f1c [1747935,68186] 0 2024-04-24T22:45:45.747 INFO:tasks.workunit.client.0.smithi063.stdout:0/958: dwrite d6/da/d2b/d60/ffa [0,4194304] 0 2024-04-24T22:45:45.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/959: chown d6/d64/d24/l46 89 1 2024-04-24T22:45:45.750 INFO:tasks.workunit.client.0.smithi063.stdout:7/949: mknod d0/d4/de/d6f/c13c 0 2024-04-24T22:45:45.751 INFO:tasks.workunit.client.1.smithi192.stdout:5/690: symlink d0/lee 0 2024-04-24T22:45:45.752 INFO:tasks.workunit.client.0.smithi063.stdout:5/988: read d5/d11/d23/d2d/d62/d73/d118/d49/fdf [251530,88964] 0 2024-04-24T22:45:45.752 INFO:tasks.workunit.client.1.smithi192.stdout:5/691: write d0/d13/d19/da9/fba [806297,9784] 0 2024-04-24T22:45:45.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/738: write d0/d8/d29/d4c/fc1 [215279,88116] 0 2024-04-24T22:45:45.754 INFO:tasks.workunit.client.1.smithi192.stdout:5/692: fdatasync d0/d13/d19/d64/d5f/d2c/f5c 0 2024-04-24T22:45:45.755 INFO:tasks.workunit.client.0.smithi063.stdout:7/950: mknod d0/d4/de/d95/c13d 0 2024-04-24T22:45:45.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/960: fdatasync d6/d64/d21/d25/d62/d63/d89/ff3 0 2024-04-24T22:45:45.759 INFO:tasks.workunit.client.0.smithi063.stdout:0/961: readlink d6/da/d2b/l77 0 2024-04-24T22:45:45.760 INFO:tasks.workunit.client.1.smithi192.stdout:3/826: creat d2/d27/d28/dfd/f107 x:0 0 0 2024-04-24T22:45:45.762 INFO:tasks.workunit.client.1.smithi192.stdout:5/693: dwrite d0/f4a [0,4194304] 0 2024-04-24T22:45:45.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/827: dread - d2/d4/d12/d39/d97/fdf zero size 2024-04-24T22:45:45.763 INFO:tasks.workunit.client.1.smithi192.stdout:5/694: readlink d0/d13/d3b/d79/le8 0 2024-04-24T22:45:45.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/739: dread d0/d8/d3d/f62 [0,4194304] 0 2024-04-24T22:45:45.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/740: write d0/d8/d22/dcb/de0/de7/f100 [792310,40747] 0 2024-04-24T22:45:45.768 INFO:tasks.workunit.client.1.smithi192.stdout:8/811: creat db/ffb x:0 0 0 2024-04-24T22:45:45.769 INFO:tasks.workunit.client.1.smithi192.stdout:5/695: mknod d0/d13/d19/d64/d4f/d7c/cef 0 2024-04-24T22:45:45.770 INFO:tasks.workunit.client.1.smithi192.stdout:8/812: write db/dc/d29/f47 [1232637,35270] 0 2024-04-24T22:45:45.773 INFO:tasks.workunit.client.1.smithi192.stdout:5/696: creat d0/d13/d3b/d2f/ff0 x:0 0 0 2024-04-24T22:45:45.774 INFO:tasks.workunit.client.1.smithi192.stdout:8/813: symlink db/dc/d22/d31/d33/d81/lfc 0 2024-04-24T22:45:45.775 INFO:tasks.workunit.client.1.smithi192.stdout:5/697: mkdir d0/d13/dd1/df1 0 2024-04-24T22:45:45.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/698: write d0/d13/d19/fe4 [888439,34639] 0 2024-04-24T22:45:45.778 INFO:tasks.workunit.client.1.smithi192.stdout:8/814: truncate db/dc/f80 909593 0 2024-04-24T22:45:45.780 INFO:tasks.workunit.client.1.smithi192.stdout:8/815: mknod db/dc/d22/d31/d33/d81/cfd 0 2024-04-24T22:45:45.781 INFO:tasks.workunit.client.1.smithi192.stdout:5/699: rename d0/d13/d53/f70 to d0/d13/d19/d64/d4f/d7c/ff2 0 2024-04-24T22:45:45.782 INFO:tasks.workunit.client.1.smithi192.stdout:8/816: creat db/dc/d22/d31/d33/d81/d8a/ffe x:0 0 0 2024-04-24T22:45:45.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/913: sync 2024-04-24T22:45:45.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/817: stat db/d43/d7c/d85/fd8 0 2024-04-24T22:45:45.783 INFO:tasks.workunit.client.1.smithi192.stdout:5/700: creat d0/d13/d19/d71/d7f/d9a/ff3 x:0 0 0 2024-04-24T22:45:45.783 INFO:tasks.workunit.client.1.smithi192.stdout:9/914: mknod d3/d24/c12b 0 2024-04-24T22:45:45.786 INFO:tasks.workunit.client.1.smithi192.stdout:5/701: creat d0/d13/d19/d64/d5f/d2c/d3c/d91/ff4 x:0 0 0 2024-04-24T22:45:45.788 INFO:tasks.workunit.client.1.smithi192.stdout:9/915: truncate d3/dd/de/d17/d89/d8b/f3e 2317793 0 2024-04-24T22:45:45.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/818: link db/dc/l6a db/d43/d76/d92/dad/lff 0 2024-04-24T22:45:45.794 INFO:tasks.workunit.client.1.smithi192.stdout:5/702: dwrite d0/d13/d19/d64/d5f/fd4 [0,4194304] 0 2024-04-24T22:45:45.799 INFO:tasks.workunit.client.1.smithi192.stdout:5/703: creat d0/d13/d3b/ff5 x:0 0 0 2024-04-24T22:45:45.800 INFO:tasks.workunit.client.1.smithi192.stdout:5/704: chown d0/d13/d19/f4b 15318372 1 2024-04-24T22:45:45.802 INFO:tasks.workunit.client.1.smithi192.stdout:2/927: write d4/d29/d60/f47 [318437,95997] 0 2024-04-24T22:45:45.804 INFO:tasks.workunit.client.0.smithi063.stdout:7/951: creat d0/d4/de/d6f/d2e/d55/ddc/f13e x:0 0 0 2024-04-24T22:45:45.808 INFO:tasks.workunit.client.1.smithi192.stdout:5/705: dwrite d0/d13/d19/da9/fba [0,4194304] 0 2024-04-24T22:45:45.809 INFO:tasks.workunit.client.1.smithi192.stdout:0/825: write d3/d27/d28/dcc/d5c/fe2 [132372,85701] 0 2024-04-24T22:45:45.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/828: write d2/d27/d28/f36 [723533,12927] 0 2024-04-24T22:45:45.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/962: truncate d6/d64/d21/d25/d62/d63/f82 3958445 0 2024-04-24T22:45:45.816 INFO:tasks.workunit.client.1.smithi192.stdout:3/829: unlink d2/d27/d28/dab/dca/ff4 0 2024-04-24T22:45:45.817 INFO:tasks.workunit.client.1.smithi192.stdout:5/706: dread d0/d1b/f6d [0,4194304] 0 2024-04-24T22:45:45.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/826: dwrite d3/f1c [0,4194304] 0 2024-04-24T22:45:45.823 INFO:tasks.workunit.client.1.smithi192.stdout:6/741: dwrite d0/d8/d29/d4c/d4d/f9e [0,4194304] 0 2024-04-24T22:45:45.824 INFO:tasks.workunit.client.1.smithi192.stdout:0/827: chown d3/d27/d3c/d3b/l7c 16944 1 2024-04-24T22:45:45.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/830: creat d2/d27/dc6/da0/f108 x:0 0 0 2024-04-24T22:45:45.826 INFO:tasks.workunit.client.1.smithi192.stdout:6/742: chown d0/d8/d18/c99 1441954 1 2024-04-24T22:45:45.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/828: creat d3/d27/d28/dcc/d5c/d80/d83/dfb/f112 x:0 0 0 2024-04-24T22:45:45.839 INFO:tasks.workunit.client.1.smithi192.stdout:0/829: dread - d3/d27/d6a/deb/ff1 zero size 2024-04-24T22:45:45.843 INFO:tasks.workunit.client.1.smithi192.stdout:9/916: dwrite d3/dd/fa4 [0,4194304] 0 2024-04-24T22:45:45.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/952: dwrite d0/d4/de/d17/d35/d69/fb0 [0,4194304] 0 2024-04-24T22:45:45.844 INFO:tasks.workunit.client.1.smithi192.stdout:9/917: fsync d3/d59/f114 0 2024-04-24T22:45:45.846 INFO:tasks.workunit.client.1.smithi192.stdout:0/830: fsync d3/d27/d28/dcc/d5c/d80/d83/ff9 0 2024-04-24T22:45:45.849 INFO:tasks.workunit.client.1.smithi192.stdout:9/918: fsync d3/dd/de/d17/d89/f122 0 2024-04-24T22:45:45.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/928: dwrite d4/d9/f2c [0,4194304] 0 2024-04-24T22:45:45.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/831: symlink d3/d27/d28/dcc/d5c/l113 0 2024-04-24T22:45:45.852 INFO:tasks.workunit.client.1.smithi192.stdout:2/929: chown d4/d9/d37/f91 39447 1 2024-04-24T22:45:45.853 INFO:tasks.workunit.client.0.smithi063.stdout:7/953: dread d0/d4/de/d17/fc6 [0,4194304] 0 2024-04-24T22:45:45.854 INFO:tasks.workunit.client.0.smithi063.stdout:7/954: read d0/d4/de/d17/fc6 [151616,66641] 0 2024-04-24T22:45:45.856 INFO:tasks.workunit.client.1.smithi192.stdout:9/919: dread d3/d59/f5a [0,4194304] 0 2024-04-24T22:45:45.860 INFO:tasks.workunit.client.1.smithi192.stdout:2/930: rename d4/d9/f18 to d4/d9/d10/da7/df5/f12d 0 2024-04-24T22:45:45.864 INFO:tasks.workunit.client.1.smithi192.stdout:0/832: dwrite d3/d27/d28/d31/f52 [4194304,4194304] 0 2024-04-24T22:45:45.865 INFO:tasks.workunit.client.0.smithi063.stdout:7/955: rename d0/d4/de/d17/d28/d31/d99/d113/l129 to d0/d4/de/d6f/d2e/d76/l13f 0 2024-04-24T22:45:45.866 INFO:tasks.workunit.client.0.smithi063.stdout:7/956: chown d0/d4/de/d6f/d2e 0 1 2024-04-24T22:45:45.866 INFO:tasks.workunit.client.1.smithi192.stdout:9/920: fdatasync d3/dd/de/da7/fd3 0 2024-04-24T22:45:45.868 INFO:tasks.workunit.client.1.smithi192.stdout:2/931: mkdir d4/d9/d10/d6c/d8c/da5/de2/d12e 0 2024-04-24T22:45:45.869 INFO:tasks.workunit.client.1.smithi192.stdout:6/743: dwrite d0/d8/d22/f9a [0,4194304] 0 2024-04-24T22:45:45.871 INFO:tasks.workunit.client.0.smithi063.stdout:5/989: sync 2024-04-24T22:45:45.872 INFO:tasks.workunit.client.0.smithi063.stdout:7/957: read d0/d4/fe1 [91669,17343] 0 2024-04-24T22:45:45.873 INFO:tasks.workunit.client.1.smithi192.stdout:5/707: dread d0/d1b/f24 [0,4194304] 0 2024-04-24T22:45:45.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/958: chown d0/d4/de/d17/d35/d69/da0/la8 185154 1 2024-04-24T22:45:45.874 INFO:tasks.workunit.client.0.smithi063.stdout:7/959: fsync d0/d4/de/d17/f8a 0 2024-04-24T22:45:45.875 INFO:tasks.workunit.client.1.smithi192.stdout:9/921: mkdir d3/d59/d7d/d12c 0 2024-04-24T22:45:45.880 INFO:tasks.workunit.client.1.smithi192.stdout:2/932: unlink d4/d9/d10/d6c/d6f/da8/f11e 0 2024-04-24T22:45:45.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/744: mkdir d0/d8/d18/df1/df2/d103 0 2024-04-24T22:45:45.881 INFO:tasks.workunit.client.1.smithi192.stdout:6/745: stat d0/d8/d3d/d3e/d40/d44/c98 0 2024-04-24T22:45:45.884 INFO:tasks.workunit.client.1.smithi192.stdout:6/746: unlink d0/d8/d18/ca3 0 2024-04-24T22:45:45.884 INFO:tasks.workunit.client.1.smithi192.stdout:6/747: stat d0/d8/d29/d5a/c8b 0 2024-04-24T22:45:45.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/960: dwrite d0/d4/de/d6f/d5c/d9c/f103 [0,4194304] 0 2024-04-24T22:45:45.887 INFO:tasks.workunit.client.1.smithi192.stdout:6/748: creat d0/d8/d3d/d3e/d40/d44/f104 x:0 0 0 2024-04-24T22:45:45.888 INFO:tasks.workunit.client.0.smithi063.stdout:7/961: symlink d0/d4/de/d17/d35/d69/da0/l140 0 2024-04-24T22:45:45.892 INFO:tasks.workunit.client.1.smithi192.stdout:6/749: link d0/d8/d3d/d3e/d40/l64 d0/d8/d3d/d3e/d40/l105 0 2024-04-24T22:45:45.893 INFO:tasks.workunit.client.1.smithi192.stdout:3/831: write d2/d4/d12/d1a/d1d/d55/f24 [7679351,106476] 0 2024-04-24T22:45:45.894 INFO:tasks.workunit.client.0.smithi063.stdout:0/963: sync 2024-04-24T22:45:45.896 INFO:tasks.workunit.client.1.smithi192.stdout:6/750: symlink d0/d8/d3d/d3e/d40/l106 0 2024-04-24T22:45:45.896 INFO:tasks.workunit.client.0.smithi063.stdout:7/962: dwrite d0/d4/de/dfc/f109 [0,4194304] 0 2024-04-24T22:45:45.900 INFO:tasks.workunit.client.1.smithi192.stdout:6/751: creat d0/d8/d3d/d9d/f107 x:0 0 0 2024-04-24T22:45:45.902 INFO:tasks.workunit.client.0.smithi063.stdout:7/963: mkdir d0/d4/de/d17/d35/d69/dc9/d120/d141 0 2024-04-24T22:45:45.903 INFO:tasks.workunit.client.1.smithi192.stdout:6/752: creat d0/d8/d3d/d9d/da5/f108 x:0 0 0 2024-04-24T22:45:45.904 INFO:tasks.workunit.client.0.smithi063.stdout:5/990: dread d5/d11/d23/d2d/d62/d73/d118/f3d [0,4194304] 0 2024-04-24T22:45:45.904 INFO:tasks.workunit.client.0.smithi063.stdout:0/964: rmdir d6/d64/d21/d25/d62/d63/d89/d95/da4/dff 0 2024-04-24T22:45:45.906 INFO:tasks.workunit.client.0.smithi063.stdout:5/991: write d5/d11/d23/d2d/d62/d73/d118/f158 [522638,6789] 0 2024-04-24T22:45:45.906 INFO:tasks.workunit.client.0.smithi063.stdout:7/964: creat d0/d4/de/dfc/f142 x:0 0 0 2024-04-24T22:45:45.907 INFO:tasks.workunit.client.1.smithi192.stdout:6/753: mknod d0/d8/c109 0 2024-04-24T22:45:45.908 INFO:tasks.workunit.client.0.smithi063.stdout:0/965: mknod d6/d64/d24/dd1/c145 0 2024-04-24T22:45:45.909 INFO:tasks.workunit.client.1.smithi192.stdout:6/754: dread d0/d8/d3d/f62 [0,4194304] 0 2024-04-24T22:45:45.911 INFO:tasks.workunit.client.0.smithi063.stdout:0/966: creat d6/d64/d21/d25/d62/d63/d89/d95/df6/f146 x:0 0 0 2024-04-24T22:45:45.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/755: mknod d0/d8/d18/c10a 0 2024-04-24T22:45:45.915 INFO:tasks.workunit.client.0.smithi063.stdout:0/967: truncate d6/dd/f54 1836300 0 2024-04-24T22:45:45.918 INFO:tasks.workunit.client.0.smithi063.stdout:0/968: truncate d6/d64/da0/d33/fc3 5316814 0 2024-04-24T22:45:45.918 INFO:tasks.workunit.client.1.smithi192.stdout:6/756: dread d0/d8/d22/dcb/de0/de7/f100 [0,4194304] 0 2024-04-24T22:45:45.920 INFO:tasks.workunit.client.0.smithi063.stdout:0/969: fsync d6/d64/d21/d25/d31/da3/fc4 0 2024-04-24T22:45:45.922 INFO:tasks.workunit.client.1.smithi192.stdout:6/757: mkdir d0/d8/d18/d10b 0 2024-04-24T22:45:45.923 INFO:tasks.workunit.client.1.smithi192.stdout:0/833: dread d3/d27/d28/d31/dbc/fd8 [0,4194304] 0 2024-04-24T22:45:45.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/832: dread d2/d4/d12/d1a/d1d/f3f [0,4194304] 0 2024-04-24T22:45:45.924 INFO:tasks.workunit.client.1.smithi192.stdout:3/833: chown d2/d27/dc6/f60 350 1 2024-04-24T22:45:45.924 INFO:tasks.workunit.client.1.smithi192.stdout:0/834: rmdir d3/d40 39 2024-04-24T22:45:45.927 INFO:tasks.workunit.client.1.smithi192.stdout:6/758: dread d0/d8/d29/f3b [0,4194304] 0 2024-04-24T22:45:45.927 INFO:tasks.workunit.client.1.smithi192.stdout:0/835: dread d3/d27/d28/dcc/d5c/d80/fb7 [0,4194304] 0 2024-04-24T22:45:45.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/836: readlink d3/d27/d28/d2e/l82 0 2024-04-24T22:45:45.929 INFO:tasks.workunit.client.1.smithi192.stdout:0/837: write d3/d27/d28/d31/f52 [1717071,24869] 0 2024-04-24T22:45:45.930 INFO:tasks.workunit.client.1.smithi192.stdout:6/759: symlink d0/d8/d29/d4c/d4d/d83/dde/l10c 0 2024-04-24T22:45:45.933 INFO:tasks.workunit.client.0.smithi063.stdout:7/965: dread d0/d4/de/d6f/f5b [0,4194304] 0 2024-04-24T22:45:45.933 INFO:tasks.workunit.client.1.smithi192.stdout:6/760: mkdir d0/d8/d22/dcb/de0/d10d 0 2024-04-24T22:45:45.934 INFO:tasks.workunit.client.0.smithi063.stdout:7/966: chown d0/d4/de/d17/d35/d69/da0/la8 1 1 2024-04-24T22:45:45.938 INFO:tasks.workunit.client.0.smithi063.stdout:7/967: rmdir d0/df9/dfd 0 2024-04-24T22:45:45.938 INFO:tasks.workunit.client.1.smithi192.stdout:6/761: creat d0/d8/d29/d5a/f10e x:0 0 0 2024-04-24T22:45:45.941 INFO:tasks.workunit.client.0.smithi063.stdout:7/968: creat d0/d4/de/d17/d35/d6c/d7a/f143 x:0 0 0 2024-04-24T22:45:45.946 INFO:tasks.workunit.client.1.smithi192.stdout:7/981: sync 2024-04-24T22:45:45.956 INFO:tasks.workunit.client.1.smithi192.stdout:5/708: write d0/d13/d19/f1d [1685396,129718] 0 2024-04-24T22:45:45.963 INFO:tasks.workunit.client.1.smithi192.stdout:3/834: dread d2/d27/d28/dab/fc2 [0,4194304] 0 2024-04-24T22:45:45.974 INFO:tasks.workunit.client.0.smithi063.stdout:5/992: truncate d5/d11/d76/d92/dea/ff0 3367843 0 2024-04-24T22:45:45.984 INFO:tasks.workunit.client.1.smithi192.stdout:5/709: creat d0/d13/d19/d64/d4f/d6c/ff6 x:0 0 0 2024-04-24T22:45:45.984 INFO:tasks.workunit.client.0.smithi063.stdout:0/970: dwrite d6/d64/d21/d25/d31/da3/f100 [0,4194304] 0 2024-04-24T22:45:45.985 INFO:tasks.workunit.client.0.smithi063.stdout:0/971: dread d6/d64/d21/d25/d62/d63/d7c/f12c [0,4194304] 0 2024-04-24T22:45:45.987 INFO:tasks.workunit.client.1.smithi192.stdout:0/838: dwrite d3/d6d/d79/f7a [0,4194304] 0 2024-04-24T22:45:45.987 INFO:tasks.workunit.client.1.smithi192.stdout:3/835: rmdir d2/d27/dc6 39 2024-04-24T22:45:45.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/993: link d5/d11/d18/c37 d5/d11/d23/d2d/d62/d73/d118/d49/c159 0 2024-04-24T22:45:45.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/836: dwrite d2/d4/d4e/ff1 [0,4194304] 0 2024-04-24T22:45:45.997 INFO:tasks.workunit.client.0.smithi063.stdout:5/994: read d5/d11/d23/d2d/d62/d81/d10c/f13d [2748706,88806] 0 2024-04-24T22:45:45.998 INFO:tasks.workunit.client.1.smithi192.stdout:3/837: chown d2/d4/d4e/le5 92469 1 2024-04-24T22:45:45.999 INFO:tasks.workunit.client.0.smithi063.stdout:0/972: mkdir d6/d64/d21/d25/d62/d63/d7c/d147 0 2024-04-24T22:45:46.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/969: write d0/d4/de/d17/d35/f79 [2726002,72230] 0 2024-04-24T22:45:46.005 INFO:tasks.workunit.client.1.smithi192.stdout:0/839: rename d3/d27/d28/dcc/d5c/d80/d83/dd9/cde to d3/d27/d6a/c114 0 2024-04-24T22:45:46.010 INFO:tasks.workunit.client.1.smithi192.stdout:7/982: creat d2/db/d25/f12f x:0 0 0 2024-04-24T22:45:46.012 INFO:tasks.workunit.client.1.smithi192.stdout:3/838: chown d2/d4/d12/d1a/d1d/d55/d5f/dbd/lc9 125 1 2024-04-24T22:45:46.013 INFO:tasks.workunit.client.0.smithi063.stdout:5/995: truncate d5/d11/d76/f9a 3665010 0 2024-04-24T22:45:46.013 INFO:tasks.workunit.client.1.smithi192.stdout:3/839: chown d2/d27/d28/d38/ldd 41 1 2024-04-24T22:45:46.013 INFO:tasks.workunit.client.1.smithi192.stdout:5/710: creat d0/d1b/d62/ff7 x:0 0 0 2024-04-24T22:45:46.014 INFO:tasks.workunit.client.0.smithi063.stdout:7/970: mknod d0/d4/de/d6f/d2e/d76/d9f/dbd/dd1/d12c/d138/c144 0 2024-04-24T22:45:46.015 INFO:tasks.workunit.client.1.smithi192.stdout:5/711: write d0/d13/d19/d64/d5f/f72 [2449281,119664] 0 2024-04-24T22:45:46.015 INFO:tasks.workunit.client.0.smithi063.stdout:7/971: stat d0/d4/de/d17/d35/d6c 0 2024-04-24T22:45:46.016 INFO:tasks.workunit.client.0.smithi063.stdout:7/972: write d0/d4/de/f7c [5212051,35371] 0 2024-04-24T22:45:46.017 INFO:tasks.workunit.client.1.smithi192.stdout:0/840: unlink d3/d27/d28/dcc/d5c/d80/d83/ff9 0 2024-04-24T22:45:46.018 INFO:tasks.workunit.client.0.smithi063.stdout:7/973: write d0/d4/de/d17/d35/f54 [1043634,9278] 0 2024-04-24T22:45:46.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/973: mknod d6/d64/d21/d25/c148 0 2024-04-24T22:45:46.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/974: write d0/d4/de/d17/d35/d69/f6b [2266056,766] 0 2024-04-24T22:45:46.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/712: dread d0/d13/d19/d64/d4f/d52/f61 [0,4194304] 0 2024-04-24T22:45:46.021 INFO:tasks.workunit.client.0.smithi063.stdout:0/974: write d6/d64/d21/d25/d62/d63/d89/d95/df6/f11e [964407,90441] 0 2024-04-24T22:45:46.025 INFO:tasks.workunit.client.1.smithi192.stdout:0/841: rename d3/d27/d28/d2e/l82 to d3/d27/d28/dcc/da8/d111/l115 0 2024-04-24T22:45:46.028 INFO:tasks.workunit.client.1.smithi192.stdout:7/983: unlink d2/db/d2a/l97 0 2024-04-24T22:45:46.029 INFO:tasks.workunit.client.0.smithi063.stdout:7/975: mkdir d0/d4/de/d17/d35/d69/dc9/d145 0 2024-04-24T22:45:46.032 INFO:tasks.workunit.client.1.smithi192.stdout:0/842: dwrite d3/d27/d28/dcc/d5c/d80/d83/dfb/f112 [0,4194304] 0 2024-04-24T22:45:46.034 INFO:tasks.workunit.client.1.smithi192.stdout:0/843: read - d3/d27/d28/dcc/f10d zero size 2024-04-24T22:45:46.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/840: symlink d2/d4/d12/d1a/d1d/d7b/l109 0 2024-04-24T22:45:46.039 INFO:tasks.workunit.client.0.smithi063.stdout:7/976: rmdir d0/d4/de/d17/d35/d6c/d7a 39 2024-04-24T22:45:46.041 INFO:tasks.workunit.client.0.smithi063.stdout:7/977: write d0/d4/de/d17/d35/fd5 [1732179,116181] 0 2024-04-24T22:45:46.041 INFO:tasks.workunit.client.0.smithi063.stdout:0/975: read d6/d64/f92 [960280,38934] 0 2024-04-24T22:45:46.043 INFO:tasks.workunit.client.0.smithi063.stdout:0/976: chown d6/d64/d21/d80/fd7 166803279 1 2024-04-24T22:45:46.043 INFO:tasks.workunit.client.1.smithi192.stdout:7/984: rename d2/db/d21/d42/d5b/d63/lc9 to d2/db/d2a/l130 0 2024-04-24T22:45:46.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/985: dread d2/db/d25/f96 [0,4194304] 0 2024-04-24T22:45:46.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/986: chown d2/db/d21/d42/d67/d9b/d6e/db5/db6/f8d 150054 1 2024-04-24T22:45:46.047 INFO:tasks.workunit.client.1.smithi192.stdout:6/762: truncate d0/d8/d29/f30 3060334 0 2024-04-24T22:45:46.049 INFO:tasks.workunit.client.1.smithi192.stdout:0/844: symlink d3/d27/d6a/deb/l116 0 2024-04-24T22:45:46.052 INFO:tasks.workunit.client.1.smithi192.stdout:3/841: mkdir d2/d4/d12/dbc/d10a 0 2024-04-24T22:45:46.053 INFO:tasks.workunit.client.0.smithi063.stdout:7/978: mkdir d0/d4/de/d6f/d5c/d146 0 2024-04-24T22:45:46.054 INFO:tasks.workunit.client.1.smithi192.stdout:7/987: rmdir d2/db/d21/dd5 39 2024-04-24T22:45:46.055 INFO:tasks.workunit.client.1.smithi192.stdout:5/713: creat d0/d13/d19/d71/ff8 x:0 0 0 2024-04-24T22:45:46.056 INFO:tasks.workunit.client.1.smithi192.stdout:5/714: read d0/d13/d19/da9/fba [71145,19915] 0 2024-04-24T22:45:46.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/845: symlink d3/d27/d6a/dc4/l117 0 2024-04-24T22:45:46.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/977: dread d6/da/d2b/d60/faf [0,4194304] 0 2024-04-24T22:45:46.069 INFO:tasks.workunit.client.1.smithi192.stdout:0/846: mkdir d3/d27/d3c/d6c/db3/d118 0 2024-04-24T22:45:46.073 INFO:tasks.workunit.client.0.smithi063.stdout:7/979: dread d0/d4/de/d6f/f47 [0,4194304] 0 2024-04-24T22:45:46.074 INFO:tasks.workunit.client.0.smithi063.stdout:5/996: truncate d5/d11/d23/d2d/d62/d73/d118/d36/d131/f130 465971 0 2024-04-24T22:45:46.075 INFO:tasks.workunit.client.0.smithi063.stdout:5/997: chown d5/d11/d23/d2d/d62/d81/da4/da6/d103 29999347 1 2024-04-24T22:45:46.078 INFO:tasks.workunit.client.0.smithi063.stdout:7/980: creat d0/d4/de/d17/d28/d31/d99/d113/f147 x:0 0 0 2024-04-24T22:45:46.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/998: dwrite d5/d11/d23/d2d/d62/d73/d118/d49/d3b/dc5/f11d [0,4194304] 0 2024-04-24T22:45:46.083 INFO:tasks.workunit.client.0.smithi063.stdout:5/999: write d5/d1d/d7d/ffb [858059,102596] 0 2024-04-24T22:45:46.084 INFO:tasks.workunit.client.0.smithi063.stdout:7/981: rmdir d0/d4 39 2024-04-24T22:45:46.087 INFO:tasks.workunit.client.1.smithi192.stdout:6/763: write d0/d8/f34 [403753,7725] 0 2024-04-24T22:45:46.089 INFO:tasks.workunit.client.1.smithi192.stdout:3/842: write d2/d4/d12/d1a/d1d/f3f [3052409,17311] 0 2024-04-24T22:45:46.091 INFO:tasks.workunit.client.1.smithi192.stdout:5/715: dread d0/d13/d19/d64/d4f/d6c/f9c [0,4194304] 0 2024-04-24T22:45:46.091 INFO:tasks.workunit.client.1.smithi192.stdout:5/716: stat d0/d5b/f7a 0 2024-04-24T22:45:46.092 INFO:tasks.workunit.client.1.smithi192.stdout:6/764: symlink d0/d8/d29/d4c/db0/l10f 0 2024-04-24T22:45:46.095 INFO:tasks.workunit.client.1.smithi192.stdout:3/843: dread - d2/d4/d12/dbc/fa1 zero size 2024-04-24T22:45:46.096 INFO:tasks.workunit.client.0.smithi063.stdout:0/978: write d6/d3a/fee [1418639,42393] 0 2024-04-24T22:45:46.096 INFO:tasks.workunit.client.1.smithi192.stdout:3/844: readlink d2/d4/d12/d1a/d1d/d7b/df7/d54/l6d 0 2024-04-24T22:45:46.097 INFO:tasks.workunit.client.1.smithi192.stdout:5/717: truncate d0/d13/d19/d83/f87 2604421 0 2024-04-24T22:45:46.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/979: unlink d6/d64/d21/d25/d62/d63/d7c/f109 0 2024-04-24T22:45:46.106 INFO:tasks.workunit.client.1.smithi192.stdout:7/988: write d2/db/d21/d42/d5b/f6b [104745,44427] 0 2024-04-24T22:45:46.108 INFO:tasks.workunit.client.0.smithi063.stdout:0/980: rename d6/d64/da0/d98/cd3 to d6/d64/da0/d33/deb/c149 0 2024-04-24T22:45:46.109 INFO:tasks.workunit.client.0.smithi063.stdout:0/981: write d6/d64/d24/d42/dd4/f96 [1899023,19400] 0 2024-04-24T22:45:46.110 INFO:tasks.workunit.client.1.smithi192.stdout:5/718: creat d0/d13/d19/d83/dad/ff9 x:0 0 0 2024-04-24T22:45:46.111 INFO:tasks.workunit.client.1.smithi192.stdout:0/847: truncate d3/d27/d28/dcc/d5c/fe2 2770655 0 2024-04-24T22:45:46.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/848: write d3/d27/d28/dcc/d5c/d80/d83/dfb/f106 [7638979,50500] 0 2024-04-24T22:45:46.113 INFO:tasks.workunit.client.0.smithi063.stdout:0/982: dread d6/d64/d21/d25/f55 [0,4194304] 0 2024-04-24T22:45:46.114 INFO:tasks.workunit.client.1.smithi192.stdout:6/765: rmdir d0/d8/d3d/d9d/da5/dc8/ddb 0 2024-04-24T22:45:46.115 INFO:tasks.workunit.client.1.smithi192.stdout:6/766: readlink d0/d8/d29/d5a/l69 0 2024-04-24T22:45:46.116 INFO:tasks.workunit.client.1.smithi192.stdout:6/767: write d0/d8/d29/d5a/fb6 [2038768,119089] 0 2024-04-24T22:45:46.120 INFO:tasks.workunit.client.0.smithi063.stdout:7/982: dwrite d0/d4/de/d6f/d2e/ff0 [0,4194304] 0 2024-04-24T22:45:46.136 INFO:tasks.workunit.client.1.smithi192.stdout:4/770: sync 2024-04-24T22:45:46.137 INFO:tasks.workunit.client.1.smithi192.stdout:4/771: stat d2/db/d10/l94 0 2024-04-24T22:45:46.141 INFO:tasks.workunit.client.1.smithi192.stdout:0/849: truncate d3/d27/d28/dcc/d5c/d80/fb7 863605 0 2024-04-24T22:45:46.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/983: mknod d0/d4/de/d17/d28/d31/d99/d113/c148 0 2024-04-24T22:45:46.143 INFO:tasks.workunit.client.1.smithi192.stdout:7/989: chown d2/db/d25/dda/c110 1 1 2024-04-24T22:45:46.146 INFO:tasks.workunit.client.1.smithi192.stdout:3/845: write d2/d27/d28/dab/fc2 [1131937,114390] 0 2024-04-24T22:45:46.146 INFO:tasks.workunit.client.1.smithi192.stdout:5/719: symlink d0/d1b/lfa 0 2024-04-24T22:45:46.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/984: chown d0/d4/de/d6f/c40 2441456 1 2024-04-24T22:45:46.149 INFO:tasks.workunit.client.1.smithi192.stdout:6/768: mkdir d0/d8/d3d/d9d/da5/dc8/d110 0 2024-04-24T22:45:46.150 INFO:tasks.workunit.client.1.smithi192.stdout:7/990: dwrite d2/db/d21/d42/d67/f118 [0,4194304] 0 2024-04-24T22:45:46.151 INFO:tasks.workunit.client.1.smithi192.stdout:4/772: write d2/db/d5d/db3/fc2 [828762,79283] 0 2024-04-24T22:45:46.157 INFO:tasks.workunit.client.0.smithi063.stdout:7/985: rename d0/d4/de/f25 to d0/d4/de/dfc/de7/f149 0 2024-04-24T22:45:46.161 INFO:tasks.workunit.client.0.smithi063.stdout:7/986: truncate d0/d4/de/d17/d35/d69/dc9/d120/f12d 204562 0 2024-04-24T22:45:46.162 INFO:tasks.workunit.client.1.smithi192.stdout:4/773: mkdir d2/db/d75/d10d 0 2024-04-24T22:45:46.163 INFO:tasks.workunit.client.0.smithi063.stdout:0/983: write d6/d64/da0/d98/dd6/f130 [151422,67773] 0 2024-04-24T22:45:46.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/846: dread d2/d27/dc6/f60 [0,4194304] 0 2024-04-24T22:45:46.169 INFO:tasks.workunit.client.0.smithi063.stdout:7/987: symlink d0/d4/de/d17/d35/d69/dc9/d145/l14a 0 2024-04-24T22:45:46.170 INFO:tasks.workunit.client.0.smithi063.stdout:7/988: chown d0/d4/de/d6f/d2e/d55/ddc/c101 12 1 2024-04-24T22:45:46.170 INFO:tasks.workunit.client.1.smithi192.stdout:4/774: creat d2/d5c/dbb/f10e x:0 0 0 2024-04-24T22:45:46.171 INFO:tasks.workunit.client.1.smithi192.stdout:4/775: stat d2/db/d75/da1/d7d/d9c/fc4 0 2024-04-24T22:45:46.175 INFO:tasks.workunit.client.0.smithi063.stdout:7/989: mknod d0/d4/de/dfc/de7/c14b 0 2024-04-24T22:45:46.175 INFO:tasks.workunit.client.0.smithi063.stdout:7/990: chown d0/d4/de/d17 252056 1 2024-04-24T22:45:46.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/769: dread d0/d8/d3d/d9d/fd0 [0,4194304] 0 2024-04-24T22:45:46.176 INFO:tasks.workunit.client.0.smithi063.stdout:0/984: link d6/da/d2b/l34 d6/d64/da0/d33/l14a 0 2024-04-24T22:45:46.176 INFO:tasks.workunit.client.0.smithi063.stdout:7/991: rename d0 to d0/d4/de/d17/d35/d69/d14c 22 2024-04-24T22:45:46.178 INFO:tasks.workunit.client.1.smithi192.stdout:0/850: dread d3/d27/f43 [0,4194304] 0 2024-04-24T22:45:46.178 INFO:tasks.workunit.client.1.smithi192.stdout:5/720: getdents d0/d13/d19/d64/d4f/d7c 0 2024-04-24T22:45:46.179 INFO:tasks.workunit.client.1.smithi192.stdout:7/991: truncate d2/db/d2a/ff7 181154 0 2024-04-24T22:45:46.179 INFO:tasks.workunit.client.1.smithi192.stdout:7/992: fdatasync d2/db/d21/d42/d67/d9b/d6e/fe4 0 2024-04-24T22:45:46.182 INFO:tasks.workunit.client.0.smithi063.stdout:0/985: dwrite d6/d64/d21/d25/d62/d63/d7c/f10b [0,4194304] 0 2024-04-24T22:45:46.183 INFO:tasks.workunit.client.1.smithi192.stdout:6/770: creat d0/d8/d29/d4c/d4d/f111 x:0 0 0 2024-04-24T22:45:46.185 INFO:tasks.workunit.client.1.smithi192.stdout:5/721: creat d0/d13/d19/d64/d4f/d7c/ffb x:0 0 0 2024-04-24T22:45:46.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/993: creat d2/db/d21/d42/d67/d9b/d6e/db5/db6/f131 x:0 0 0 2024-04-24T22:45:46.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/992: dread d0/d4/de/dfc/ff7 [0,4194304] 0 2024-04-24T22:45:46.191 INFO:tasks.workunit.client.1.smithi192.stdout:3/847: rename d2/d27/d28/dfd/f107 to d2/d4/f10b 0 2024-04-24T22:45:46.192 INFO:tasks.workunit.client.1.smithi192.stdout:7/994: write d2/db/d21/d42/d67/d9b/d6e/db5/db6/fff [2975673,19925] 0 2024-04-24T22:45:46.194 INFO:tasks.workunit.client.1.smithi192.stdout:3/848: rename d2 to d2/d27/d28/d10c 22 2024-04-24T22:45:46.197 INFO:tasks.workunit.client.0.smithi063.stdout:7/993: rmdir d0/d4/de/d17/d28/d31/d99/d113 39 2024-04-24T22:45:46.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/776: dwrite d2/db/d5d/d77/d82/fa3 [0,4194304] 0 2024-04-24T22:45:46.199 INFO:tasks.workunit.client.0.smithi063.stdout:0/986: write d6/d64/d21/d25/d62/d63/f6e [3289627,34981] 0 2024-04-24T22:45:46.204 INFO:tasks.workunit.client.1.smithi192.stdout:4/777: truncate d2/db/d75/da1/d36/d5b/d28/d2f/fe6 574868 0 2024-04-24T22:45:46.204 INFO:tasks.workunit.client.1.smithi192.stdout:0/851: creat d3/d27/d28/d2e/f119 x:0 0 0 2024-04-24T22:45:46.205 INFO:tasks.workunit.client.1.smithi192.stdout:7/995: dwrite d2/db/d21/d42/d67/d9b/dc1/f125 [0,4194304] 0 2024-04-24T22:45:46.212 INFO:tasks.workunit.client.1.smithi192.stdout:3/849: unlink d2/d4/d4e/l84 0 2024-04-24T22:45:46.212 INFO:tasks.workunit.client.1.smithi192.stdout:6/771: write d0/d8/d3d/d3e/d40/d44/f73 [5219060,13243] 0 2024-04-24T22:45:46.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/994: write d0/d4/de/d6f/d2e/f85 [582491,53259] 0 2024-04-24T22:45:46.214 INFO:tasks.workunit.client.0.smithi063.stdout:0/987: mknod d6/d64/d21/d25/d62/d63/c14b 0 2024-04-24T22:45:46.217 INFO:tasks.workunit.client.1.smithi192.stdout:0/852: dwrite d3/d27/d28/f56 [4194304,4194304] 0 2024-04-24T22:45:46.218 INFO:tasks.workunit.client.1.smithi192.stdout:0/853: chown d3/d6d/d79 1327944 1 2024-04-24T22:45:46.218 INFO:tasks.workunit.client.0.smithi063.stdout:7/995: truncate d0/d4/de/d17/d28/d31/d99/ff4 585607 0 2024-04-24T22:45:46.219 INFO:tasks.workunit.client.0.smithi063.stdout:7/996: chown d0/d2f 182229710 1 2024-04-24T22:45:46.221 INFO:tasks.workunit.client.0.smithi063.stdout:0/988: rename d6/da to d6/d64/da0/d98/d11b/d14c 0 2024-04-24T22:45:46.221 INFO:tasks.workunit.client.1.smithi192.stdout:3/850: dwrite d2/d27/d28/f30 [0,4194304] 0 2024-04-24T22:45:46.223 INFO:tasks.workunit.client.0.smithi063.stdout:0/989: write d6/d64/da0/d98/d11b/d14c/f10 [1481308,30087] 0 2024-04-24T22:45:46.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/997: mkdir d0/d4/de/d17/dd7/d14d 0 2024-04-24T22:45:46.230 INFO:tasks.workunit.client.1.smithi192.stdout:5/722: write d0/d13/f3f [1218809,50332] 0 2024-04-24T22:45:46.231 INFO:tasks.workunit.client.0.smithi063.stdout:7/998: unlink d0/d4/de/d6f/d5c/d8b/fe5 0 2024-04-24T22:45:46.231 INFO:tasks.workunit.client.0.smithi063.stdout:0/990: dwrite d6/d64/d21/d25/d62/d63/f6e [0,4194304] 0 2024-04-24T22:45:46.232 INFO:tasks.workunit.client.1.smithi192.stdout:5/723: write d0/d13/d19/d83/dad/fd8 [792520,54240] 0 2024-04-24T22:45:46.233 INFO:tasks.workunit.client.1.smithi192.stdout:0/854: link d3/d27/d28/d31/f85 d3/d27/d28/dcc/d5c/d8a/f11a 0 2024-04-24T22:45:46.235 INFO:tasks.workunit.client.1.smithi192.stdout:5/724: read d0/f4a [3814727,126020] 0 2024-04-24T22:45:46.235 INFO:tasks.workunit.client.1.smithi192.stdout:3/851: mknod d2/d4/c10d 0 2024-04-24T22:45:46.236 INFO:tasks.workunit.client.0.smithi063.stdout:7/999: dread - d0/d4/de/d17/d35/d69/db1/fc5 zero size 2024-04-24T22:45:46.238 INFO:tasks.workunit.client.0.smithi063.stdout:0/991: creat d6/d64/da0/d33/deb/f14d x:0 0 0 2024-04-24T22:45:46.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/992: dread - d6/d64/d21/d25/d62/d75/f13d zero size 2024-04-24T22:45:46.241 INFO:tasks.workunit.client.1.smithi192.stdout:6/772: dread d0/d8/d22/f32 [4194304,4194304] 0 2024-04-24T22:45:46.241 INFO:tasks.workunit.client.1.smithi192.stdout:7/996: write d2/db/d36/f64 [536314,129908] 0 2024-04-24T22:45:46.244 INFO:tasks.workunit.client.1.smithi192.stdout:0/855: mkdir d3/d27/d28/dcc/d5c/d11b 0 2024-04-24T22:45:46.246 INFO:tasks.workunit.client.0.smithi063.stdout:0/993: creat d6/d64/d24/d42/dad/d126/f14e x:0 0 0 2024-04-24T22:45:46.247 INFO:tasks.workunit.client.0.smithi063.stdout:0/994: readlink d6/d64/d24/l46 0 2024-04-24T22:45:46.247 INFO:tasks.workunit.client.1.smithi192.stdout:4/778: dwrite d2/da5/fcb [0,4194304] 0 2024-04-24T22:45:46.248 INFO:tasks.workunit.client.1.smithi192.stdout:6/773: chown d0/d8/d29/d4c/d4d/d83/fd3 223 1 2024-04-24T22:45:46.253 INFO:tasks.workunit.client.0.smithi063.stdout:0/995: rename d6/d64/d21/d25/d62/d63/d89/d95 to d6/d64/d21/d80/d14f 0 2024-04-24T22:45:46.254 INFO:tasks.workunit.client.1.smithi192.stdout:6/774: rmdir d0/d8/d18/df1 39 2024-04-24T22:45:46.255 INFO:tasks.workunit.client.1.smithi192.stdout:6/775: readlink d0/d8/d22/dcb/ld5 0 2024-04-24T22:45:46.256 INFO:tasks.workunit.client.1.smithi192.stdout:4/779: fdatasync d2/db/d75/da1/d36/d5b/d64/f91 0 2024-04-24T22:45:46.261 INFO:tasks.workunit.client.1.smithi192.stdout:6/776: mkdir d0/d8/d3d/d9d/da5/dc8/d112 0 2024-04-24T22:45:46.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/856: link d3/c1f d3/d27/d28/dcc/da8/c11c 0 2024-04-24T22:45:46.266 INFO:tasks.workunit.client.1.smithi192.stdout:7/997: rename d2/l1d to d2/db/d21/dd5/l132 0 2024-04-24T22:45:46.268 INFO:tasks.workunit.client.1.smithi192.stdout:7/998: write d2/db/d21/d42/d67/dae/dbb/f123 [29722,27758] 0 2024-04-24T22:45:46.271 INFO:tasks.workunit.client.1.smithi192.stdout:4/780: truncate d2/d9/d25/d2e/f66 941574 0 2024-04-24T22:45:46.278 INFO:tasks.workunit.client.1.smithi192.stdout:7/999: dwrite d2/db/d21/d42/d67/dae/dbb/f123 [0,4194304] 0 2024-04-24T22:45:46.284 INFO:tasks.workunit.client.0.smithi063.stdout:0/996: dread d6/d64/d21/f86 [0,4194304] 0 2024-04-24T22:45:46.288 INFO:tasks.workunit.client.1.smithi192.stdout:3/852: dwrite d2/d4/d12/d1a/d1d/d7b/ddb/f104 [0,4194304] 0 2024-04-24T22:45:46.289 INFO:tasks.workunit.client.0.smithi063.stdout:0/997: creat d6/d64/d24/f150 x:0 0 0 2024-04-24T22:45:46.290 INFO:tasks.workunit.client.0.smithi063.stdout:0/998: write d6/d64/d24/d42/dd4/f13e [468133,18411] 0 2024-04-24T22:45:46.291 INFO:tasks.workunit.client.1.smithi192.stdout:0/857: dread d3/f39 [0,4194304] 0 2024-04-24T22:45:46.292 INFO:tasks.workunit.client.1.smithi192.stdout:5/725: dwrite d0/d13/f40 [0,4194304] 0 2024-04-24T22:45:46.294 INFO:tasks.workunit.client.0.smithi063.stdout:0/999: creat d6/d64/d24/d42/dad/d126/f151 x:0 0 0 2024-04-24T22:45:46.295 INFO:tasks.workunit.client.1.smithi192.stdout:6/777: dwrite d0/d8/d3d/d9d/fb2 [0,4194304] 0 2024-04-24T22:45:46.295 INFO:tasks.workunit.client.1.smithi192.stdout:3/853: chown d2/d4/d12/dbc/d10a 32186691 1 2024-04-24T22:45:46.299 INFO:tasks.workunit.client.1.smithi192.stdout:0/858: creat d3/d27/d28/d2e/dc7/f11d x:0 0 0 2024-04-24T22:45:46.299 INFO:tasks.workunit.client.1.smithi192.stdout:4/781: dwrite d2/fd0 [0,4194304] 0 2024-04-24T22:45:46.301 INFO:tasks.workunit.client.0.smithi063.stderr:+ rm -rf -- ./tmp.4gY25p0yZa 2024-04-24T22:45:46.301 INFO:tasks.workunit.client.1.smithi192.stdout:5/726: rename d0/f31 to d0/d13/dd1/df1/ffc 0 2024-04-24T22:45:46.304 INFO:tasks.workunit.client.1.smithi192.stdout:3/854: unlink d2/d4/d12/d1a/d1d/l58 0 2024-04-24T22:45:46.304 INFO:tasks.workunit.client.1.smithi192.stdout:6/778: truncate d0/d8/d3d/d3e/d7e/fec 167869 0 2024-04-24T22:45:46.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/782: dread - d2/d9/d25/d2e/fde zero size 2024-04-24T22:45:46.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/783: read - d2/d5c/dce/fd8 zero size 2024-04-24T22:45:46.312 INFO:tasks.workunit.client.1.smithi192.stdout:4/784: creat d2/db/d75/da1/d7d/f10f x:0 0 0 2024-04-24T22:45:46.313 INFO:tasks.workunit.client.1.smithi192.stdout:4/785: write d2/d9/f8a [3194270,63376] 0 2024-04-24T22:45:46.314 INFO:tasks.workunit.client.1.smithi192.stdout:0/859: creat d3/d27/f11e x:0 0 0 2024-04-24T22:45:46.320 INFO:tasks.workunit.client.1.smithi192.stdout:4/786: mkdir d2/db/d75/da1/d36/d5b/d28/df6/d110 0 2024-04-24T22:45:46.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/787: readlink d2/db/d75/da1/d36/d5b/d28/lc7 0 2024-04-24T22:45:46.323 INFO:tasks.workunit.client.1.smithi192.stdout:4/788: stat d2/db/d75/d10d 0 2024-04-24T22:45:46.324 INFO:tasks.workunit.client.1.smithi192.stdout:0/860: dwrite d3/d27/d6a/deb/f105 [0,4194304] 0 2024-04-24T22:45:46.330 INFO:tasks.workunit.client.1.smithi192.stdout:4/789: dread d2/db/d75/da1/d36/d5b/d28/d2f/fb1 [0,4194304] 0 2024-04-24T22:45:46.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/861: mknod d3/d27/d3c/d6c/db3/df0/c11f 0 2024-04-24T22:45:46.341 INFO:tasks.workunit.client.1.smithi192.stdout:0/862: creat d3/d27/d28/d2e/dc7/f120 x:0 0 0 2024-04-24T22:45:46.346 INFO:tasks.workunit.client.1.smithi192.stdout:6/779: write d0/d8/d3d/d3e/f4b [1018157,74911] 0 2024-04-24T22:45:46.348 INFO:tasks.workunit.client.1.smithi192.stdout:2/933: sync 2024-04-24T22:45:46.348 INFO:tasks.workunit.client.1.smithi192.stdout:9/922: sync 2024-04-24T22:45:46.348 INFO:tasks.workunit.client.1.smithi192.stdout:8/819: sync 2024-04-24T22:45:46.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/673: sync 2024-04-24T22:45:46.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/727: dwrite d0/d13/d19/d83/fc6 [0,4194304] 0 2024-04-24T22:45:46.351 INFO:tasks.workunit.client.1.smithi192.stdout:3/855: dwrite d2/d4/d12/fd5 [0,4194304] 0 2024-04-24T22:45:46.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/856: fdatasync d2/d4/d12/f88 0 2024-04-24T22:45:46.357 INFO:tasks.workunit.client.1.smithi192.stdout:4/790: dread d2/db/d75/da1/d36/d5b/f24 [0,4194304] 0 2024-04-24T22:45:46.358 INFO:tasks.workunit.client.1.smithi192.stdout:3/857: write d2/d4/d12/d1a/d1d/d55/d5f/db1/ff8 [411702,108614] 0 2024-04-24T22:45:46.358 INFO:tasks.workunit.client.1.smithi192.stdout:1/674: dwrite d9/f1a [0,4194304] 0 2024-04-24T22:45:46.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/820: read db/dc/f1c [1085321,97917] 0 2024-04-24T22:45:46.366 INFO:tasks.workunit.client.1.smithi192.stdout:5/728: rmdir d0/d13/d19/d83 39 2024-04-24T22:45:46.372 INFO:tasks.workunit.client.1.smithi192.stdout:2/934: chown d4/d9/d10/d6c/d6f/da8/fb0 121152 1 2024-04-24T22:45:46.380 INFO:tasks.workunit.client.1.smithi192.stdout:6/780: mknod d0/d8/d18/d2e/de8/c113 0 2024-04-24T22:45:46.382 INFO:tasks.workunit.client.1.smithi192.stdout:4/791: dwrite d2/db/d75/da1/d53/fcc [0,4194304] 0 2024-04-24T22:45:46.387 INFO:tasks.workunit.client.1.smithi192.stdout:2/935: mkdir d4/d9/d37/d12f 0 2024-04-24T22:45:46.387 INFO:tasks.workunit.client.1.smithi192.stdout:0/863: dwrite d3/d27/d3c/fab [0,4194304] 0 2024-04-24T22:45:46.390 INFO:tasks.workunit.client.1.smithi192.stdout:3/858: truncate d2/f1e 4680802 0 2024-04-24T22:45:46.396 INFO:tasks.workunit.client.1.smithi192.stdout:4/792: unlink d2/d5c/dbb/ld7 0 2024-04-24T22:45:46.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/923: dread d3/dd/de/d17/f68 [0,4194304] 0 2024-04-24T22:45:46.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/729: unlink d0/d1b/c2e 0 2024-04-24T22:45:46.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/793: truncate d2/db/d75/da1/d36/d5b/d64/ff7 554976 0 2024-04-24T22:45:46.401 INFO:tasks.workunit.client.1.smithi192.stdout:1/675: write d9/f42 [4364609,3950] 0 2024-04-24T22:45:46.402 INFO:tasks.workunit.client.1.smithi192.stdout:6/781: rename d0/d8/d18/d2e/c77 to d0/d8/d3d/d3e/d40/d44/dc7/c114 0 2024-04-24T22:45:46.405 INFO:tasks.workunit.client.1.smithi192.stdout:8/821: write db/d43/d7c/fd2 [516202,6014] 0 2024-04-24T22:45:46.406 INFO:tasks.workunit.client.1.smithi192.stdout:8/822: chown db/d43/cda 2 1 2024-04-24T22:45:46.410 INFO:tasks.workunit.client.1.smithi192.stdout:5/730: fdatasync d0/d1b/d62/fa7 0 2024-04-24T22:45:46.410 INFO:tasks.workunit.client.1.smithi192.stdout:5/731: chown d0/d1b/d50 2766 1 2024-04-24T22:45:46.411 INFO:tasks.workunit.client.1.smithi192.stdout:4/794: rmdir d2/d5c 39 2024-04-24T22:45:46.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/676: mknod d9/d15/d5e/ce4 0 2024-04-24T22:45:46.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/677: dread - d9/d15/d2d/d48/d93/da4/fc9 zero size 2024-04-24T22:45:46.418 INFO:tasks.workunit.client.1.smithi192.stdout:0/864: symlink d3/l121 0 2024-04-24T22:45:46.422 INFO:tasks.workunit.client.1.smithi192.stdout:3/859: symlink d2/d4/d12/d39/def/l10e 0 2024-04-24T22:45:46.424 INFO:tasks.workunit.client.1.smithi192.stdout:0/865: dwrite d3/d27/d28/fc3 [0,4194304] 0 2024-04-24T22:45:46.430 INFO:tasks.workunit.client.1.smithi192.stdout:9/924: write d3/d24/f44 [2055441,38754] 0 2024-04-24T22:45:46.431 INFO:tasks.workunit.client.1.smithi192.stdout:5/732: unlink d0/d13/d19/d64/d5f/fc4 0 2024-04-24T22:45:46.433 INFO:tasks.workunit.client.1.smithi192.stdout:1/678: symlink d9/d15/d5e/le5 0 2024-04-24T22:45:46.434 INFO:tasks.workunit.client.1.smithi192.stdout:1/679: chown d9/d15/d5e/d90/da9/ddd 3491776 1 2024-04-24T22:45:46.435 INFO:tasks.workunit.client.1.smithi192.stdout:6/782: mknod d0/d8/d3d/c115 0 2024-04-24T22:45:46.436 INFO:tasks.workunit.client.1.smithi192.stdout:2/936: link d4/d9/d10/d6c/d6f/da8/ffc d4/d9/d10/d6c/d6f/da8/f130 0 2024-04-24T22:45:46.437 INFO:tasks.workunit.client.1.smithi192.stdout:6/783: truncate d0/d8/d29/d4c/d4d/d6a/ff9 287590 0 2024-04-24T22:45:46.441 INFO:tasks.workunit.client.1.smithi192.stdout:9/925: symlink d3/dd/de/da7/l12d 0 2024-04-24T22:45:46.442 INFO:tasks.workunit.client.1.smithi192.stdout:9/926: chown d3/dd/d53/dab 36 1 2024-04-24T22:45:46.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/866: fdatasync d3/d27/d28/fc3 0 2024-04-24T22:45:46.446 INFO:tasks.workunit.client.1.smithi192.stdout:6/784: dwrite d0/d8/d29/d4c/d4d/d6a/ff9 [0,4194304] 0 2024-04-24T22:45:46.446 INFO:tasks.workunit.client.1.smithi192.stdout:4/795: symlink d2/d5c/d107/l111 0 2024-04-24T22:45:46.448 INFO:tasks.workunit.client.1.smithi192.stdout:8/823: dread db/f56 [0,4194304] 0 2024-04-24T22:45:46.448 INFO:tasks.workunit.client.1.smithi192.stdout:6/785: dread d0/f7f [0,4194304] 0 2024-04-24T22:45:46.451 INFO:tasks.workunit.client.1.smithi192.stdout:6/786: dread d0/d8/d3d/d3e/d40/fcf [0,4194304] 0 2024-04-24T22:45:46.454 INFO:tasks.workunit.client.1.smithi192.stdout:0/867: symlink d3/d27/d3c/d3b/l122 0 2024-04-24T22:45:46.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/796: dread d2/db/f7a [0,4194304] 0 2024-04-24T22:45:46.463 INFO:tasks.workunit.client.1.smithi192.stdout:0/868: mknod d3/d27/d28/dcc/da8/d111/c123 0 2024-04-24T22:45:46.465 INFO:tasks.workunit.client.1.smithi192.stdout:9/927: dread d3/f58 [0,4194304] 0 2024-04-24T22:45:46.468 INFO:tasks.workunit.client.1.smithi192.stdout:3/860: write d2/d27/dc6/f77 [830405,4325] 0 2024-04-24T22:45:46.471 INFO:tasks.workunit.client.1.smithi192.stdout:5/733: write d0/d13/d3b/d2f/f46 [11931,20519] 0 2024-04-24T22:45:46.471 INFO:tasks.workunit.client.1.smithi192.stdout:2/937: write d4/d9/d10/f58 [2129094,112851] 0 2024-04-24T22:45:46.473 INFO:tasks.workunit.client.1.smithi192.stdout:9/928: rename d3/dd/d53/cd2 to d3/dd/d62/d6d/d91/c12e 0 2024-04-24T22:45:46.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/787: link d0/d8/d29/d4c/db0/fcc d0/d8/d3d/d3e/d40/d44/f116 0 2024-04-24T22:45:46.474 INFO:tasks.workunit.client.1.smithi192.stdout:9/929: chown d3/dd/de/d17/f68 30 1 2024-04-24T22:45:46.476 INFO:tasks.workunit.client.1.smithi192.stdout:9/930: stat d3/d5d/da1/d5e/d98/f103 0 2024-04-24T22:45:46.476 INFO:tasks.workunit.client.1.smithi192.stdout:4/797: symlink d2/db/d75/d10d/l112 0 2024-04-24T22:45:46.476 INFO:tasks.workunit.client.1.smithi192.stdout:8/824: link db/d5d/f7e db/dc/f100 0 2024-04-24T22:45:46.480 INFO:tasks.workunit.client.1.smithi192.stdout:9/931: mknod d3/dd/de/d17/d89/d8b/c12f 0 2024-04-24T22:45:46.488 INFO:tasks.workunit.client.1.smithi192.stdout:4/798: creat d2/db/d75/da1/ddf/f113 x:0 0 0 2024-04-24T22:45:46.488 INFO:tasks.workunit.client.1.smithi192.stdout:0/869: truncate d3/d27/d28/d2e/f63 3121267 0 2024-04-24T22:45:46.490 INFO:tasks.workunit.client.1.smithi192.stdout:9/932: fsync d3/d24/f97 0 2024-04-24T22:45:46.491 INFO:tasks.workunit.client.1.smithi192.stdout:0/870: truncate d3/d27/d28/dcc/ffe 987252 0 2024-04-24T22:45:46.492 INFO:tasks.workunit.client.1.smithi192.stdout:6/788: truncate d0/d8/d18/d2e/f51 1957479 0 2024-04-24T22:45:46.497 INFO:tasks.workunit.client.1.smithi192.stdout:8/825: dwrite db/dc/d22/d31/d33/d8f/fa2 [0,4194304] 0 2024-04-24T22:45:46.498 INFO:tasks.workunit.client.1.smithi192.stdout:8/826: chown db/d43/d71/cd9 27239 1 2024-04-24T22:45:46.499 INFO:tasks.workunit.client.1.smithi192.stdout:4/799: rename d2/db/d75/da1/d36/d5b/d28/lae to d2/db/d5d/dfc/l114 0 2024-04-24T22:45:46.500 INFO:tasks.workunit.client.1.smithi192.stdout:9/933: dread d3/dd/de/d17/f8f [0,4194304] 0 2024-04-24T22:45:46.500 INFO:tasks.workunit.client.1.smithi192.stdout:4/800: chown d2/db/d75/da1/f4c 45654731 1 2024-04-24T22:45:46.503 INFO:tasks.workunit.client.1.smithi192.stdout:5/734: link d0/d13/d19/f60 d0/d13/d19/d71/ffd 0 2024-04-24T22:45:46.506 INFO:tasks.workunit.client.1.smithi192.stdout:5/735: chown d0/d13/d19/d64/d5f/d2c/ldd 12399 1 2024-04-24T22:45:46.507 INFO:tasks.workunit.client.1.smithi192.stdout:6/789: stat d0/d8/d3d/d9d/dd6/ddc/lf6 0 2024-04-24T22:45:46.511 INFO:tasks.workunit.client.1.smithi192.stdout:4/801: dwrite d2/da5/fcb [4194304,4194304] 0 2024-04-24T22:45:46.517 INFO:tasks.workunit.client.1.smithi192.stdout:6/790: creat d0/d8/d29/d5a/d63/f117 x:0 0 0 2024-04-24T22:45:46.519 INFO:tasks.workunit.client.1.smithi192.stdout:8/827: write db/dc/d29/d9f/fa4 [298049,106698] 0 2024-04-24T22:45:46.522 INFO:tasks.workunit.client.1.smithi192.stdout:6/791: creat d0/d8/d29/d4c/d4d/d6a/f118 x:0 0 0 2024-04-24T22:45:46.524 INFO:tasks.workunit.client.1.smithi192.stdout:4/802: creat d2/db/d5d/f115 x:0 0 0 2024-04-24T22:45:46.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/934: dread d3/dd/de/dba/f81 [0,4194304] 0 2024-04-24T22:45:46.524 INFO:tasks.workunit.client.1.smithi192.stdout:6/792: readlink d0/d8/d3d/d3e/d40/l65 0 2024-04-24T22:45:46.531 INFO:tasks.workunit.client.1.smithi192.stdout:9/935: creat d3/dd/d53/deb/f130 x:0 0 0 2024-04-24T22:45:46.533 INFO:tasks.workunit.client.1.smithi192.stdout:0/871: dread d3/d27/d28/dcc/f6b [0,4194304] 0 2024-04-24T22:45:46.533 INFO:tasks.workunit.client.1.smithi192.stdout:8/828: dread db/dc/d22/d31/f4d [0,4194304] 0 2024-04-24T22:45:46.535 INFO:tasks.workunit.client.1.smithi192.stdout:5/736: dread d0/d13/d19/f1d [0,4194304] 0 2024-04-24T22:45:46.537 INFO:tasks.workunit.client.1.smithi192.stdout:0/872: mkdir d3/d27/d28/d2e/dc7/d124 0 2024-04-24T22:45:46.537 INFO:tasks.workunit.client.1.smithi192.stdout:0/873: chown d3/f39 11937 1 2024-04-24T22:45:46.540 INFO:tasks.workunit.client.1.smithi192.stdout:8/829: symlink db/d43/d7c/d85/l101 0 2024-04-24T22:45:46.540 INFO:tasks.workunit.client.1.smithi192.stdout:6/793: write d0/d8/d29/d58/f9f [988103,49716] 0 2024-04-24T22:45:46.541 INFO:tasks.workunit.client.1.smithi192.stdout:9/936: mkdir d3/dd/de/d17/d89/db4/db9/d10a/d131 0 2024-04-24T22:45:46.543 INFO:tasks.workunit.client.1.smithi192.stdout:8/830: dread db/dc/d29/d9f/fa4 [0,4194304] 0 2024-04-24T22:45:46.544 INFO:tasks.workunit.client.1.smithi192.stdout:8/831: chown db/dc/d29/f89 9715 1 2024-04-24T22:45:46.545 INFO:tasks.workunit.client.1.smithi192.stdout:8/832: truncate db/d5d/d74/fea 703756 0 2024-04-24T22:45:46.547 INFO:tasks.workunit.client.1.smithi192.stdout:8/833: write db/dc/d22/d31/d33/d81/d8a/ffe [878959,41474] 0 2024-04-24T22:45:46.548 INFO:tasks.workunit.client.1.smithi192.stdout:8/834: stat db/dc/d29/d9f/cce 0 2024-04-24T22:45:46.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/937: rmdir d3/dd 39 2024-04-24T22:45:46.557 INFO:tasks.workunit.client.1.smithi192.stdout:6/794: creat d0/d8/d18/dff/f119 x:0 0 0 2024-04-24T22:45:46.560 INFO:tasks.workunit.client.1.smithi192.stdout:5/737: dwrite d0/d13/d19/d64/d5f/d2c/f3d [0,4194304] 0 2024-04-24T22:45:46.562 INFO:tasks.workunit.client.1.smithi192.stdout:0/874: dwrite d3/f39 [0,4194304] 0 2024-04-24T22:45:46.563 INFO:tasks.workunit.client.1.smithi192.stdout:9/938: dwrite d3/d5d/da1/f70 [0,4194304] 0 2024-04-24T22:45:46.565 INFO:tasks.workunit.client.1.smithi192.stdout:6/795: creat d0/d8/d3d/d9d/dd6/ddc/f11a x:0 0 0 2024-04-24T22:45:46.571 INFO:tasks.workunit.client.1.smithi192.stdout:9/939: unlink d3/d59/f113 0 2024-04-24T22:45:46.573 INFO:tasks.workunit.client.1.smithi192.stdout:6/796: symlink d0/d8/d29/d5a/l11b 0 2024-04-24T22:45:46.573 INFO:tasks.workunit.client.1.smithi192.stdout:6/797: chown d0/d8/d29/d5a/ddf 111958385 1 2024-04-24T22:45:46.575 INFO:tasks.workunit.client.1.smithi192.stdout:5/738: rename d0/d1b/fc7 to d0/d13/ffe 0 2024-04-24T22:45:46.576 INFO:tasks.workunit.client.1.smithi192.stdout:9/940: fsync d3/dd/de/d17/f68 0 2024-04-24T22:45:46.580 INFO:tasks.workunit.client.1.smithi192.stdout:5/739: rename d0/d13/d19/d71/d7f/d9a/ff3 to d0/d13/d19/d64/d5f/d2c/fff 0 2024-04-24T22:45:46.581 INFO:tasks.workunit.client.1.smithi192.stdout:6/798: link d0/d8/d3d/d3e/d7e/cd4 d0/d8/d29/d4c/d4d/d83/dde/c11c 0 2024-04-24T22:45:46.583 INFO:tasks.workunit.client.1.smithi192.stdout:9/941: link d3/dd/d62/f92 d3/d59/d7d/d9b/dd1/f132 0 2024-04-24T22:45:46.586 INFO:tasks.workunit.client.1.smithi192.stdout:9/942: fdatasync d3/dd/de/da7/d116/fff 0 2024-04-24T22:45:46.587 INFO:tasks.workunit.client.1.smithi192.stdout:6/799: dwrite d0/d8/d22/dcb/de0/fe4 [0,4194304] 0 2024-04-24T22:45:46.588 INFO:tasks.workunit.client.1.smithi192.stdout:6/800: read - d0/d8/d29/d4c/d4d/d83/fd7 zero size 2024-04-24T22:45:46.611 INFO:tasks.workunit.client.1.smithi192.stdout:6/801: chown d0/d8/d3d/d3e/d40/l64 62 1 2024-04-24T22:45:46.611 INFO:tasks.workunit.client.1.smithi192.stdout:9/943: mknod d3/d59/d7d/c133 0 2024-04-24T22:45:46.612 INFO:tasks.workunit.client.1.smithi192.stdout:9/944: readlink d3/dd/d62/d6d/d91/l9a 0 2024-04-24T22:45:46.613 INFO:tasks.workunit.client.1.smithi192.stdout:6/802: mknod d0/d8/d3d/d3e/d40/d44/dc7/c11d 0 2024-04-24T22:45:46.615 INFO:tasks.workunit.client.1.smithi192.stdout:9/945: creat d3/dd/de/da7/d11f/f134 x:0 0 0 2024-04-24T22:45:46.616 INFO:tasks.workunit.client.1.smithi192.stdout:6/803: creat d0/d8/d29/d5a/d63/f11e x:0 0 0 2024-04-24T22:45:46.619 INFO:tasks.workunit.client.1.smithi192.stdout:6/804: dread d0/d8/d3d/d3e/d40/fcf [0,4194304] 0 2024-04-24T22:45:46.620 INFO:tasks.workunit.client.1.smithi192.stdout:0/875: write d3/d27/d28/d2e/f4c [3217373,68488] 0 2024-04-24T22:45:46.621 INFO:tasks.workunit.client.1.smithi192.stdout:6/805: symlink d0/d8/d22/dcb/de0/l11f 0 2024-04-24T22:45:46.626 INFO:tasks.workunit.client.1.smithi192.stdout:0/876: rmdir d3/d27/d28/dcc/d5c/d80/d83/dfb 39 2024-04-24T22:45:46.628 INFO:tasks.workunit.client.1.smithi192.stdout:6/806: creat d0/d8/d3d/d3e/d7e/f120 x:0 0 0 2024-04-24T22:45:46.632 INFO:tasks.workunit.client.1.smithi192.stdout:6/807: creat d0/d8/d3d/d3e/d40/d44/dc7/f121 x:0 0 0 2024-04-24T22:45:46.635 INFO:tasks.workunit.client.1.smithi192.stdout:6/808: dread - d0/d8/d3d/d3e/fac zero size 2024-04-24T22:45:46.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/740: dwrite d0/d1b/f6d [0,4194304] 0 2024-04-24T22:45:46.645 INFO:tasks.workunit.client.1.smithi192.stdout:0/877: dread d3/fc0 [0,4194304] 0 2024-04-24T22:45:46.649 INFO:tasks.workunit.client.1.smithi192.stdout:9/946: dwrite d3/d59/d7d/d9b/dd1/f105 [0,4194304] 0 2024-04-24T22:45:46.649 INFO:tasks.workunit.client.1.smithi192.stdout:6/809: link d0/d8/d29/f3b d0/d8/d29/f122 0 2024-04-24T22:45:46.650 INFO:tasks.workunit.client.1.smithi192.stdout:9/947: stat d3/dd/d53/deb 0 2024-04-24T22:45:46.656 INFO:tasks.workunit.client.1.smithi192.stdout:0/878: creat d3/d27/d3c/d6c/f125 x:0 0 0 2024-04-24T22:45:46.658 INFO:tasks.workunit.client.1.smithi192.stdout:0/879: write d3/d27/d28/d31/dbc/dea/f101 [529736,122400] 0 2024-04-24T22:45:46.663 INFO:tasks.workunit.client.1.smithi192.stdout:0/880: creat d3/d27/d3c/d3b/f126 x:0 0 0 2024-04-24T22:45:46.665 INFO:tasks.workunit.client.1.smithi192.stdout:5/741: dwrite d0/d13/d19/f57 [0,4194304] 0 2024-04-24T22:45:46.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/810: truncate d0/d8/d3d/d9d/fd0 1823692 0 2024-04-24T22:45:46.675 INFO:tasks.workunit.client.1.smithi192.stdout:9/948: dwrite d3/dd/f6c [0,4194304] 0 2024-04-24T22:45:46.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/742: fsync d0/d13/f39 0 2024-04-24T22:45:46.678 INFO:tasks.workunit.client.1.smithi192.stdout:0/881: link d3/d27/d28/fc3 d3/d27/d28/d31/dbc/dea/f127 0 2024-04-24T22:45:46.679 INFO:tasks.workunit.client.1.smithi192.stdout:6/811: creat d0/d8/d22/dcb/de0/de7/f123 x:0 0 0 2024-04-24T22:45:46.686 INFO:tasks.workunit.client.1.smithi192.stdout:0/882: dread d3/d6d/d79/f8b [0,4194304] 0 2024-04-24T22:45:46.694 INFO:tasks.workunit.client.1.smithi192.stdout:0/883: unlink d3/d27/d28/d2e/dc7/f120 0 2024-04-24T22:45:46.695 INFO:tasks.workunit.client.1.smithi192.stdout:5/743: rename d0/d13/d53/f94 to d0/d13/d19/d71/dcf/f100 0 2024-04-24T22:45:46.695 INFO:tasks.workunit.client.1.smithi192.stdout:0/884: chown d3/l66 825 1 2024-04-24T22:45:46.697 INFO:tasks.workunit.client.1.smithi192.stdout:9/949: dwrite d3/dd/d62/d6d/fc5 [0,4194304] 0 2024-04-24T22:45:46.699 INFO:tasks.workunit.client.1.smithi192.stdout:9/950: write d3/d24/f44 [3641877,98523] 0 2024-04-24T22:45:46.701 INFO:tasks.workunit.client.1.smithi192.stdout:0/885: getdents d3/d27/d28/d2e/dc7/d124 0 2024-04-24T22:45:46.702 INFO:tasks.workunit.client.1.smithi192.stdout:6/812: write d0/d8/f2a [1390589,17401] 0 2024-04-24T22:45:46.704 INFO:tasks.workunit.client.1.smithi192.stdout:5/744: read d0/d13/d3b/d2f/f84 [302001,42209] 0 2024-04-24T22:45:46.704 INFO:tasks.workunit.client.1.smithi192.stdout:0/886: chown d3/d27/d28/dcc/d5c/d80/d83/lf8 43 1 2024-04-24T22:45:46.705 INFO:tasks.workunit.client.1.smithi192.stdout:9/951: dread d3/d5d/da1/f70 [0,4194304] 0 2024-04-24T22:45:46.708 INFO:tasks.workunit.client.1.smithi192.stdout:0/887: read d3/d27/d28/d2e/dc7/fd0 [12833,17727] 0 2024-04-24T22:45:46.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/952: creat d3/dd/de/da7/d116/ddb/f135 x:0 0 0 2024-04-24T22:45:46.709 INFO:tasks.workunit.client.1.smithi192.stdout:0/888: write d3/d27/d28/dcc/f4d [55936,61705] 0 2024-04-24T22:45:46.710 INFO:tasks.workunit.client.1.smithi192.stdout:6/813: link d0/d8/d29/d5a/d63/fc9 d0/d8/d18/d2e/de8/f124 0 2024-04-24T22:45:46.712 INFO:tasks.workunit.client.1.smithi192.stdout:6/814: fsync d0/d8/d3d/d3e/fea 0 2024-04-24T22:45:46.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/889: read d3/d27/d28/f20 [1953344,89015] 0 2024-04-24T22:45:46.720 INFO:tasks.workunit.client.1.smithi192.stdout:9/953: dwrite d3/d59/d7d/d9b/dd1/f105 [4194304,4194304] 0 2024-04-24T22:45:46.721 INFO:tasks.workunit.client.1.smithi192.stdout:0/890: creat d3/d27/d28/d2e/dc7/d124/f128 x:0 0 0 2024-04-24T22:45:46.731 INFO:tasks.workunit.client.1.smithi192.stdout:6/815: write d0/d8/d3d/d3e/d40/d44/f116 [542735,108410] 0 2024-04-24T22:45:46.734 INFO:tasks.workunit.client.1.smithi192.stdout:6/816: creat d0/d8/d3d/d3e/d7e/f125 x:0 0 0 2024-04-24T22:45:46.735 INFO:tasks.workunit.client.1.smithi192.stdout:0/891: dread d3/d27/d28/dcc/fa1 [0,4194304] 0 2024-04-24T22:45:46.737 INFO:tasks.workunit.client.1.smithi192.stdout:6/817: creat d0/d8/d3d/d3e/d40/d44/f126 x:0 0 0 2024-04-24T22:45:46.738 INFO:tasks.workunit.client.1.smithi192.stdout:6/818: dread - d0/d8/d3d/d9d/f107 zero size 2024-04-24T22:45:46.740 INFO:tasks.workunit.client.1.smithi192.stdout:6/819: chown d0/d8/d29/d4c/d4d/d83/dde/l10c 50 1 2024-04-24T22:45:46.742 INFO:tasks.workunit.client.1.smithi192.stdout:0/892: dwrite d3/d27/d3c/d6c/f125 [0,4194304] 0 2024-04-24T22:45:46.745 INFO:tasks.workunit.client.1.smithi192.stdout:6/820: dread d0/d8/d29/d4c/d4d/f56 [0,4194304] 0 2024-04-24T22:45:46.747 INFO:tasks.workunit.client.1.smithi192.stdout:6/821: truncate d0/d8/d3d/d3e/d40/d44/fe3 392517 0 2024-04-24T22:45:46.751 INFO:tasks.workunit.client.1.smithi192.stdout:6/822: mkdir d0/d8/d3d/dc6/d127 0 2024-04-24T22:45:46.753 INFO:tasks.workunit.client.1.smithi192.stdout:6/823: truncate d0/f82 908517 0 2024-04-24T22:45:46.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/824: chown d0/d8/d3d/d9d/da5/dc8/d112 7205070 1 2024-04-24T22:45:46.759 INFO:tasks.workunit.client.1.smithi192.stdout:6/825: dwrite d0/d8/d3d/d3e/d7e/fec [0,4194304] 0 2024-04-24T22:45:46.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/826: rename d0/d8/d18/l2f to d0/d8/d29/d4c/l128 0 2024-04-24T22:45:46.806 INFO:tasks.workunit.client.1.smithi192.stdout:6/827: rename d0/d8/d3d/d3e/d40/l64 to d0/d8/d3d/d3e/d40/l129 0 2024-04-24T22:45:46.813 INFO:tasks.workunit.client.1.smithi192.stdout:6/828: mkdir d0/d8/d18/d12a 0 2024-04-24T22:45:46.814 INFO:tasks.workunit.client.1.smithi192.stdout:6/829: readlink d0/d8/d29/d5a/d63/leb 0 2024-04-24T22:45:46.816 INFO:tasks.workunit.client.1.smithi192.stdout:6/830: mknod d0/d8/d29/d5a/d63/c12b 0 2024-04-24T22:45:46.819 INFO:tasks.workunit.client.1.smithi192.stdout:6/831: rename d0/d8/d22/f9a to d0/d8/d3d/dc6/f12c 0 2024-04-24T22:45:46.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/832: dwrite d0/d8/d29/d5a/fb6 [0,4194304] 0 2024-04-24T22:45:47.005 INFO:tasks.workunit.client.1.smithi192.stdout:1/680: sync 2024-04-24T22:45:47.011 INFO:tasks.workunit.client.1.smithi192.stdout:1/681: dread d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:47.011 INFO:tasks.workunit.client.1.smithi192.stdout:1/682: stat d9/f4f 0 2024-04-24T22:45:47.013 INFO:tasks.workunit.client.1.smithi192.stdout:1/683: dread d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:47.298 INFO:tasks.workunit.client.1.smithi192.stdout:3/861: sync 2024-04-24T22:45:47.299 INFO:tasks.workunit.client.1.smithi192.stdout:3/862: fsync d2/d4/d12/d1a/d1d/d55/f24 0 2024-04-24T22:45:47.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:46 smithi063 ceph-mon[26912]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 101 MiB/s rd, 220 MiB/s wr, 552 op/s 2024-04-24T22:45:47.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/863: dread d2/d4/fd2 [0,4194304] 0 2024-04-24T22:45:47.313 INFO:tasks.workunit.client.1.smithi192.stdout:3/864: write d2/d27/dc6/d4f/f7a [3352376,9151] 0 2024-04-24T22:45:47.318 INFO:tasks.workunit.client.1.smithi192.stdout:3/865: dread d2/d27/d28/f36 [0,4194304] 0 2024-04-24T22:45:47.326 INFO:tasks.workunit.client.1.smithi192.stdout:3/866: getdents d2/d27/dc6/d4f 0 2024-04-24T22:45:47.327 INFO:tasks.workunit.client.1.smithi192.stdout:3/867: write d2/d27/fdc [307412,116094] 0 2024-04-24T22:45:47.335 INFO:tasks.workunit.client.1.smithi192.stdout:3/868: fsync d2/d27/dc6/d41/dac/fe6 0 2024-04-24T22:45:47.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/869: rename d2/d27/d28/da5 to d2/d27/d28/d38/d10f 0 2024-04-24T22:45:47.346 INFO:tasks.workunit.client.1.smithi192.stdout:3/870: getdents d2/d27/dc6/d41/dac 0 2024-04-24T22:45:47.350 INFO:tasks.workunit.client.1.smithi192.stdout:2/938: sync 2024-04-24T22:45:47.350 INFO:tasks.workunit.client.1.smithi192.stdout:4/803: sync 2024-04-24T22:45:47.350 INFO:tasks.workunit.client.1.smithi192.stdout:8/835: sync 2024-04-24T22:45:47.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/836: dread db/d75/fc9 [0,4194304] 0 2024-04-24T22:45:47.352 INFO:tasks.workunit.client.1.smithi192.stdout:8/837: chown db/dc/ld 1952574065 1 2024-04-24T22:45:47.355 INFO:tasks.workunit.client.1.smithi192.stdout:4/804: dwrite d2/d9/f8a [0,4194304] 0 2024-04-24T22:45:47.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/871: creat d2/d4/d12/d1a/d1d/d55/d5f/f110 x:0 0 0 2024-04-24T22:45:47.360 INFO:tasks.workunit.client.1.smithi192.stdout:2/939: symlink d4/d9/d37/l131 0 2024-04-24T22:45:47.363 INFO:tasks.workunit.client.1.smithi192.stdout:4/805: symlink d2/db/d75/da1/d36/d5b/d28/l116 0 2024-04-24T22:45:47.364 INFO:tasks.workunit.client.1.smithi192.stdout:3/872: truncate d2/d4/d12/f1c 1170633 0 2024-04-24T22:45:47.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/873: readlink d2/d27/d28/d38/l64 0 2024-04-24T22:45:47.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/940: stat d4/f41 0 2024-04-24T22:45:47.371 INFO:tasks.workunit.client.1.smithi192.stdout:4/806: creat d2/db/d5d/d108/f117 x:0 0 0 2024-04-24T22:45:47.373 INFO:tasks.workunit.client.1.smithi192.stdout:3/874: symlink d2/d27/d28/dab/l111 0 2024-04-24T22:45:47.374 INFO:tasks.workunit.client.1.smithi192.stdout:3/875: chown d2/d4/d12/d1a/d1d/d55/d5f/db1/ccc 1 1 2024-04-24T22:45:47.378 INFO:tasks.workunit.client.1.smithi192.stdout:3/876: read - d2/d27/fe7 zero size 2024-04-24T22:45:47.380 INFO:tasks.workunit.client.1.smithi192.stdout:8/838: write db/f5a [3941713,38414] 0 2024-04-24T22:45:47.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/839: dread - db/dc/d22/d31/d33/d8f/fe2 zero size 2024-04-24T22:45:47.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/840: fdatasync db/dc/d29/f47 0 2024-04-24T22:45:47.385 INFO:tasks.workunit.client.1.smithi192.stdout:2/941: dread d4/d9/d37/f4e [0,4194304] 0 2024-04-24T22:45:47.387 INFO:tasks.workunit.client.1.smithi192.stdout:4/807: link d2/db/d75/da1/d53/lbc d2/d9/d25/l118 0 2024-04-24T22:45:47.391 INFO:tasks.workunit.client.1.smithi192.stdout:2/942: unlink d4/d9/d10/d6c/d95/db7/dbf/fd9 0 2024-04-24T22:45:47.391 INFO:tasks.workunit.client.1.smithi192.stdout:2/943: readlink d4/d9/d37/l3a 0 2024-04-24T22:45:47.407 INFO:tasks.workunit.client.1.smithi192.stdout:2/944: fdatasync d4/d9/d37/f3e 0 2024-04-24T22:45:47.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/841: dread db/dc/f8e [0,4194304] 0 2024-04-24T22:45:47.416 INFO:tasks.workunit.client.1.smithi192.stdout:2/945: truncate d4/d9/d10/d6c/d6f/da8/f130 24322 0 2024-04-24T22:45:47.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/842: mknod db/ddf/c102 0 2024-04-24T22:45:47.419 INFO:tasks.workunit.client.1.smithi192.stdout:8/843: rename db/d5d/dcb/fef to db/d43/d76/f103 0 2024-04-24T22:45:47.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:46 smithi192 ceph-mon[33991]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 101 MiB/s rd, 220 MiB/s wr, 552 op/s 2024-04-24T22:45:47.428 INFO:tasks.workunit.client.1.smithi192.stdout:8/844: getdents db/dc/d22/da3 0 2024-04-24T22:45:47.429 INFO:tasks.workunit.client.1.smithi192.stdout:8/845: dread - db/dc/d22/d31/d33/d8f/fe7 zero size 2024-04-24T22:45:47.444 INFO:tasks.workunit.client.1.smithi192.stdout:4/808: dwrite d2/d9/d25/d2e/f66 [0,4194304] 0 2024-04-24T22:45:47.455 INFO:tasks.workunit.client.1.smithi192.stdout:8/846: dwrite db/dc/d22/f66 [0,4194304] 0 2024-04-24T22:45:47.459 INFO:tasks.workunit.client.1.smithi192.stdout:8/847: creat db/dc/d29/f104 x:0 0 0 2024-04-24T22:45:47.460 INFO:tasks.workunit.client.1.smithi192.stdout:8/848: write f8 [1955332,51478] 0 2024-04-24T22:45:47.463 INFO:tasks.workunit.client.1.smithi192.stdout:4/809: dwrite d2/db/d75/da1/d36/d5b/f6e [0,4194304] 0 2024-04-24T22:45:47.465 INFO:tasks.workunit.client.1.smithi192.stdout:4/810: mknod d2/da5/c119 0 2024-04-24T22:45:47.470 INFO:tasks.workunit.client.1.smithi192.stdout:4/811: rename d2/d9/d25/fe3 to d2/db/d75/da1/d36/d70/f11a 0 2024-04-24T22:45:47.471 INFO:tasks.workunit.client.1.smithi192.stdout:4/812: write d2/d9/f8a [2512687,73447] 0 2024-04-24T22:45:47.472 INFO:tasks.workunit.client.1.smithi192.stdout:8/849: write db/f56 [138823,94349] 0 2024-04-24T22:45:47.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/813: mknod d2/db/d75/da1/d36/c11b 0 2024-04-24T22:45:47.477 INFO:tasks.workunit.client.1.smithi192.stdout:8/850: write db/dc/d29/f86 [1073798,40466] 0 2024-04-24T22:45:47.479 INFO:tasks.workunit.client.1.smithi192.stdout:8/851: creat db/d5d/dcb/f105 x:0 0 0 2024-04-24T22:45:47.479 INFO:tasks.workunit.client.1.smithi192.stdout:4/814: truncate d2/db/d75/da1/d36/d5b/f37 2309772 0 2024-04-24T22:45:47.482 INFO:tasks.workunit.client.1.smithi192.stdout:4/815: creat d2/db/d5d/d77/f11c x:0 0 0 2024-04-24T22:45:47.483 INFO:tasks.workunit.client.1.smithi192.stdout:4/816: stat d2/db/d10/led 0 2024-04-24T22:45:47.485 INFO:tasks.workunit.client.1.smithi192.stdout:4/817: dread - d2/db/d75/da1/d7d/d9c/fe2 zero size 2024-04-24T22:45:47.486 INFO:tasks.workunit.client.1.smithi192.stdout:8/852: dwrite db/d43/d76/f103 [0,4194304] 0 2024-04-24T22:45:47.488 INFO:tasks.workunit.client.1.smithi192.stdout:4/818: getdents d2/db/d75/da1/d36/dfe 0 2024-04-24T22:45:47.491 INFO:tasks.workunit.client.1.smithi192.stdout:4/819: rmdir d2/db/d75/da1/d7d/d9c 39 2024-04-24T22:45:47.492 INFO:tasks.workunit.client.1.smithi192.stdout:8/853: dwrite db/ffb [0,4194304] 0 2024-04-24T22:45:47.494 INFO:tasks.workunit.client.1.smithi192.stdout:4/820: readlink d2/db/d75/da1/d36/d70/ld9 0 2024-04-24T22:45:47.497 INFO:tasks.workunit.client.1.smithi192.stdout:8/854: creat db/dc/d22/d31/d33/d81/f106 x:0 0 0 2024-04-24T22:45:47.501 INFO:tasks.workunit.client.1.smithi192.stdout:8/855: creat db/dc/d22/d31/d33/db7/f107 x:0 0 0 2024-04-24T22:45:47.505 INFO:tasks.workunit.client.1.smithi192.stdout:8/856: creat db/dc/d22/d31/f108 x:0 0 0 2024-04-24T22:45:47.508 INFO:tasks.workunit.client.1.smithi192.stdout:8/857: unlink db/dc/d22/dca/fe5 0 2024-04-24T22:45:47.510 INFO:tasks.workunit.client.1.smithi192.stdout:8/858: creat db/dc/d29/f109 x:0 0 0 2024-04-24T22:45:47.511 INFO:tasks.workunit.client.1.smithi192.stdout:5/745: sync 2024-04-24T22:45:47.511 INFO:tasks.workunit.client.1.smithi192.stdout:0/893: sync 2024-04-24T22:45:47.511 INFO:tasks.workunit.client.1.smithi192.stdout:9/954: sync 2024-04-24T22:45:47.511 INFO:tasks.workunit.client.1.smithi192.stdout:6/833: sync 2024-04-24T22:45:47.512 INFO:tasks.workunit.client.1.smithi192.stdout:8/859: rmdir db/d43/d71 39 2024-04-24T22:45:47.513 INFO:tasks.workunit.client.1.smithi192.stdout:1/684: sync 2024-04-24T22:45:47.516 INFO:tasks.workunit.client.1.smithi192.stdout:1/685: chown d9/d15/d2d/d31/f62 1 1 2024-04-24T22:45:47.517 INFO:tasks.workunit.client.1.smithi192.stdout:0/894: dread d3/d27/d6a/deb/f105 [0,4194304] 0 2024-04-24T22:45:47.517 INFO:tasks.workunit.client.1.smithi192.stdout:5/746: dread d0/d13/d19/d64/d5f/d2c/f3d [0,4194304] 0 2024-04-24T22:45:47.518 INFO:tasks.workunit.client.1.smithi192.stdout:9/955: rename d3/dd/de to d3/dd/dce/d136 0 2024-04-24T22:45:47.524 INFO:tasks.workunit.client.1.smithi192.stdout:1/686: dwrite d9/d15/d2d/d31/fab [0,4194304] 0 2024-04-24T22:45:47.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/821: dwrite d2/db/d5d/d77/f92 [0,4194304] 0 2024-04-24T22:45:47.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/747: stat d0/d1b/f69 0 2024-04-24T22:45:47.537 INFO:tasks.workunit.client.1.smithi192.stdout:9/956: dread d3/dd/dce/d136/da7/fd3 [0,4194304] 0 2024-04-24T22:45:47.538 INFO:tasks.workunit.client.1.smithi192.stdout:9/957: readlink d3/dd/dce/d136/da7/dae/l119 0 2024-04-24T22:45:47.539 INFO:tasks.workunit.client.1.smithi192.stdout:8/860: creat db/d43/d7c/ded/f10a x:0 0 0 2024-04-24T22:45:47.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/687: rmdir d9/d15/d2d/d4a 39 2024-04-24T22:45:47.547 INFO:tasks.workunit.client.1.smithi192.stdout:4/822: read d2/d9/d25/f32 [348907,3823] 0 2024-04-24T22:45:47.548 INFO:tasks.workunit.client.1.smithi192.stdout:5/748: symlink d0/d1b/d50/l101 0 2024-04-24T22:45:47.549 INFO:tasks.workunit.client.1.smithi192.stdout:4/823: chown d2/db/d75/da1/d36/d5b/d28/deb 77268 1 2024-04-24T22:45:47.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/958: chown d3/dd/dce/d136/d17/d89/d8b/ca5 37 1 2024-04-24T22:45:47.555 INFO:tasks.workunit.client.1.smithi192.stdout:1/688: fsync d9/dd/d74/f7a 0 2024-04-24T22:45:47.559 INFO:tasks.workunit.client.1.smithi192.stdout:6/834: rename d0/d8/d22/l6c to d0/d8/d18/l12d 0 2024-04-24T22:45:47.560 INFO:tasks.workunit.client.1.smithi192.stdout:6/835: fsync d0/d8/f76 0 2024-04-24T22:45:47.560 INFO:tasks.workunit.client.1.smithi192.stdout:0/895: write d3/d40/f9b [2222100,15748] 0 2024-04-24T22:45:47.562 INFO:tasks.workunit.client.1.smithi192.stdout:5/749: chown d0/d1b/l26 31600 1 2024-04-24T22:45:47.563 INFO:tasks.workunit.client.1.smithi192.stdout:8/861: dread db/dc/d22/d31/d33/db7/fc7 [0,4194304] 0 2024-04-24T22:45:47.563 INFO:tasks.workunit.client.1.smithi192.stdout:5/750: stat d0/c3 0 2024-04-24T22:45:47.563 INFO:tasks.workunit.client.1.smithi192.stdout:4/824: rmdir d2/db/d75/da1/d36/d5b/d28 39 2024-04-24T22:45:47.565 INFO:tasks.workunit.client.1.smithi192.stdout:8/862: chown db/dc/ld 25 1 2024-04-24T22:45:47.572 INFO:tasks.workunit.client.1.smithi192.stdout:0/896: truncate d3/d27/d28/dcc/fbb 3192939 0 2024-04-24T22:45:47.576 INFO:tasks.workunit.client.1.smithi192.stdout:5/751: symlink d0/d13/d19/d71/dbf/l102 0 2024-04-24T22:45:47.577 INFO:tasks.workunit.client.1.smithi192.stdout:9/959: dwrite d3/f58 [0,4194304] 0 2024-04-24T22:45:47.579 INFO:tasks.workunit.client.1.smithi192.stdout:9/960: write d3/dd/dce/d136/da7/d116/ddb/f128 [771821,11745] 0 2024-04-24T22:45:47.584 INFO:tasks.workunit.client.1.smithi192.stdout:1/689: mknod d9/d15/d2d/d31/ce6 0 2024-04-24T22:45:47.585 INFO:tasks.workunit.client.1.smithi192.stdout:0/897: rmdir d3/d27/d28/dcc 39 2024-04-24T22:45:47.587 INFO:tasks.workunit.client.1.smithi192.stdout:4/825: dwrite d2/db/f52 [0,4194304] 0 2024-04-24T22:45:47.587 INFO:tasks.workunit.client.1.smithi192.stdout:6/836: mkdir d0/d8/d3d/d3e/df3/d12e 0 2024-04-24T22:45:47.589 INFO:tasks.workunit.client.1.smithi192.stdout:1/690: truncate d9/d15/d2d/d31/d2f/dae/fd9 804979 0 2024-04-24T22:45:47.593 INFO:tasks.workunit.client.1.smithi192.stdout:9/961: truncate d3/dd/dce/fec 12436 0 2024-04-24T22:45:47.596 INFO:tasks.workunit.client.1.smithi192.stdout:0/898: fdatasync d3/d27/d6a/fc9 0 2024-04-24T22:45:47.598 INFO:tasks.workunit.client.1.smithi192.stdout:0/899: fdatasync d3/d27/d28/d31/f52 0 2024-04-24T22:45:47.598 INFO:tasks.workunit.client.1.smithi192.stdout:6/837: creat d0/d8/f12f x:0 0 0 2024-04-24T22:45:47.600 INFO:tasks.workunit.client.1.smithi192.stdout:0/900: write d3/d27/d3c/fab [1343045,3110] 0 2024-04-24T22:45:47.601 INFO:tasks.workunit.client.1.smithi192.stdout:4/826: chown d2/d9/d25/c39 5 1 2024-04-24T22:45:47.611 INFO:tasks.workunit.client.1.smithi192.stdout:5/752: dread d0/d5b/f7a [0,4194304] 0 2024-04-24T22:45:47.611 INFO:tasks.workunit.client.1.smithi192.stdout:6/838: rmdir d0/d8/d18 39 2024-04-24T22:45:47.612 INFO:tasks.workunit.client.1.smithi192.stdout:8/863: truncate db/d43/d76/feb 5638469 0 2024-04-24T22:45:47.612 INFO:tasks.workunit.client.1.smithi192.stdout:6/839: read - d0/d8/d3d/d9d/dd6/ddc/f11a zero size 2024-04-24T22:45:47.622 INFO:tasks.workunit.client.1.smithi192.stdout:8/864: mknod db/d43/d76/d92/c10b 0 2024-04-24T22:45:47.622 INFO:tasks.workunit.client.1.smithi192.stdout:6/840: dread - d0/d8/fb1 zero size 2024-04-24T22:45:47.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/691: dwrite d9/d15/f49 [0,4194304] 0 2024-04-24T22:45:47.625 INFO:tasks.workunit.client.1.smithi192.stdout:9/962: dwrite d3/dd/f30 [0,4194304] 0 2024-04-24T22:45:47.628 INFO:tasks.workunit.client.1.smithi192.stdout:6/841: read d0/d8/d29/f7c [978431,101717] 0 2024-04-24T22:45:47.629 INFO:tasks.workunit.client.1.smithi192.stdout:1/692: read d9/d15/d2d/d63/fb2 [774618,92551] 0 2024-04-24T22:45:47.635 INFO:tasks.workunit.client.1.smithi192.stdout:2/946: sync 2024-04-24T22:45:47.635 INFO:tasks.workunit.client.1.smithi192.stdout:3/877: sync 2024-04-24T22:45:47.635 INFO:tasks.workunit.client.1.smithi192.stdout:8/865: creat db/dc/d29/f10c x:0 0 0 2024-04-24T22:45:47.640 INFO:tasks.workunit.client.1.smithi192.stdout:6/842: write d0/d8/d22/dcb/de0/de7/f100 [1052807,589] 0 2024-04-24T22:45:47.642 INFO:tasks.workunit.client.1.smithi192.stdout:0/901: rename d3/d27/d28/d2e/d3f/cc6 to d3/d27/d28/d2e/c129 0 2024-04-24T22:45:47.643 INFO:tasks.workunit.client.1.smithi192.stdout:1/693: chown d9/d15/d2d/d31/d2f/c64 2808729 1 2024-04-24T22:45:47.647 INFO:tasks.workunit.client.1.smithi192.stdout:8/866: mkdir db/d43/d7c/ded/d10d 0 2024-04-24T22:45:47.647 INFO:tasks.workunit.client.1.smithi192.stdout:2/947: truncate d4/d9/d10/d6c/d8c/f8f 286688 0 2024-04-24T22:45:47.650 INFO:tasks.workunit.client.1.smithi192.stdout:3/878: dwrite d2/d27/dc6/d4f/f52 [0,4194304] 0 2024-04-24T22:45:47.653 INFO:tasks.workunit.client.1.smithi192.stdout:3/879: chown d2/d62/db2/c96 6213434 1 2024-04-24T22:45:47.656 INFO:tasks.workunit.client.1.smithi192.stdout:8/867: dwrite db/dc/f5b [0,4194304] 0 2024-04-24T22:45:47.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/753: getdents d0/d13/d19/d64 0 2024-04-24T22:45:47.667 INFO:tasks.workunit.client.1.smithi192.stdout:1/694: dread d9/d15/f2e [0,4194304] 0 2024-04-24T22:45:47.672 INFO:tasks.workunit.client.1.smithi192.stdout:9/963: dwrite d3/d24/f4e [4194304,4194304] 0 2024-04-24T22:45:47.672 INFO:tasks.workunit.client.1.smithi192.stdout:4/827: rename d2/db/d75/da1/c58 to d2/db/d5d/d108/c11d 0 2024-04-24T22:45:47.673 INFO:tasks.workunit.client.1.smithi192.stdout:3/880: rmdir d2/d27/d28/dab/dca/dee 39 2024-04-24T22:45:47.675 INFO:tasks.workunit.client.1.smithi192.stdout:2/948: mknod d4/d9/d10/d6c/d8c/c132 0 2024-04-24T22:45:47.676 INFO:tasks.workunit.client.1.smithi192.stdout:5/754: fsync d0/d13/d19/d64/d5f/d2c/fa3 0 2024-04-24T22:45:47.679 INFO:tasks.workunit.client.1.smithi192.stdout:1/695: creat d9/d15/d5e/d7c/fe7 x:0 0 0 2024-04-24T22:45:47.684 INFO:tasks.workunit.client.1.smithi192.stdout:8/868: rename db/dc/d22/d31/la9 to db/d43/d7c/d85/l10e 0 2024-04-24T22:45:47.684 INFO:tasks.workunit.client.1.smithi192.stdout:1/696: read d9/d15/d5e/d67/f71 [1584270,93665] 0 2024-04-24T22:45:47.685 INFO:tasks.workunit.client.1.smithi192.stdout:0/902: write d3/d27/d6a/f87 [743364,10785] 0 2024-04-24T22:45:47.688 INFO:tasks.workunit.client.1.smithi192.stdout:0/903: write d3/d27/d6a/f87 [2833616,55425] 0 2024-04-24T22:45:47.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/881: write d2/d4/f16 [4921627,106074] 0 2024-04-24T22:45:47.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/843: write d0/d8/d18/d2e/fa0 [411134,72324] 0 2024-04-24T22:45:47.691 INFO:tasks.workunit.client.1.smithi192.stdout:2/949: truncate d4/d29/f11d 81499 0 2024-04-24T22:45:47.691 INFO:tasks.workunit.client.1.smithi192.stdout:6/844: chown d0/d8/d18/d10b 1 1 2024-04-24T22:45:47.692 INFO:tasks.workunit.client.1.smithi192.stdout:3/882: read d2/d4/d4e/ff1 [1071210,53376] 0 2024-04-24T22:45:47.694 INFO:tasks.workunit.client.1.smithi192.stdout:3/883: readlink d2/d27/dc6/da0/le1 0 2024-04-24T22:45:47.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/828: rename d2/d5c/lef to d2/da5/l11e 0 2024-04-24T22:45:47.698 INFO:tasks.workunit.client.1.smithi192.stdout:6/845: dwrite d0/d8/d18/dff/f119 [0,4194304] 0 2024-04-24T22:45:47.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/869: creat db/d5d/f10f x:0 0 0 2024-04-24T22:45:47.701 INFO:tasks.workunit.client.1.smithi192.stdout:8/870: dread - db/dc/d29/f109 zero size 2024-04-24T22:45:47.705 INFO:tasks.workunit.client.1.smithi192.stdout:1/697: chown d9/d15/d2d/c53 3912425 1 2024-04-24T22:45:47.708 INFO:tasks.workunit.client.1.smithi192.stdout:9/964: dwrite d3/f31 [4194304,4194304] 0 2024-04-24T22:45:47.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/965: readlink d3/d5d/da1/d5e/d98/la0 0 2024-04-24T22:45:47.710 INFO:tasks.workunit.client.1.smithi192.stdout:9/966: readlink d3/dd/dce/d136/da7/l12d 0 2024-04-24T22:45:47.711 INFO:tasks.workunit.client.1.smithi192.stdout:9/967: dread - d3/dd/dce/d136/da7/d116/ddb/f135 zero size 2024-04-24T22:45:47.711 INFO:tasks.workunit.client.1.smithi192.stdout:3/884: creat d2/d27/dc6/da0/f112 x:0 0 0 2024-04-24T22:45:47.714 INFO:tasks.workunit.client.1.smithi192.stdout:8/871: mkdir db/ddf/d110 0 2024-04-24T22:45:47.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/698: unlink d9/d15/d2d/d48/d93/da4/dbb/dac/fde 0 2024-04-24T22:45:47.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/968: creat d3/d24/f137 x:0 0 0 2024-04-24T22:45:47.732 INFO:tasks.workunit.client.1.smithi192.stdout:6/846: creat d0/d8/d3d/d3e/de1/f130 x:0 0 0 2024-04-24T22:45:47.737 INFO:tasks.workunit.client.1.smithi192.stdout:1/699: chown d9/d15/d2d/d31/c81 28666 1 2024-04-24T22:45:47.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/872: write db/dc/d29/f4f [2007154,53469] 0 2024-04-24T22:45:47.739 INFO:tasks.workunit.client.1.smithi192.stdout:1/700: read d9/d51/d70/f95 [332803,88878] 0 2024-04-24T22:45:47.740 INFO:tasks.workunit.client.1.smithi192.stdout:9/969: rmdir d3/dd/dce/d136/d17/d89/de1 39 2024-04-24T22:45:47.742 INFO:tasks.workunit.client.1.smithi192.stdout:0/904: rename d3/d27/d28/d2e/d3f/fc8 to d3/d27/d28/d2e/f12a 0 2024-04-24T22:45:47.745 INFO:tasks.workunit.client.1.smithi192.stdout:2/950: dwrite d4/d9/d10/d6c/d8c/f8f [0,4194304] 0 2024-04-24T22:45:47.753 INFO:tasks.workunit.client.1.smithi192.stdout:3/885: truncate d2/d27/fdc 246515 0 2024-04-24T22:45:47.755 INFO:tasks.workunit.client.1.smithi192.stdout:5/755: getdents d0/d13/d19/d71/daf 0 2024-04-24T22:45:47.759 INFO:tasks.workunit.client.1.smithi192.stdout:5/756: write d0/d13/d19/d64/d5f/d2c/d3c/d91/ff4 [812464,82423] 0 2024-04-24T22:45:47.761 INFO:tasks.workunit.client.1.smithi192.stdout:5/757: write d0/d1b/d62/ff7 [551373,88080] 0 2024-04-24T22:45:47.763 INFO:tasks.workunit.client.1.smithi192.stdout:6/847: dwrite d0/d8/d29/d4c/db0/fb5 [0,4194304] 0 2024-04-24T22:45:47.768 INFO:tasks.workunit.client.1.smithi192.stdout:9/970: truncate d3/d59/d7d/f8a 78576 0 2024-04-24T22:45:47.771 INFO:tasks.workunit.client.1.smithi192.stdout:4/829: rename d2/db/d75/da1/d36/d5b/f6e to d2/d5c/d107/f11f 0 2024-04-24T22:45:47.772 INFO:tasks.workunit.client.1.smithi192.stdout:0/905: mkdir d3/d27/d28/d2e/d3f/d12b 0 2024-04-24T22:45:47.776 INFO:tasks.workunit.client.1.smithi192.stdout:8/873: creat db/d5d/df0/f111 x:0 0 0 2024-04-24T22:45:47.777 INFO:tasks.workunit.client.1.smithi192.stdout:5/758: mknod d0/d1b/d62/dc1/c103 0 2024-04-24T22:45:47.778 INFO:tasks.workunit.client.1.smithi192.stdout:5/759: fdatasync d0/d13/d19/f4b 0 2024-04-24T22:45:47.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/848: stat d0/f82 0 2024-04-24T22:45:47.780 INFO:tasks.workunit.client.1.smithi192.stdout:6/849: chown d0/d8/f31 446274 1 2024-04-24T22:45:47.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/886: dread d2/d27/dc6/d4f/f5a [0,4194304] 0 2024-04-24T22:45:47.784 INFO:tasks.workunit.client.1.smithi192.stdout:8/874: dwrite db/dc/d22/d31/d33/d81/d8a/ffe [0,4194304] 0 2024-04-24T22:45:47.786 INFO:tasks.workunit.client.1.smithi192.stdout:8/875: write db/d43/d7c/ded/f10a [234842,128863] 0 2024-04-24T22:45:47.790 INFO:tasks.workunit.client.1.smithi192.stdout:0/906: stat d3/d27/d28/dcc/d5c/d8a/l97 0 2024-04-24T22:45:47.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/951: write d4/d9/d10/da7/df5/f12d [189574,23905] 0 2024-04-24T22:45:47.797 INFO:tasks.workunit.client.1.smithi192.stdout:1/701: link d9/f4f d9/d15/d2d/d63/d68/fe8 0 2024-04-24T22:45:47.800 INFO:tasks.workunit.client.1.smithi192.stdout:2/952: write d4/d9/d37/f9c [3816687,38411] 0 2024-04-24T22:45:47.802 INFO:tasks.workunit.client.1.smithi192.stdout:5/760: dread d0/d13/d19/d64/d5f/d2c/f51 [0,4194304] 0 2024-04-24T22:45:47.802 INFO:tasks.workunit.client.1.smithi192.stdout:9/971: dwrite d3/dd/dce/d136/d17/d89/d8b/f99 [0,4194304] 0 2024-04-24T22:45:47.804 INFO:tasks.workunit.client.1.smithi192.stdout:1/702: write d9/d15/d2d/d31/d2f/f4c [1393507,46367] 0 2024-04-24T22:45:47.804 INFO:tasks.workunit.client.1.smithi192.stdout:3/887: dread d2/d27/fdc [0,4194304] 0 2024-04-24T22:45:47.806 INFO:tasks.workunit.client.1.smithi192.stdout:4/830: mkdir d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120 0 2024-04-24T22:45:47.813 INFO:tasks.workunit.client.1.smithi192.stdout:8/876: dwrite db/d5d/f84 [0,4194304] 0 2024-04-24T22:45:47.817 INFO:tasks.workunit.client.1.smithi192.stdout:9/972: symlink d3/dd/dce/l138 0 2024-04-24T22:45:47.818 INFO:tasks.workunit.client.1.smithi192.stdout:1/703: dread - d9/d51/f9c zero size 2024-04-24T22:45:47.839 INFO:tasks.workunit.client.1.smithi192.stdout:0/907: rename d3/d27/d28/f19 to d3/d27/d28/dcc/d5c/d11b/f12c 0 2024-04-24T22:45:47.840 INFO:tasks.workunit.client.1.smithi192.stdout:0/908: fsync d3/d27/d28/d2e/dc7/d124/f128 0 2024-04-24T22:45:47.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/953: write d4/d9/d10/d6c/ffb [1004077,57927] 0 2024-04-24T22:45:47.844 INFO:tasks.workunit.client.1.smithi192.stdout:3/888: symlink d2/d4/d12/d39/d97/l113 0 2024-04-24T22:45:47.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/954: readlink d4/d29/l9e 0 2024-04-24T22:45:47.845 INFO:tasks.workunit.client.1.smithi192.stdout:6/850: creat d0/d8/f131 x:0 0 0 2024-04-24T22:45:47.848 INFO:tasks.workunit.client.1.smithi192.stdout:1/704: rmdir d9/d15/d2d/d48/d93/da4/dbb 39 2024-04-24T22:45:47.849 INFO:tasks.workunit.client.1.smithi192.stdout:0/909: dwrite d3/d27/d3c/fab [4194304,4194304] 0 2024-04-24T22:45:47.849 INFO:tasks.workunit.client.1.smithi192.stdout:5/761: write d0/d1b/f37 [1620370,14801] 0 2024-04-24T22:45:47.854 INFO:tasks.workunit.client.1.smithi192.stdout:4/831: write d2/d9/d25/d2e/fde [106481,44964] 0 2024-04-24T22:45:47.855 INFO:tasks.workunit.client.1.smithi192.stdout:3/889: rmdir d2/d27/d28/d38/d10f 39 2024-04-24T22:45:47.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/890: write d2/d27/d28/dab/dca/fcf [317357,32784] 0 2024-04-24T22:45:47.858 INFO:tasks.workunit.client.1.smithi192.stdout:6/851: mknod d0/d8/d29/d5a/c132 0 2024-04-24T22:45:47.865 INFO:tasks.workunit.client.1.smithi192.stdout:1/705: mkdir d9/d15/d2d/d48/d93/da4/dd2/dbf/de9 0 2024-04-24T22:45:47.867 INFO:tasks.workunit.client.1.smithi192.stdout:4/832: unlink d2/db/d5d/d77/d82/l102 0 2024-04-24T22:45:47.867 INFO:tasks.workunit.client.1.smithi192.stdout:4/833: chown d2/d9/f8a 0 1 2024-04-24T22:45:47.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/955: mkdir d4/d9/d10/d6c/d95/db7/dbf/d109/d133 0 2024-04-24T22:45:47.869 INFO:tasks.workunit.client.1.smithi192.stdout:3/891: creat d2/d27/dc6/d41/f114 x:0 0 0 2024-04-24T22:45:47.870 INFO:tasks.workunit.client.1.smithi192.stdout:6/852: creat d0/d8/d3d/dc6/f133 x:0 0 0 2024-04-24T22:45:47.870 INFO:tasks.workunit.client.1.smithi192.stdout:3/892: chown d2/d4/d12/d1a/d1d/cb8 14413440 1 2024-04-24T22:45:47.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/973: rename d3/d5d/da1/ce2 to d3/d59/d7d/c139 0 2024-04-24T22:45:47.874 INFO:tasks.workunit.client.1.smithi192.stdout:5/762: mknod d0/d13/d19/d71/c104 0 2024-04-24T22:45:47.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/853: creat d0/d8/d3d/d9d/da5/f134 x:0 0 0 2024-04-24T22:45:47.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/893: truncate d2/d27/dc6/da0/fce 882843 0 2024-04-24T22:45:47.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/706: rmdir d9/d15/d2d/d4a 39 2024-04-24T22:45:47.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/854: truncate d0/d8/f31 530907 0 2024-04-24T22:45:47.893 INFO:tasks.workunit.client.1.smithi192.stdout:4/834: dread d2/d5c/f99 [0,4194304] 0 2024-04-24T22:45:47.895 INFO:tasks.workunit.client.1.smithi192.stdout:4/835: chown d2/db/d75/da1/d36/d5b/d28/d2f/d7e/fa4 112244 1 2024-04-24T22:45:47.895 INFO:tasks.workunit.client.1.smithi192.stdout:0/910: getdents d3/d27/d28/dcc/d5c/d80 0 2024-04-24T22:45:47.904 INFO:tasks.workunit.client.1.smithi192.stdout:2/956: dwrite d4/d9/d10/d6c/fdb [0,4194304] 0 2024-04-24T22:45:47.904 INFO:tasks.workunit.client.1.smithi192.stdout:0/911: mkdir d3/d27/d28/d2e/d3f/d12d 0 2024-04-24T22:45:47.907 INFO:tasks.workunit.client.1.smithi192.stdout:9/974: dwrite d3/d59/ff6 [0,4194304] 0 2024-04-24T22:45:47.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/855: rename d0/d8/d3d/ca9 to d0/d8/d3d/d9d/c135 0 2024-04-24T22:45:47.916 INFO:tasks.workunit.client.1.smithi192.stdout:3/894: dwrite d2/d4/d12/d1a/f81 [0,4194304] 0 2024-04-24T22:45:47.920 INFO:tasks.workunit.client.1.smithi192.stdout:5/763: dwrite d0/d1b/f89 [0,4194304] 0 2024-04-24T22:45:47.923 INFO:tasks.workunit.client.1.smithi192.stdout:5/764: write d0/d13/f40 [2408776,35790] 0 2024-04-24T22:45:47.924 INFO:tasks.workunit.client.1.smithi192.stdout:9/975: rename d3/dd/dce/d136/dba to d3/dd/d53/dab/dcf/d13a 0 2024-04-24T22:45:47.929 INFO:tasks.workunit.client.1.smithi192.stdout:0/912: mkdir d3/d27/d28/d2e/d3f/d12d/d12e 0 2024-04-24T22:45:47.929 INFO:tasks.workunit.client.1.smithi192.stdout:2/957: mkdir d4/d9/d10/d134 0 2024-04-24T22:45:47.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/765: mkdir d0/d5b/d105 0 2024-04-24T22:45:47.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/913: fsync d3/d27/d28/dcc/ffe 0 2024-04-24T22:45:47.932 INFO:tasks.workunit.client.1.smithi192.stdout:4/836: dwrite d2/d5c/f99 [0,4194304] 0 2024-04-24T22:45:47.933 INFO:tasks.workunit.client.1.smithi192.stdout:0/914: chown d3/d27/d28/dcc/df3 46799 1 2024-04-24T22:45:47.940 INFO:tasks.workunit.client.1.smithi192.stdout:6/856: creat d0/d8/d22/dcb/de0/d10d/f136 x:0 0 0 2024-04-24T22:45:47.944 INFO:tasks.workunit.client.1.smithi192.stdout:3/895: unlink d2/d4/d12/d1a/cd4 0 2024-04-24T22:45:47.950 INFO:tasks.workunit.client.1.smithi192.stdout:5/766: mkdir d0/d13/d19/d71/d7f/d106 0 2024-04-24T22:45:47.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/837: dwrite d2/db/d5d/db3/f105 [0,4194304] 0 2024-04-24T22:45:47.965 INFO:tasks.workunit.client.1.smithi192.stdout:2/958: dread d4/d9/d37/f94 [0,4194304] 0 2024-04-24T22:45:47.967 INFO:tasks.workunit.client.1.smithi192.stdout:9/976: dwrite d3/f45 [0,4194304] 0 2024-04-24T22:45:48.199 INFO:tasks.workunit.client.1.smithi192.stdout:3/896: rmdir d2/d4/d12/d1a/d1d/d7b/df7 39 2024-04-24T22:45:48.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/897: dread - d2/d4/f10b zero size 2024-04-24T22:45:48.200 INFO:tasks.workunit.client.1.smithi192.stdout:4/838: mknod d2/d9/d6b/c121 0 2024-04-24T22:45:48.201 INFO:tasks.workunit.client.1.smithi192.stdout:3/898: chown d2/d27/d28/dab/dca/fcf 2440442 1 2024-04-24T22:45:48.205 INFO:tasks.workunit.client.1.smithi192.stdout:5/767: creat d0/d13/d19/d71/d88/f107 x:0 0 0 2024-04-24T22:45:48.206 INFO:tasks.workunit.client.1.smithi192.stdout:2/959: rename d4/d9/l72 to d4/d9/d10/d6c/d95/db7/dbf/d109/d133/l135 0 2024-04-24T22:45:48.214 INFO:tasks.workunit.client.1.smithi192.stdout:5/768: rename d0/d13/d3b/d2f to d0/d13/dd1/d108 0 2024-04-24T22:45:48.216 INFO:tasks.workunit.client.1.smithi192.stdout:5/769: truncate d0/d13/d19/d64/d5f/fd4 4921010 0 2024-04-24T22:45:48.219 INFO:tasks.workunit.client.1.smithi192.stdout:5/770: creat d0/d13/d19/d64/d4f/d7c/f109 x:0 0 0 2024-04-24T22:45:48.224 INFO:tasks.workunit.client.1.smithi192.stdout:2/960: dread d4/f27 [0,4194304] 0 2024-04-24T22:45:48.227 INFO:tasks.workunit.client.1.smithi192.stdout:2/961: write d4/d29/d60/d9a/f7b [2514384,6811] 0 2024-04-24T22:45:48.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/962: dread d4/d9/fe7 [0,4194304] 0 2024-04-24T22:45:48.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/977: write d3/d5d/fd4 [580425,130038] 0 2024-04-24T22:45:48.242 INFO:tasks.workunit.client.1.smithi192.stdout:3/899: write d2/d4/d12/d39/fd0 [72308,119472] 0 2024-04-24T22:45:48.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/978: rmdir d3/dd/dce 39 2024-04-24T22:45:48.246 INFO:tasks.workunit.client.1.smithi192.stdout:3/900: read - d2/d4/d12/d1a/d1d/d7b/fcd zero size 2024-04-24T22:45:48.266 INFO:tasks.workunit.client.1.smithi192.stdout:4/839: dwrite d2/db/d75/da1/d36/d5b/f37 [0,4194304] 0 2024-04-24T22:45:48.270 INFO:tasks.workunit.client.1.smithi192.stdout:5/771: dwrite d0/d1b/d50/f7d [0,4194304] 0 2024-04-24T22:45:48.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/901: dwrite d2/d27/dc6/d4f/f7a [0,4194304] 0 2024-04-24T22:45:48.275 INFO:tasks.workunit.client.1.smithi192.stdout:5/772: mkdir d0/d13/d19/d64/d10a 0 2024-04-24T22:45:48.276 INFO:tasks.workunit.client.1.smithi192.stdout:9/979: dwrite d3/dd/dce/d136/d17/f4f [0,4194304] 0 2024-04-24T22:45:48.277 INFO:tasks.workunit.client.1.smithi192.stdout:4/840: mkdir d2/db/d75/da1/d36/d5b/d28/df6/d110/d122 0 2024-04-24T22:45:48.281 INFO:tasks.workunit.client.1.smithi192.stdout:5/773: fdatasync d0/d13/d19/d64/d5f/d2c/d3c/d91/ff4 0 2024-04-24T22:45:48.283 INFO:tasks.workunit.client.1.smithi192.stdout:5/774: stat d0/d13/d19/fe4 0 2024-04-24T22:45:48.286 INFO:tasks.workunit.client.1.smithi192.stdout:3/902: creat d2/d2d/dfe/f115 x:0 0 0 2024-04-24T22:45:48.288 INFO:tasks.workunit.client.1.smithi192.stdout:5/775: fdatasync d0/d1b/d62/fb4 0 2024-04-24T22:45:48.289 INFO:tasks.workunit.client.1.smithi192.stdout:3/903: readlink d2/d27/dc6/d41/dac/lc4 0 2024-04-24T22:45:48.289 INFO:tasks.workunit.client.1.smithi192.stdout:9/980: dwrite d3/d59/dbf/fd9 [0,4194304] 0 2024-04-24T22:45:48.292 INFO:tasks.workunit.client.1.smithi192.stdout:9/981: rmdir d3/dd/d53/ddd 39 2024-04-24T22:45:48.292 INFO:tasks.workunit.client.1.smithi192.stdout:5/776: truncate d0/d13/d19/d64/d4f/d6c/ff6 566797 0 2024-04-24T22:45:48.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/841: link d2/db/d75/da1/d36/d5b/f24 d2/db/d10/f123 0 2024-04-24T22:45:48.298 INFO:tasks.workunit.client.1.smithi192.stdout:4/842: rmdir d2/db/d75/da1/d7d 39 2024-04-24T22:45:48.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/904: rename d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/f102 to d2/d4/d12/dbc/db0/f116 0 2024-04-24T22:45:48.303 INFO:tasks.workunit.client.1.smithi192.stdout:9/982: creat d3/dd/dce/d136/d17/f13b x:0 0 0 2024-04-24T22:45:48.303 INFO:tasks.workunit.client.1.smithi192.stdout:3/905: dread - d2/f9f zero size 2024-04-24T22:45:48.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/843: creat d2/db/d75/da1/d7d/f124 x:0 0 0 2024-04-24T22:45:48.309 INFO:tasks.workunit.client.1.smithi192.stdout:3/906: mknod d2/d4/d12/d1a/d1d/c117 0 2024-04-24T22:45:48.310 INFO:tasks.workunit.client.1.smithi192.stdout:3/907: write d2/d4/f69 [1587296,30184] 0 2024-04-24T22:45:48.312 INFO:tasks.workunit.client.1.smithi192.stdout:4/844: creat d2/db/d75/da1/d36/d5b/d28/deb/f125 x:0 0 0 2024-04-24T22:45:48.317 INFO:tasks.workunit.client.1.smithi192.stdout:4/845: creat d2/db/d10/f126 x:0 0 0 2024-04-24T22:45:48.321 INFO:tasks.workunit.client.1.smithi192.stdout:4/846: creat d2/d5c/d107/f127 x:0 0 0 2024-04-24T22:45:48.324 INFO:tasks.workunit.client.1.smithi192.stdout:5/777: write d0/d13/d19/d64/d5f/d2c/f51 [368969,83942] 0 2024-04-24T22:45:48.330 INFO:tasks.workunit.client.1.smithi192.stdout:4/847: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/fe6 [0,4194304] 0 2024-04-24T22:45:48.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/778: rename d0/d1b/dcb to d0/d13/d19/da9/d10b 0 2024-04-24T22:45:48.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/848: unlink d2/db/d75/da1/d7d/f124 0 2024-04-24T22:45:48.346 INFO:tasks.workunit.client.1.smithi192.stdout:3/908: dwrite d2/d27/dc6/d4f/f8c [0,4194304] 0 2024-04-24T22:45:48.348 INFO:tasks.workunit.client.1.smithi192.stdout:4/849: read d2/db/d10/fb6 [224831,14158] 0 2024-04-24T22:45:48.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/779: mkdir d0/d10c 0 2024-04-24T22:45:48.355 INFO:tasks.workunit.client.1.smithi192.stdout:4/850: dread - d2/db/d10/fc6 zero size 2024-04-24T22:45:48.364 INFO:tasks.workunit.client.1.smithi192.stdout:4/851: getdents d2/db/d5d 0 2024-04-24T22:45:48.367 INFO:tasks.workunit.client.1.smithi192.stdout:4/852: fsync d2/db/d75/da1/d36/d5b/f24 0 2024-04-24T22:45:48.369 INFO:tasks.workunit.client.1.smithi192.stdout:4/853: chown d2/db/d75/da1/f4c 2 1 2024-04-24T22:45:48.374 INFO:tasks.workunit.client.1.smithi192.stdout:4/854: creat d2/da5/f128 x:0 0 0 2024-04-24T22:45:48.374 INFO:tasks.workunit.client.1.smithi192.stdout:3/909: dread d2/d27/dc6/fd [0,4194304] 0 2024-04-24T22:45:48.375 INFO:tasks.workunit.client.1.smithi192.stdout:4/855: write d2/d5c/dce/fd8 [524044,116749] 0 2024-04-24T22:45:48.376 INFO:tasks.workunit.client.1.smithi192.stdout:4/856: stat d2/db/d10/l1a 0 2024-04-24T22:45:48.385 INFO:tasks.workunit.client.1.smithi192.stdout:3/910: creat d2/f118 x:0 0 0 2024-04-24T22:45:48.386 INFO:tasks.workunit.client.1.smithi192.stdout:3/911: fsync d2/d2d/dfe/f115 0 2024-04-24T22:45:48.388 INFO:tasks.workunit.client.1.smithi192.stdout:3/912: creat d2/d27/dc6/d4f/f119 x:0 0 0 2024-04-24T22:45:48.390 INFO:tasks.workunit.client.1.smithi192.stdout:3/913: truncate d2/d27/dc6/d41/f114 313081 0 2024-04-24T22:45:48.392 INFO:tasks.workunit.client.1.smithi192.stdout:3/914: rmdir d2/d4/d12/dbc 39 2024-04-24T22:45:48.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/857: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/d7e/f98 [0,4194304] 0 2024-04-24T22:45:48.402 INFO:tasks.workunit.client.1.smithi192.stdout:3/915: getdents d2/d27/dc6/d41 0 2024-04-24T22:45:48.410 INFO:tasks.workunit.client.1.smithi192.stdout:4/858: mkdir d2/db/d75/da1/d7d/d129 0 2024-04-24T22:45:48.413 INFO:tasks.workunit.client.1.smithi192.stdout:3/916: dwrite d2/d27/d28/f36 [0,4194304] 0 2024-04-24T22:45:48.420 INFO:tasks.workunit.client.1.smithi192.stdout:4/859: link d2/da5/l11e d2/db/d75/d10d/l12a 0 2024-04-24T22:45:48.422 INFO:tasks.workunit.client.1.smithi192.stdout:3/917: link d2/d27/dc6/da0/f112 d2/d27/dc6/d41/dad/f11a 0 2024-04-24T22:45:48.423 INFO:tasks.workunit.client.1.smithi192.stdout:4/860: creat d2/db/d75/da1/ddf/f12b x:0 0 0 2024-04-24T22:45:48.427 INFO:tasks.workunit.client.1.smithi192.stdout:4/861: creat d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/f12c x:0 0 0 2024-04-24T22:45:48.433 INFO:tasks.workunit.client.1.smithi192.stdout:4/862: dwrite d2/db/d5d/f115 [0,4194304] 0 2024-04-24T22:45:48.435 INFO:tasks.workunit.client.1.smithi192.stdout:4/863: chown d2/db/d5d/d77/f92 1 1 2024-04-24T22:45:48.441 INFO:tasks.workunit.client.1.smithi192.stdout:3/918: dwrite d2/d27/dc6/d41/fe8 [0,4194304] 0 2024-04-24T22:45:48.444 INFO:tasks.workunit.client.1.smithi192.stdout:3/919: dread d2/d27/dc6/f47 [0,4194304] 0 2024-04-24T22:45:48.451 INFO:tasks.workunit.client.1.smithi192.stdout:3/920: write d2/d27/dc6/d41/fe8 [4272361,55043] 0 2024-04-24T22:45:48.454 INFO:tasks.workunit.client.1.smithi192.stdout:3/921: creat d2/d4/d12/d39/def/f11b x:0 0 0 2024-04-24T22:45:48.456 INFO:tasks.workunit.client.1.smithi192.stdout:3/922: creat d2/d4/d12/d39/def/f11c x:0 0 0 2024-04-24T22:45:48.458 INFO:tasks.workunit.client.1.smithi192.stdout:3/923: write d2/d4/d12/d1a/d1d/d55/d95/f92 [2280099,110649] 0 2024-04-24T22:45:48.458 INFO:tasks.workunit.client.1.smithi192.stdout:3/924: stat d2/d27/dc6/d41/dad/cea 0 2024-04-24T22:45:48.459 INFO:tasks.workunit.client.1.smithi192.stdout:3/925: chown d2/d4/d12/d1a/d1d/f3f 52537 1 2024-04-24T22:45:48.463 INFO:tasks.workunit.client.1.smithi192.stdout:3/926: mknod d2/d4/d12/c11d 0 2024-04-24T22:45:48.464 INFO:tasks.workunit.client.1.smithi192.stdout:3/927: truncate d2/d27/dc6/f9 269089 0 2024-04-24T22:45:48.465 INFO:tasks.workunit.client.1.smithi192.stdout:3/928: write d2/d27/dc6/da0/f108 [604185,77899] 0 2024-04-24T22:45:48.468 INFO:tasks.workunit.client.1.smithi192.stdout:3/929: mkdir d2/d4/d12/d1a/d1d/d55/d11e 0 2024-04-24T22:45:48.470 INFO:tasks.workunit.client.1.smithi192.stdout:3/930: link d2/d4/d12/d39/d97/fdf d2/d27/dc6/d41/f11f 0 2024-04-24T22:45:48.492 INFO:tasks.workunit.client.1.smithi192.stdout:3/931: dwrite d2/d27/d28/d38/d10f/d93/faa [0,4194304] 0 2024-04-24T22:45:48.500 INFO:tasks.workunit.client.1.smithi192.stdout:3/932: link d2/d27/d28/f30 d2/d4/d12/d1a/d1d/f120 0 2024-04-24T22:45:48.500 INFO:tasks.workunit.client.1.smithi192.stdout:3/933: fdatasync d2/d27/dc6/da0/f108 0 2024-04-24T22:45:48.502 INFO:tasks.workunit.client.1.smithi192.stdout:3/934: chown d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/cc8 9947681 1 2024-04-24T22:45:48.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/935: dwrite d2/d27/dc6/fd [0,4194304] 0 2024-04-24T22:45:48.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/936: chown d2/f13 22 1 2024-04-24T22:45:48.519 INFO:tasks.workunit.client.1.smithi192.stdout:3/937: dread d2/d4/d12/d1a/d1d/d7b/ddb/f104 [0,4194304] 0 2024-04-24T22:45:48.527 INFO:tasks.workunit.client.1.smithi192.stdout:3/938: dwrite d2/f33 [0,4194304] 0 2024-04-24T22:45:48.532 INFO:tasks.workunit.client.1.smithi192.stdout:3/939: mkdir d2/d27/d28/dab/dca/d121 0 2024-04-24T22:45:48.534 INFO:tasks.workunit.client.1.smithi192.stdout:3/940: truncate d2/d27/d28/d38/d10f/d93/faa 4802069 0 2024-04-24T22:45:48.538 INFO:tasks.workunit.client.1.smithi192.stdout:3/941: creat d2/d4/d12/d1a/f122 x:0 0 0 2024-04-24T22:45:48.540 INFO:tasks.workunit.client.1.smithi192.stdout:3/942: fsync d2/f1e 0 2024-04-24T22:45:48.547 INFO:tasks.workunit.client.1.smithi192.stdout:3/943: dwrite d2/d27/f6f [0,4194304] 0 2024-04-24T22:45:48.550 INFO:tasks.workunit.client.1.smithi192.stdout:3/944: mkdir d2/d4/d12/dbc/db0/d123 0 2024-04-24T22:45:48.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/945: dread d2/d27/dc6/f47 [0,4194304] 0 2024-04-24T22:45:48.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/946: rmdir d2/d27/dc6/d4f/dbb 0 2024-04-24T22:45:48.561 INFO:tasks.workunit.client.1.smithi192.stdout:3/947: getdents d2/d27/d28/dfd 0 2024-04-24T22:45:48.563 INFO:tasks.workunit.client.1.smithi192.stdout:3/948: fsync d2/d27/dc6/da0/f112 0 2024-04-24T22:45:48.571 INFO:tasks.workunit.client.1.smithi192.stdout:3/949: write d2/d27/dc6/f2c [2414682,106446] 0 2024-04-24T22:45:48.575 INFO:tasks.workunit.client.1.smithi192.stdout:3/950: rmdir d2/d4/d12/dbc/d10a 0 2024-04-24T22:45:48.579 INFO:tasks.workunit.client.1.smithi192.stdout:3/951: unlink d2/d4/d12/d1a/d1d/d7b/df7/d54/fb3 0 2024-04-24T22:45:48.586 INFO:tasks.workunit.client.1.smithi192.stdout:3/952: write d2/d27/d28/dab/dca/fed [152045,33161] 0 2024-04-24T22:45:48.587 INFO:tasks.workunit.client.1.smithi192.stdout:3/953: chown d2/d27/dc6/f47 424583071 1 2024-04-24T22:45:48.591 INFO:tasks.workunit.client.1.smithi192.stdout:3/954: chown d2/d4/d12/d1a/d1d/d7b/df7/c83 61 1 2024-04-24T22:45:48.593 INFO:tasks.workunit.client.1.smithi192.stdout:3/955: truncate d2/d27/dc6/da0/f112 707368 0 2024-04-24T22:45:48.594 INFO:tasks.workunit.client.1.smithi192.stdout:3/956: chown d2/d27/d28/dab/dca 519905 1 2024-04-24T22:45:48.595 INFO:tasks.workunit.client.1.smithi192.stdout:3/957: write d2/d4/f16 [6174618,9490] 0 2024-04-24T22:45:48.596 INFO:tasks.workunit.client.1.smithi192.stdout:3/958: creat d2/d4/d4e/f124 x:0 0 0 2024-04-24T22:45:48.597 INFO:tasks.workunit.client.1.smithi192.stdout:3/959: readlink d2/d27/d28/l31 0 2024-04-24T22:45:48.617 INFO:tasks.workunit.client.1.smithi192.stdout:3/960: dwrite d2/d27/ff5 [0,4194304] 0 2024-04-24T22:45:48.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/961: symlink d2/d62/db2/l125 0 2024-04-24T22:45:48.624 INFO:tasks.workunit.client.1.smithi192.stdout:3/962: dread - d2/d4/d12/d1a/d1d/d55/d5f/f110 zero size 2024-04-24T22:45:48.627 INFO:tasks.workunit.client.1.smithi192.stdout:3/963: getdents d2/d106 0 2024-04-24T22:45:48.630 INFO:tasks.workunit.client.1.smithi192.stdout:3/964: mknod d2/d27/c126 0 2024-04-24T22:45:48.633 INFO:tasks.workunit.client.1.smithi192.stdout:3/965: creat d2/d4/d12/d1a/d1d/d55/d5f/db1/f127 x:0 0 0 2024-04-24T22:45:48.634 INFO:tasks.workunit.client.1.smithi192.stdout:3/966: write d2/d4/d12/d39/def/f11b [692120,125895] 0 2024-04-24T22:45:48.636 INFO:tasks.workunit.client.1.smithi192.stdout:3/967: rmdir d2/d27/d28/dab/dca/dee 39 2024-04-24T22:45:48.639 INFO:tasks.workunit.client.1.smithi192.stdout:3/968: rename d2/d27/fe7 to d2/d4/d12/dbc/f128 0 2024-04-24T22:45:48.640 INFO:tasks.workunit.client.1.smithi192.stdout:3/969: fsync d2/f13 0 2024-04-24T22:45:48.646 INFO:tasks.workunit.client.1.smithi192.stdout:3/970: dwrite d2/d4/d12/d1a/d1d/d55/f24 [4194304,4194304] 0 2024-04-24T22:45:48.650 INFO:tasks.workunit.client.1.smithi192.stdout:3/971: rename d2/f118 to d2/d106/f129 0 2024-04-24T22:45:48.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/972: creat d2/d4/d12/d1a/d1d/d55/d5f/d9b/f12a x:0 0 0 2024-04-24T22:45:48.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/973: write d2/d4/d12/d1a/d1d/fd9 [131391,33450] 0 2024-04-24T22:45:48.668 INFO:tasks.workunit.client.1.smithi192.stdout:8/877: sync 2024-04-24T22:45:48.669 INFO:tasks.workunit.client.1.smithi192.stdout:3/974: write d2/d27/dc6/f47 [1054084,88728] 0 2024-04-24T22:45:48.672 INFO:tasks.workunit.client.1.smithi192.stdout:8/878: mkdir db/d5d/df0/d112 0 2024-04-24T22:45:48.674 INFO:tasks.workunit.client.1.smithi192.stdout:3/975: fdatasync d2/d27/dc6/f87 0 2024-04-24T22:45:48.676 INFO:tasks.workunit.client.1.smithi192.stdout:3/976: truncate d2/d4/f10b 280744 0 2024-04-24T22:45:48.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/977: creat d2/d4/d12/f12b x:0 0 0 2024-04-24T22:45:48.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/879: dwrite db/dc/f24 [0,4194304] 0 2024-04-24T22:45:48.698 INFO:tasks.workunit.client.1.smithi192.stdout:8/880: truncate db/dc/f5b 3013449 0 2024-04-24T22:45:48.699 INFO:tasks.workunit.client.1.smithi192.stdout:8/881: chown db/dc/d22/l54 14506 1 2024-04-24T22:45:48.706 INFO:tasks.workunit.client.1.smithi192.stdout:8/882: dwrite db/d43/d76/d92/fd1 [0,4194304] 0 2024-04-24T22:45:48.707 INFO:tasks.workunit.client.1.smithi192.stdout:8/883: dread - db/dc/d29/f104 zero size 2024-04-24T22:45:48.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/884: write db/dc/d22/d31/f45 [1172921,67016] 0 2024-04-24T22:45:48.711 INFO:tasks.workunit.client.1.smithi192.stdout:8/885: mkdir db/d5d/d113 0 2024-04-24T22:45:48.715 INFO:tasks.workunit.client.1.smithi192.stdout:8/886: creat db/d43/d71/f114 x:0 0 0 2024-04-24T22:45:48.717 INFO:tasks.workunit.client.1.smithi192.stdout:8/887: creat db/dc/d22/dca/f115 x:0 0 0 2024-04-24T22:45:48.719 INFO:tasks.workunit.client.1.smithi192.stdout:8/888: mknod db/d43/d7c/c116 0 2024-04-24T22:45:48.720 INFO:tasks.workunit.client.1.smithi192.stdout:8/889: readlink db/dc/d22/l54 0 2024-04-24T22:45:48.733 INFO:tasks.workunit.client.1.smithi192.stdout:8/890: dwrite db/d75/fc4 [0,4194304] 0 2024-04-24T22:45:48.736 INFO:tasks.workunit.client.1.smithi192.stdout:8/891: dread - db/d75/f98 zero size 2024-04-24T22:45:48.739 INFO:tasks.workunit.client.1.smithi192.stdout:8/892: dread f9 [0,4194304] 0 2024-04-24T22:45:48.740 INFO:tasks.workunit.client.1.smithi192.stdout:8/893: write db/dc/d22/dca/f115 [552229,76116] 0 2024-04-24T22:45:48.741 INFO:tasks.workunit.client.1.smithi192.stdout:8/894: write db/d5d/d74/fea [1076049,103994] 0 2024-04-24T22:45:48.747 INFO:tasks.workunit.client.1.smithi192.stdout:8/895: getdents db/dc/d22/d31/d33/d81/d8a 0 2024-04-24T22:45:48.747 INFO:tasks.workunit.client.1.smithi192.stdout:8/896: readlink db/dc/l12 0 2024-04-24T22:45:48.750 INFO:tasks.workunit.client.1.smithi192.stdout:8/897: creat db/d43/d7c/ded/d10d/f117 x:0 0 0 2024-04-24T22:45:48.752 INFO:tasks.workunit.client.1.smithi192.stdout:8/898: write db/dc/d22/d31/d33/d8f/fe7 [366047,97637] 0 2024-04-24T22:45:48.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/899: write db/dc/d22/d31/f4d [1243228,48443] 0 2024-04-24T22:45:48.768 INFO:tasks.workunit.client.1.smithi192.stdout:8/900: creat db/d5d/df0/f118 x:0 0 0 2024-04-24T22:45:48.772 INFO:tasks.workunit.client.1.smithi192.stdout:8/901: fdatasync db/dc/d22/d31/d33/d81/d8a/f8b 0 2024-04-24T22:45:48.773 INFO:tasks.workunit.client.1.smithi192.stdout:8/902: stat db/d43/d7c/d85/db4/cf4 0 2024-04-24T22:45:48.774 INFO:tasks.workunit.client.1.smithi192.stdout:8/903: readlink db/d43/d7c/d85/l101 0 2024-04-24T22:45:48.775 INFO:tasks.workunit.client.1.smithi192.stdout:8/904: chown db/c7d 0 1 2024-04-24T22:45:48.825 INFO:tasks.workunit.client.1.smithi192.stdout:1/707: sync 2024-04-24T22:45:48.837 INFO:tasks.workunit.client.1.smithi192.stdout:1/708: dread d9/dd/d74/f88 [0,4194304] 0 2024-04-24T22:45:48.840 INFO:tasks.workunit.client.1.smithi192.stdout:1/709: symlink d9/d15/d2d/d4a/lea 0 2024-04-24T22:45:48.846 INFO:tasks.workunit.client.1.smithi192.stdout:1/710: dwrite d9/d15/d2d/d48/d93/da4/dd2/dbf/fd7 [0,4194304] 0 2024-04-24T22:45:48.849 INFO:tasks.workunit.client.1.smithi192.stdout:1/711: dread d9/d51/d70/f7f [0,4194304] 0 2024-04-24T22:45:48.854 INFO:tasks.workunit.client.1.smithi192.stdout:1/712: creat d9/d15/d5e/feb x:0 0 0 2024-04-24T22:45:48.856 INFO:tasks.workunit.client.1.smithi192.stdout:1/713: mkdir d9/d15/d2d/d63/dec 0 2024-04-24T22:45:48.863 INFO:tasks.workunit.client.1.smithi192.stdout:1/714: dwrite d9/d15/d5e/feb [0,4194304] 0 2024-04-24T22:45:48.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/715: dread - d9/d15/d2d/d48/d93/f98 zero size 2024-04-24T22:45:48.874 INFO:tasks.workunit.client.1.smithi192.stdout:1/716: symlink d9/d15/d2d/d31/led 0 2024-04-24T22:45:48.875 INFO:tasks.workunit.client.1.smithi192.stdout:1/717: rmdir d9/d15/d5e/d90/da9 39 2024-04-24T22:45:48.876 INFO:tasks.workunit.client.1.smithi192.stdout:1/718: chown d9/d15/d5e/l80 2 1 2024-04-24T22:45:48.877 INFO:tasks.workunit.client.1.smithi192.stdout:1/719: write d9/d15/d2d/d63/d68/f76 [2697120,93150] 0 2024-04-24T22:45:48.881 INFO:tasks.workunit.client.1.smithi192.stdout:1/720: creat d9/d15/d2d/d63/d68/d9d/fee x:0 0 0 2024-04-24T22:45:48.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/721: mkdir d9/d15/d5e/d90/da9/ddd/def 0 2024-04-24T22:45:48.888 INFO:tasks.workunit.client.1.smithi192.stdout:1/722: fsync d9/dd/d74/f7a 0 2024-04-24T22:45:48.889 INFO:tasks.workunit.client.1.smithi192.stdout:1/723: chown d9/d15/d2d/ccb 13 1 2024-04-24T22:45:48.890 INFO:tasks.workunit.client.1.smithi192.stdout:1/724: chown d9/d15/d2d/lb0 18 1 2024-04-24T22:45:48.895 INFO:tasks.workunit.client.1.smithi192.stdout:1/725: read d9/d15/d5e/d67/fa2 [2775220,111191] 0 2024-04-24T22:45:48.900 INFO:tasks.workunit.client.1.smithi192.stdout:1/726: mknod d9/d15/d2d/d48/d93/da4/db5/ddb/cf0 0 2024-04-24T22:45:48.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/727: dwrite d9/d15/d2d/d31/d2f/f85 [0,4194304] 0 2024-04-24T22:45:48.916 INFO:tasks.workunit.client.1.smithi192.stdout:1/728: creat d9/d15/d2d/d48/d93/dc5/ff1 x:0 0 0 2024-04-24T22:45:48.918 INFO:tasks.workunit.client.1.smithi192.stdout:1/729: dread - d9/d15/d2d/d48/d93/da4/fc9 zero size 2024-04-24T22:45:48.921 INFO:tasks.workunit.client.1.smithi192.stdout:1/730: mkdir d9/d15/d2d/d48/d93/dc5/dd6/df2 0 2024-04-24T22:45:48.923 INFO:tasks.workunit.client.1.smithi192.stdout:1/731: creat d9/dd/d74/dcd/ff3 x:0 0 0 2024-04-24T22:45:48.930 INFO:tasks.workunit.client.1.smithi192.stdout:1/732: dwrite d9/d15/d2d/d31/d2f/dae/fd9 [0,4194304] 0 2024-04-24T22:45:48.934 INFO:tasks.workunit.client.1.smithi192.stdout:1/733: dread d9/d6f/f9b [0,4194304] 0 2024-04-24T22:45:48.936 INFO:tasks.workunit.client.1.smithi192.stdout:1/734: mknod d9/d15/d2d/d63/d68/d9d/cf4 0 2024-04-24T22:45:48.943 INFO:tasks.workunit.client.1.smithi192.stdout:1/735: getdents d9/d15/d2d/d31/d2f 0 2024-04-24T22:45:48.944 INFO:tasks.workunit.client.1.smithi192.stdout:1/736: read d9/d15/d2d/d31/d2f/f85 [3392008,60871] 0 2024-04-24T22:45:48.945 INFO:tasks.workunit.client.1.smithi192.stdout:1/737: stat d9/d15/f49 0 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:9/983: sync 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:0/915: sync 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:6/857: sync 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:5/780: sync 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:2/963: sync 2024-04-24T22:45:48.948 INFO:tasks.workunit.client.1.smithi192.stdout:4/864: sync 2024-04-24T22:45:48.957 INFO:tasks.workunit.client.1.smithi192.stdout:0/916: read d3/d27/f43 [2666123,87846] 0 2024-04-24T22:45:48.958 INFO:tasks.workunit.client.1.smithi192.stdout:6/858: write d0/d8/d29/d4c/db0/fb5 [3803116,76636] 0 2024-04-24T22:45:48.962 INFO:tasks.workunit.client.1.smithi192.stdout:0/917: dread d3/d27/d28/d2e/dc7/fd0 [0,4194304] 0 2024-04-24T22:45:48.974 INFO:tasks.workunit.client.1.smithi192.stdout:0/918: dread d3/d27/d6a/f87 [0,4194304] 0 2024-04-24T22:45:48.995 INFO:tasks.workunit.client.1.smithi192.stdout:8/905: sync 2024-04-24T22:45:48.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/978: sync 2024-04-24T22:45:48.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/979: write d2/d4/f69 [1062823,118459] 0 2024-04-24T22:45:48.997 INFO:tasks.workunit.client.1.smithi192.stdout:3/980: chown d2/d27/dc6/d41/dad 5457087 1 2024-04-24T22:45:49.002 INFO:tasks.workunit.client.1.smithi192.stdout:1/738: link d9/d51/d70/f7f d9/d15/d2d/d63/d68/d9d/ff5 0 2024-04-24T22:45:49.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/865: rmdir d2/db/d75/da1/dc9 39 2024-04-24T22:45:49.008 INFO:tasks.workunit.client.1.smithi192.stdout:5/781: truncate d0/d13/d19/d64/d5f/f27 2030016 0 2024-04-24T22:45:49.009 INFO:tasks.workunit.client.1.smithi192.stdout:6/859: mknod d0/d8/d3d/d3e/d40/d44/c137 0 2024-04-24T22:45:49.010 INFO:tasks.workunit.client.1.smithi192.stdout:6/860: truncate d0/d8/d3d/dc6/f133 998539 0 2024-04-24T22:45:49.017 INFO:tasks.workunit.client.1.smithi192.stdout:6/861: dwrite d0/d8/d3d/d3e/fea [0,4194304] 0 2024-04-24T22:45:49.022 INFO:tasks.workunit.client.1.smithi192.stdout:4/866: mkdir d2/db/d75/da1/d36/d5b/d28/deb/d12d 0 2024-04-24T22:45:49.024 INFO:tasks.workunit.client.1.smithi192.stdout:6/862: dread d0/d8/d29/d5a/fb6 [4194304,4194304] 0 2024-04-24T22:45:49.025 INFO:tasks.workunit.client.1.smithi192.stdout:2/964: creat d4/d9/d9f/d126/f136 x:0 0 0 2024-04-24T22:45:49.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/981: getdents d2/d27/d28/dfd 0 2024-04-24T22:45:49.042 INFO:tasks.workunit.client.1.smithi192.stdout:2/965: mknod d4/d9/d10/da7/df5/c137 0 2024-04-24T22:45:49.044 INFO:tasks.workunit.client.1.smithi192.stdout:4/867: truncate d2/d9/f3f 14304 0 2024-04-24T22:45:49.045 INFO:tasks.workunit.client.1.smithi192.stdout:9/984: link d3/dd/dce/d136/d17/d89/d8b/f11b d3/dd/d53/dab/f13c 0 2024-04-24T22:45:49.047 INFO:tasks.workunit.client.1.smithi192.stdout:5/782: creat d0/d13/d19/d64/de7/f10d x:0 0 0 2024-04-24T22:45:49.050 INFO:tasks.workunit.client.1.smithi192.stdout:0/919: rmdir d3/d27/d28/d2e/d3f/d12d/d12e 0 2024-04-24T22:45:49.053 INFO:tasks.workunit.client.1.smithi192.stdout:1/739: creat d9/d15/d2d/d31/ff6 x:0 0 0 2024-04-24T22:45:49.056 INFO:tasks.workunit.client.1.smithi192.stdout:5/783: dwrite d0/d13/d19/d64/d4f/d7c/ffb [0,4194304] 0 2024-04-24T22:45:49.057 INFO:tasks.workunit.client.1.smithi192.stdout:3/982: dread d2/d4/f16 [4194304,4194304] 0 2024-04-24T22:45:49.057 INFO:tasks.workunit.client.1.smithi192.stdout:5/784: readlink d0/d13/d19/l4c 0 2024-04-24T22:45:49.060 INFO:tasks.workunit.client.1.smithi192.stdout:0/920: mknod d3/d27/d28/dba/c12f 0 2024-04-24T22:45:49.069 INFO:tasks.workunit.client.1.smithi192.stdout:4/868: link d2/db/d75/da1/ddf/f12b d2/db/d5d/f12e 0 2024-04-24T22:45:49.070 INFO:tasks.workunit.client.1.smithi192.stdout:0/921: dread d3/d27/d28/d31/f3a [0,4194304] 0 2024-04-24T22:45:49.072 INFO:tasks.workunit.client.1.smithi192.stdout:3/983: fsync d2/d27/dc6/d41/f11f 0 2024-04-24T22:45:49.077 INFO:tasks.workunit.client.1.smithi192.stdout:9/985: link d3/fb d3/dd/dce/d136/d17/d89/db4/db9/d10a/d131/f13d 0 2024-04-24T22:45:49.078 INFO:tasks.workunit.client.1.smithi192.stdout:8/906: dwrite db/d5d/f7e [0,4194304] 0 2024-04-24T22:45:49.080 INFO:tasks.workunit.client.1.smithi192.stdout:8/907: chown db/dc/d22/d31/d33/db7/fc7 257 1 2024-04-24T22:45:49.084 INFO:tasks.workunit.client.1.smithi192.stdout:4/869: rename d2/db/d75/da1/d36/d5b/d28/d2f/ce1 to d2/db/d75/da1/d36/d5b/d64/c12f 0 2024-04-24T22:45:49.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/863: dwrite d0/d8/d29/d5a/f6b [0,4194304] 0 2024-04-24T22:45:49.088 INFO:tasks.workunit.client.1.smithi192.stdout:0/922: creat d3/d27/d3c/d64/f130 x:0 0 0 2024-04-24T22:45:49.094 INFO:tasks.workunit.client.1.smithi192.stdout:3/984: creat d2/d27/d28/dab/f12c x:0 0 0 2024-04-24T22:45:49.095 INFO:tasks.workunit.client.1.smithi192.stdout:5/785: mkdir d0/d13/d19/d83/dad/d10e 0 2024-04-24T22:45:49.105 INFO:tasks.workunit.client.1.smithi192.stdout:2/966: dwrite d4/f69 [0,4194304] 0 2024-04-24T22:45:49.106 INFO:tasks.workunit.client.1.smithi192.stdout:2/967: stat d4/d29/d60/d9a/c97 0 2024-04-24T22:45:49.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/786: rename d0/d13/d19/d64/d5f to d0/d1b/d62/d10f 0 2024-04-24T22:45:49.116 INFO:tasks.workunit.client.1.smithi192.stdout:1/740: dwrite d9/fa [4194304,4194304] 0 2024-04-24T22:45:49.119 INFO:tasks.workunit.client.1.smithi192.stdout:8/908: symlink db/dc/d22/d31/d33/l119 0 2024-04-24T22:45:49.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/864: mkdir d0/d8/d3d/dc6/d127/d138 0 2024-04-24T22:45:49.126 INFO:tasks.workunit.client.1.smithi192.stdout:2/968: rmdir d4/d29/d60/d9a 39 2024-04-24T22:45:49.129 INFO:tasks.workunit.client.1.smithi192.stdout:6/865: dread - d0/d8/d3d/d3e/d40/d44/f104 zero size 2024-04-24T22:45:49.129 INFO:tasks.workunit.client.1.smithi192.stdout:5/787: rmdir d0/d13/dd1/d108 39 2024-04-24T22:45:49.131 INFO:tasks.workunit.client.1.smithi192.stdout:4/870: dwrite d2/db/d5d/db3/f105 [4194304,4194304] 0 2024-04-24T22:45:49.132 INFO:tasks.workunit.client.1.smithi192.stdout:1/741: write d9/d15/d5e/d90/da9/fc1 [1740371,63849] 0 2024-04-24T22:45:49.138 INFO:tasks.workunit.client.1.smithi192.stdout:5/788: mknod d0/c110 0 2024-04-24T22:45:49.138 INFO:tasks.workunit.client.1.smithi192.stdout:9/986: dwrite d3/dd/d53/fd5 [0,4194304] 0 2024-04-24T22:45:49.139 INFO:tasks.workunit.client.1.smithi192.stdout:4/871: read - d2/db/d75/da1/ddf/f12b zero size 2024-04-24T22:45:49.142 INFO:tasks.workunit.client.1.smithi192.stdout:0/923: dwrite d3/d27/d28/d2e/dc7/fd0 [0,4194304] 0 2024-04-24T22:45:49.147 INFO:tasks.workunit.client.1.smithi192.stdout:3/985: truncate d2/d27/dc6/d4f/f5a 2612487 0 2024-04-24T22:45:49.147 INFO:tasks.workunit.client.1.smithi192.stdout:6/866: write d0/d8/d29/d5a/f60 [3435509,22928] 0 2024-04-24T22:45:49.153 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:48 smithi192 ceph-mon[33991]: pgmap v22: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 117 MiB/s rd, 265 MiB/s wr, 595 op/s 2024-04-24T22:45:49.156 INFO:tasks.workunit.client.1.smithi192.stdout:8/909: getdents db/d5d/d74 0 2024-04-24T22:45:49.160 INFO:tasks.workunit.client.1.smithi192.stdout:0/924: symlink d3/d27/d6a/l131 0 2024-04-24T22:45:49.160 INFO:tasks.workunit.client.1.smithi192.stdout:4/872: chown d2/db/d75/da1/d7d/d9c/dda 6228 1 2024-04-24T22:45:49.162 INFO:tasks.workunit.client.1.smithi192.stdout:3/986: rmdir d2/d27 39 2024-04-24T22:45:49.163 INFO:tasks.workunit.client.1.smithi192.stdout:3/987: chown d2/d4/d12/d1a/d1d/d7b/l109 271 1 2024-04-24T22:45:49.167 INFO:tasks.workunit.client.1.smithi192.stdout:6/867: creat d0/d8/d29/d4c/db0/f139 x:0 0 0 2024-04-24T22:45:49.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/910: creat db/dc/d22/d31/d33/d77/f11a x:0 0 0 2024-04-24T22:45:49.172 INFO:tasks.workunit.client.1.smithi192.stdout:8/911: readlink db/d43/d71/l78 0 2024-04-24T22:45:49.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/925: unlink d3/d27/d28/f24 0 2024-04-24T22:45:49.174 INFO:tasks.workunit.client.1.smithi192.stdout:9/987: dread d3/f31 [4194304,4194304] 0 2024-04-24T22:45:49.175 INFO:tasks.workunit.client.1.smithi192.stdout:2/969: link d4/d9/fb d4/d9/d10/d6c/d8c/da5/de2/d12c/f138 0 2024-04-24T22:45:49.180 INFO:tasks.workunit.client.1.smithi192.stdout:6/868: dread - d0/d8/d3d/d3e/d7e/faf zero size 2024-04-24T22:45:49.182 INFO:tasks.workunit.client.1.smithi192.stdout:2/970: dwrite d4/d9/d10/d6c/d6f/feb [0,4194304] 0 2024-04-24T22:45:49.187 INFO:tasks.workunit.client.1.smithi192.stdout:1/742: truncate d9/f42 4330849 0 2024-04-24T22:45:49.190 INFO:tasks.workunit.client.1.smithi192.stdout:3/988: symlink d2/d4/d12/d1a/d1d/d55/d11e/l12d 0 2024-04-24T22:45:49.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/912: write db/dc/d22/d31/fd3 [257212,70726] 0 2024-04-24T22:45:49.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/926: symlink d3/d27/d28/l132 0 2024-04-24T22:45:49.210 INFO:tasks.workunit.client.1.smithi192.stdout:3/989: symlink d2/d4/d12/d1a/d1d/d55/d5f/d9b/l12e 0 2024-04-24T22:45:49.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/927: dread d3/d27/d28/d31/dbc/dea/f101 [0,4194304] 0 2024-04-24T22:45:49.212 INFO:tasks.workunit.client.1.smithi192.stdout:6/869: unlink d0/d8/d29/l3c 0 2024-04-24T22:45:49.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/913: symlink db/d5d/d74/da0/l11b 0 2024-04-24T22:45:49.214 INFO:tasks.workunit.client.1.smithi192.stdout:2/971: dread d4/d9/d10/d6c/d95/db7/fed [0,4194304] 0 2024-04-24T22:45:49.215 INFO:tasks.workunit.client.1.smithi192.stdout:9/988: creat d3/dd/d62/f13e x:0 0 0 2024-04-24T22:45:49.220 INFO:tasks.workunit.client.1.smithi192.stdout:8/914: symlink db/d43/d7c/d85/db4/l11c 0 2024-04-24T22:45:49.221 INFO:tasks.workunit.client.1.smithi192.stdout:9/989: symlink d3/d24/l13f 0 2024-04-24T22:45:49.225 INFO:tasks.workunit.client.1.smithi192.stdout:3/990: mkdir d2/d4/d12/d1a/d1d/d55/d12f 0 2024-04-24T22:45:49.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/743: write d9/d15/d2d/d48/d93/da4/dd2/fbd [513263,118458] 0 2024-04-24T22:45:49.236 INFO:tasks.workunit.client.1.smithi192.stdout:6/870: link d0/d8/d29/d4c/d4d/d6a/cbb d0/d8/d29/d58/c13a 0 2024-04-24T22:45:49.237 INFO:tasks.workunit.client.1.smithi192.stdout:9/990: dread d3/d5d/fa9 [0,4194304] 0 2024-04-24T22:45:49.239 INFO:tasks.workunit.client.1.smithi192.stdout:9/991: truncate d3/dd/dce/d136/da7/f108 5712 0 2024-04-24T22:45:49.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/744: truncate d9/d15/d2d/d31/f62 997917 0 2024-04-24T22:45:49.242 INFO:tasks.workunit.client.1.smithi192.stdout:2/972: link d4/d9/d9f/ca1 d4/d9/d10/d6c/c139 0 2024-04-24T22:45:49.244 INFO:tasks.workunit.client.1.smithi192.stdout:6/871: mknod d0/d8/d22/dcb/de0/d10d/c13b 0 2024-04-24T22:45:49.246 INFO:tasks.workunit.client.1.smithi192.stdout:6/872: stat d0/d8/d3d/d9d/dd6/ddc/f11a 0 2024-04-24T22:45:49.248 INFO:tasks.workunit.client.1.smithi192.stdout:9/992: symlink d3/dd/d62/d6d/l140 0 2024-04-24T22:45:49.249 INFO:tasks.workunit.client.1.smithi192.stdout:9/993: readlink d3/dd/d62/d6d/lc9 0 2024-04-24T22:45:49.261 INFO:tasks.workunit.client.1.smithi192.stdout:9/994: mknod d3/d59/d7d/c141 0 2024-04-24T22:45:49.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/915: dwrite db/dc/d22/d31/d33/d81/d8a/ffe [4194304,4194304] 0 2024-04-24T22:45:49.270 INFO:tasks.workunit.client.1.smithi192.stdout:8/916: dread - db/d5d/dcb/f105 zero size 2024-04-24T22:45:49.278 INFO:tasks.workunit.client.1.smithi192.stdout:8/917: symlink db/d5d/d74/d96/l11d 0 2024-04-24T22:45:49.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/918: write db/ffb [2386334,116525] 0 2024-04-24T22:45:49.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/995: dread d3/dd/dce/d136/f19 [0,4194304] 0 2024-04-24T22:45:49.282 INFO:tasks.workunit.client.1.smithi192.stdout:8/919: truncate db/dc/d22/d31/d33/d8f/fe7 1188902 0 2024-04-24T22:45:49.284 INFO:tasks.workunit.client.1.smithi192.stdout:8/920: write db/f56 [3187492,80979] 0 2024-04-24T22:45:49.287 INFO:tasks.workunit.client.1.smithi192.stdout:9/996: creat d3/dd/d53/ddd/f142 x:0 0 0 2024-04-24T22:45:49.290 INFO:tasks.workunit.client.1.smithi192.stdout:8/921: symlink db/dc/d29/d9f/l11e 0 2024-04-24T22:45:49.300 INFO:tasks.workunit.client.1.smithi192.stdout:8/922: dwrite db/dc/f8e [0,4194304] 0 2024-04-24T22:45:49.302 INFO:tasks.workunit.client.1.smithi192.stdout:9/997: rmdir d3/d5d/da1/d85 0 2024-04-24T22:45:49.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:48 smithi063 ceph-mon[26912]: pgmap v22: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 117 MiB/s rd, 265 MiB/s wr, 595 op/s 2024-04-24T22:45:49.304 INFO:tasks.workunit.client.1.smithi192.stdout:8/923: symlink db/d43/d76/d92/dad/l11f 0 2024-04-24T22:45:49.306 INFO:tasks.workunit.client.1.smithi192.stdout:9/998: dread - d3/dd/d53/dab/dcf/d13a/f101 zero size 2024-04-24T22:45:49.307 INFO:tasks.workunit.client.1.smithi192.stdout:8/924: symlink db/dc/d29/d9f/l120 0 2024-04-24T22:45:49.310 INFO:tasks.workunit.client.1.smithi192.stdout:9/999: mknod d3/d59/dbf/c143 0 2024-04-24T22:45:49.313 INFO:tasks.workunit.client.1.smithi192.stdout:1/745: dwrite d9/d51/f54 [0,4194304] 0 2024-04-24T22:45:49.317 INFO:tasks.workunit.client.1.smithi192.stdout:6/873: write d0/d8/d22/dcb/fad [285014,42960] 0 2024-04-24T22:45:49.324 INFO:tasks.workunit.client.1.smithi192.stdout:1/746: creat d9/dd/d74/dcd/ff7 x:0 0 0 2024-04-24T22:45:49.325 INFO:tasks.workunit.client.1.smithi192.stdout:2/973: dwrite d4/d29/f2b [4194304,4194304] 0 2024-04-24T22:45:49.328 INFO:tasks.workunit.client.1.smithi192.stdout:1/747: dread - d9/d51/f9c zero size 2024-04-24T22:45:49.330 INFO:tasks.workunit.client.1.smithi192.stdout:1/748: write d9/d15/d5e/feb [3240342,122975] 0 2024-04-24T22:45:49.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/749: unlink d9/d51/d70/f95 0 2024-04-24T22:45:49.349 INFO:tasks.workunit.client.1.smithi192.stdout:8/925: dread db/dc/d22/f23 [0,4194304] 0 2024-04-24T22:45:49.363 INFO:tasks.workunit.client.1.smithi192.stdout:2/974: write d4/d9/d37/f3e [254359,65080] 0 2024-04-24T22:45:49.373 INFO:tasks.workunit.client.1.smithi192.stdout:6/874: truncate d0/d8/d29/d58/f9f 439358 0 2024-04-24T22:45:49.388 INFO:tasks.workunit.client.1.smithi192.stdout:1/750: dwrite d9/d51/d70/f7f [0,4194304] 0 2024-04-24T22:45:49.389 INFO:tasks.workunit.client.1.smithi192.stdout:2/975: rename d4/d9/d10/d6c/d95/db7/ce3 to d4/d9/d10/d102/c13a 0 2024-04-24T22:45:49.392 INFO:tasks.workunit.client.1.smithi192.stdout:8/926: dread db/dc/d22/d31/fc1 [0,4194304] 0 2024-04-24T22:45:49.395 INFO:tasks.workunit.client.1.smithi192.stdout:8/927: dread - db/dc/d22/d31/f108 zero size 2024-04-24T22:45:49.395 INFO:tasks.workunit.client.1.smithi192.stdout:2/976: creat d4/d9/f13b x:0 0 0 2024-04-24T22:45:49.398 INFO:tasks.workunit.client.1.smithi192.stdout:1/751: fdatasync d9/d15/d2d/d31/d2f/fc0 0 2024-04-24T22:45:49.400 INFO:tasks.workunit.client.1.smithi192.stdout:1/752: dread d9/d6f/f9b [0,4194304] 0 2024-04-24T22:45:49.403 INFO:tasks.workunit.client.1.smithi192.stdout:6/875: rename d0/d8/d29/d4c/db0/f139 to d0/d8/d18/df1/df2/f13c 0 2024-04-24T22:45:49.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/928: creat db/dc/d22/d31/d33/d77/f121 x:0 0 0 2024-04-24T22:45:49.412 INFO:tasks.workunit.client.1.smithi192.stdout:2/977: symlink d4/d9/d10/d6c/d95/l13c 0 2024-04-24T22:45:49.418 INFO:tasks.workunit.client.1.smithi192.stdout:1/753: creat d9/d15/d2d/d48/d93/da4/db5/ff8 x:0 0 0 2024-04-24T22:45:49.423 INFO:tasks.workunit.client.1.smithi192.stdout:8/929: creat db/dc/d22/d31/d33/d8f/da6/f122 x:0 0 0 2024-04-24T22:45:49.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/930: chown db/dc/d22/d31/fd3 75353309 1 2024-04-24T22:45:49.426 INFO:tasks.workunit.client.1.smithi192.stdout:1/754: dread d9/f2c [0,4194304] 0 2024-04-24T22:45:49.431 INFO:tasks.workunit.client.1.smithi192.stdout:6/876: rename d0/d8/d22/dcb/de0/l11f to d0/d8/d18/l13d 0 2024-04-24T22:45:49.438 INFO:tasks.workunit.client.1.smithi192.stdout:8/931: dread db/dc/d29/f86 [0,4194304] 0 2024-04-24T22:45:49.445 INFO:tasks.workunit.client.1.smithi192.stdout:1/755: getdents d9/d15/d5e 0 2024-04-24T22:45:49.446 INFO:tasks.workunit.client.1.smithi192.stdout:8/932: dwrite db/dc/d22/d31/f108 [0,4194304] 0 2024-04-24T22:45:49.470 INFO:tasks.workunit.client.1.smithi192.stdout:6/877: dwrite d0/d8/d3d/d3e/fac [0,4194304] 0 2024-04-24T22:45:49.471 INFO:tasks.workunit.client.1.smithi192.stdout:6/878: chown d0/d8/d29/d5a/d63/leb 1275753 1 2024-04-24T22:45:49.479 INFO:tasks.workunit.client.1.smithi192.stdout:8/933: truncate db/dc/d22/d31/d33/f49 1351739 0 2024-04-24T22:45:49.481 INFO:tasks.workunit.client.1.smithi192.stdout:8/934: truncate db/d75/fc9 431744 0 2024-04-24T22:45:49.481 INFO:tasks.workunit.client.1.smithi192.stdout:8/935: chown db/d43/d76 31609 1 2024-04-24T22:45:49.482 INFO:tasks.workunit.client.1.smithi192.stdout:1/756: dwrite d9/dd/d74/d83/fb7 [0,4194304] 0 2024-04-24T22:45:49.484 INFO:tasks.workunit.client.1.smithi192.stdout:1/757: dread d9/d6f/f9b [0,4194304] 0 2024-04-24T22:45:49.486 INFO:tasks.workunit.client.1.smithi192.stdout:8/936: rmdir db/d43/d7c/d85 39 2024-04-24T22:45:49.487 INFO:tasks.workunit.client.1.smithi192.stdout:8/937: dread - db/d75/f98 zero size 2024-04-24T22:45:49.490 INFO:tasks.workunit.client.1.smithi192.stdout:8/938: chown db/d43/d7c/d85/df7 6 1 2024-04-24T22:45:49.492 INFO:tasks.workunit.client.1.smithi192.stdout:8/939: write db/d43/d7c/fd2 [1386672,119398] 0 2024-04-24T22:45:49.494 INFO:tasks.workunit.client.1.smithi192.stdout:1/758: symlink d9/lf9 0 2024-04-24T22:45:49.495 INFO:tasks.workunit.client.1.smithi192.stdout:1/759: dread - d9/d15/d5e/d90/da9/ddd/fe0 zero size 2024-04-24T22:45:49.498 INFO:tasks.workunit.client.1.smithi192.stdout:8/940: mknod db/d5d/dcb/c123 0 2024-04-24T22:45:49.500 INFO:tasks.workunit.client.1.smithi192.stdout:1/760: mkdir d9/d15/d2d/d4a/dfa 0 2024-04-24T22:45:49.501 INFO:tasks.workunit.client.1.smithi192.stdout:1/761: fsync d9/dd/d74/dcd/fcf 0 2024-04-24T22:45:49.502 INFO:tasks.workunit.client.1.smithi192.stdout:8/941: creat db/d5d/d74/da0/f124 x:0 0 0 2024-04-24T22:45:49.506 INFO:tasks.workunit.client.1.smithi192.stdout:8/942: mkdir db/ddf/d125 0 2024-04-24T22:45:49.508 INFO:tasks.workunit.client.1.smithi192.stdout:1/762: dwrite d9/d15/d2d/d31/d2f/f4c [0,4194304] 0 2024-04-24T22:45:49.508 INFO:tasks.workunit.client.1.smithi192.stdout:8/943: chown db/dc/d22/d31/d33 5548114 1 2024-04-24T22:45:49.511 INFO:tasks.workunit.client.1.smithi192.stdout:8/944: chown db/d43/d76/cc3 549374861 1 2024-04-24T22:45:49.513 INFO:tasks.workunit.client.1.smithi192.stdout:6/879: dwrite d0/d8/f6f [0,4194304] 0 2024-04-24T22:45:49.516 INFO:tasks.workunit.client.1.smithi192.stdout:6/880: fsync d0/d8/d22/dcb/de0/fe4 0 2024-04-24T22:45:49.517 INFO:tasks.workunit.client.1.smithi192.stdout:6/881: dread - d0/d8/d29/d4c/d4d/f72 zero size 2024-04-24T22:45:49.520 INFO:tasks.workunit.client.1.smithi192.stdout:8/945: symlink db/dc/d22/da3/l126 0 2024-04-24T22:45:49.522 INFO:tasks.workunit.client.1.smithi192.stdout:8/946: rmdir db/d43/d76 39 2024-04-24T22:45:49.525 INFO:tasks.workunit.client.1.smithi192.stdout:6/882: rename d0/d8/d22/dcb/de0/de7 to d0/d8/d3d/d13e 0 2024-04-24T22:45:49.528 INFO:tasks.workunit.client.1.smithi192.stdout:8/947: mkdir db/dc/d22/d31/d33/d8f/d127 0 2024-04-24T22:45:49.529 INFO:tasks.workunit.client.1.smithi192.stdout:8/948: write db/dc/f8e [1641687,85487] 0 2024-04-24T22:45:49.530 INFO:tasks.workunit.client.1.smithi192.stdout:6/883: mkdir d0/d8/d3d/dc6/d13f 0 2024-04-24T22:45:49.534 INFO:tasks.workunit.client.1.smithi192.stdout:6/884: creat d0/d8/d29/d5a/f140 x:0 0 0 2024-04-24T22:45:49.540 INFO:tasks.workunit.client.1.smithi192.stdout:6/885: creat d0/d8/d3d/d3e/d40/f141 x:0 0 0 2024-04-24T22:45:49.541 INFO:tasks.workunit.client.1.smithi192.stdout:8/949: mknod db/dc/d22/d31/d33/d8f/d127/c128 0 2024-04-24T22:45:49.544 INFO:tasks.workunit.client.1.smithi192.stdout:6/886: write d0/d8/d22/f27 [5371295,45357] 0 2024-04-24T22:45:49.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/763: dwrite d9/d15/d2d/d4a/f5b [0,4194304] 0 2024-04-24T22:45:49.547 INFO:tasks.workunit.client.1.smithi192.stdout:8/950: stat db/d43/d7c/d85/fa7 0 2024-04-24T22:45:49.553 INFO:tasks.workunit.client.1.smithi192.stdout:6/887: symlink d0/d8/d18/d2e/l142 0 2024-04-24T22:45:49.554 INFO:tasks.workunit.client.1.smithi192.stdout:1/764: mknod d9/d15/d2d/d48/d93/da4/cfb 0 2024-04-24T22:45:49.559 INFO:tasks.workunit.client.1.smithi192.stdout:6/888: mkdir d0/d8/d18/d143 0 2024-04-24T22:45:49.562 INFO:tasks.workunit.client.1.smithi192.stdout:8/951: dread db/f65 [0,4194304] 0 2024-04-24T22:45:49.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/765: rename d9/d15/d2d/d48/d93/da4/dbb to d9/d15/d2d/d48/d93/dfc 0 2024-04-24T22:45:49.564 INFO:tasks.workunit.client.1.smithi192.stdout:6/889: symlink d0/d8/d22/dcb/de0/d10d/l144 0 2024-04-24T22:45:49.570 INFO:tasks.workunit.client.1.smithi192.stdout:1/766: rename d9/dd/cc3 to d9/d15/d2d/d48/d93/da4/dd2/dbf/de9/cfd 0 2024-04-24T22:45:49.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/952: getdents db/dc/d22/d31/d33/d8f 0 2024-04-24T22:45:49.578 INFO:tasks.workunit.client.1.smithi192.stdout:8/953: mknod db/dc/d22/c129 0 2024-04-24T22:45:49.580 INFO:tasks.workunit.client.1.smithi192.stdout:1/767: write d9/d15/d2d/d63/f79 [4465669,80703] 0 2024-04-24T22:45:49.597 INFO:tasks.workunit.client.1.smithi192.stdout:1/768: dread f3 [0,4194304] 0 2024-04-24T22:45:49.598 INFO:tasks.workunit.client.1.smithi192.stdout:1/769: write d9/dd/d74/d83/fb7 [5118217,116492] 0 2024-04-24T22:45:49.770 INFO:tasks.workunit.client.1.smithi192.stdout:2/978: sync 2024-04-24T22:45:49.771 INFO:tasks.workunit.client.1.smithi192.stdout:3/991: sync 2024-04-24T22:45:49.771 INFO:tasks.workunit.client.1.smithi192.stdout:4/873: sync 2024-04-24T22:45:49.771 INFO:tasks.workunit.client.1.smithi192.stdout:0/928: sync 2024-04-24T22:45:49.771 INFO:tasks.workunit.client.1.smithi192.stdout:5/789: sync 2024-04-24T22:45:49.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/954: creat db/d5d/df0/d112/f12a x:0 0 0 2024-04-24T22:45:49.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/979: fsync d4/d9/d37/f4e 0 2024-04-24T22:45:49.861 INFO:tasks.workunit.client.1.smithi192.stdout:3/992: dwrite d2/d27/d28/dab/dca/fcf [0,4194304] 0 2024-04-24T22:45:49.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/929: symlink d3/d27/d28/d31/dbc/dea/l133 0 2024-04-24T22:45:49.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/874: unlink d2/db/d75/da1/d36/d5b/d64/l81 0 2024-04-24T22:45:49.893 INFO:tasks.workunit.client.1.smithi192.stdout:2/980: creat d4/d9/d10/d118/f13d x:0 0 0 2024-04-24T22:45:49.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/993: dread - d2/d4/fde zero size 2024-04-24T22:45:49.899 INFO:tasks.workunit.client.1.smithi192.stdout:0/930: creat d3/d27/d28/dcc/d5c/d80/d94/f134 x:0 0 0 2024-04-24T22:45:49.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/955: fsync db/dc/d22/d31/d33/db7/fc7 0 2024-04-24T22:45:49.904 INFO:tasks.workunit.client.1.smithi192.stdout:8/956: fdatasync db/dc/d22/d31/d33/fbc 0 2024-04-24T22:45:49.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/770: rmdir d9/d15/d2d/d63/dec 0 2024-04-24T22:45:49.905 INFO:tasks.workunit.client.1.smithi192.stdout:2/981: write d4/d9/d10/d6c/d95/dc0/fd8 [286497,14972] 0 2024-04-24T22:45:49.907 INFO:tasks.workunit.client.1.smithi192.stdout:3/994: creat d2/d4/d12/d1a/d1d/d55/d5f/db1/dc1/f130 x:0 0 0 2024-04-24T22:45:49.911 INFO:tasks.workunit.client.1.smithi192.stdout:4/875: mknod d2/db/d75/da1/dc9/df0/c130 0 2024-04-24T22:45:49.912 INFO:tasks.workunit.client.1.smithi192.stdout:4/876: chown d2/db/d75/da1/d7d/d9c/caf 84208 1 2024-04-24T22:45:49.917 INFO:tasks.workunit.client.1.smithi192.stdout:6/890: sync 2024-04-24T22:45:49.917 INFO:tasks.workunit.client.1.smithi192.stdout:6/891: chown d0/d8/f34 33321855 1 2024-04-24T22:45:49.918 INFO:tasks.workunit.client.1.smithi192.stdout:1/771: creat d9/d15/d5e/ffe x:0 0 0 2024-04-24T22:45:49.922 INFO:tasks.workunit.client.1.smithi192.stdout:3/995: mkdir d2/d27/dc6/da0/d131 0 2024-04-24T22:45:49.923 INFO:tasks.workunit.client.1.smithi192.stdout:2/982: dread d4/d29/f87 [0,4194304] 0 2024-04-24T22:45:49.925 INFO:tasks.workunit.client.1.smithi192.stdout:4/877: mknod d2/db/c131 0 2024-04-24T22:45:49.927 INFO:tasks.workunit.client.1.smithi192.stdout:8/957: unlink db/dc/d29/l4b 0 2024-04-24T22:45:49.931 INFO:tasks.workunit.client.1.smithi192.stdout:1/772: mkdir d9/dd/d74/dcd/dff 0 2024-04-24T22:45:49.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/983: fsync d4/d9/d10/f1e 0 2024-04-24T22:45:49.937 INFO:tasks.workunit.client.1.smithi192.stdout:4/878: creat d2/d5c/dce/f132 x:0 0 0 2024-04-24T22:45:49.945 INFO:tasks.workunit.client.1.smithi192.stdout:4/879: mkdir d2/d5c/dbb/de8/d133 0 2024-04-24T22:45:49.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/984: link d4/d9/d37/l131 d4/d29/d60/d9a/db3/l13e 0 2024-04-24T22:45:49.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/880: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/f12c [0,4194304] 0 2024-04-24T22:45:49.971 INFO:tasks.workunit.client.1.smithi192.stdout:4/881: dread d2/f17 [0,4194304] 0 2024-04-24T22:45:49.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/882: write d2/db/d5d/db3/fc2 [506003,45231] 0 2024-04-24T22:45:49.975 INFO:tasks.workunit.client.1.smithi192.stdout:4/883: write d2/db/d75/da1/d36/d5b/d28/d2f/d7e/f98 [2513243,114401] 0 2024-04-24T22:45:50.008 INFO:tasks.workunit.client.1.smithi192.stdout:4/884: creat d2/d5c/d107/f134 x:0 0 0 2024-04-24T22:45:50.008 INFO:tasks.workunit.client.1.smithi192.stdout:4/885: readlink d2/db/d75/da1/d36/d5b/d28/lc7 0 2024-04-24T22:45:50.012 INFO:tasks.workunit.client.1.smithi192.stdout:4/886: truncate d2/f3 3264190 0 2024-04-24T22:45:50.014 INFO:tasks.workunit.client.1.smithi192.stdout:4/887: mkdir d2/db/d75/da1/dc9/df0/d135 0 2024-04-24T22:45:50.017 INFO:tasks.workunit.client.1.smithi192.stdout:4/888: fsync d2/db/d75/da1/d7d/fca 0 2024-04-24T22:45:50.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/889: mkdir d2/db/d75/da1/d36/d5b/d28/deb/d12d/d136 0 2024-04-24T22:45:50.024 INFO:tasks.workunit.client.1.smithi192.stdout:4/890: symlink d2/db/d5d/db3/l137 0 2024-04-24T22:45:50.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/891: mknod d2/db/d75/da1/d7d/c138 0 2024-04-24T22:45:50.075 INFO:tasks.workunit.client.1.smithi192.stdout:5/790: dwrite d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:50.093 INFO:tasks.workunit.client.1.smithi192.stdout:0/931: write d3/d27/d28/dcc/f4d [3552547,108443] 0 2024-04-24T22:45:50.098 INFO:tasks.workunit.client.1.smithi192.stdout:0/932: symlink d3/d27/d28/d2e/d3f/l135 0 2024-04-24T22:45:50.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/933: write d3/d27/d28/dcc/d5c/d80/d94/f134 [31168,97413] 0 2024-04-24T22:45:50.117 INFO:tasks.workunit.client.1.smithi192.stdout:6/892: rename d0/d8/d3d/d3e/d7e/faf to d0/d8/d22/f145 0 2024-04-24T22:45:50.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/958: rename db/d5d/dcb/f105 to db/dc/d22/d31/d33/d8f/d127/f12b 0 2024-04-24T22:45:50.125 INFO:tasks.workunit.client.1.smithi192.stdout:6/893: dwrite d0/d8/d3d/d13e/f123 [0,4194304] 0 2024-04-24T22:45:50.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/894: chown d0/d8/d29/d58/fbe 30 1 2024-04-24T22:45:50.128 INFO:tasks.workunit.client.1.smithi192.stdout:0/934: rename d3/d27/d6a/dc4/c109 to d3/d27/d28/d2e/c136 0 2024-04-24T22:45:50.129 INFO:tasks.workunit.client.1.smithi192.stdout:0/935: fsync d3/d27/d28/dcc/fa1 0 2024-04-24T22:45:50.131 INFO:tasks.workunit.client.1.smithi192.stdout:0/936: chown d3/d6d/d79/fd5 30442 1 2024-04-24T22:45:50.132 INFO:tasks.workunit.client.1.smithi192.stdout:1/773: mkdir d9/d15/d2d/d63/d68/d100 0 2024-04-24T22:45:50.134 INFO:tasks.workunit.client.1.smithi192.stdout:6/895: dwrite d0/d8/f6f [0,4194304] 0 2024-04-24T22:45:50.137 INFO:tasks.workunit.client.1.smithi192.stdout:6/896: stat d0/d8/d18/d2e/f51 0 2024-04-24T22:45:50.139 INFO:tasks.workunit.client.1.smithi192.stdout:6/897: chown d0/d8/d22/dcb/de0/d10d/f136 29114025 1 2024-04-24T22:45:50.142 INFO:tasks.workunit.client.1.smithi192.stdout:2/985: rmdir d4/d9/d10/d6c/d8c/da5/de2 39 2024-04-24T22:45:50.158 INFO:tasks.workunit.client.1.smithi192.stdout:4/892: write d2/db/d10/fc6 [150769,44157] 0 2024-04-24T22:45:50.160 INFO:tasks.workunit.client.1.smithi192.stdout:5/791: mknod d0/d13/d3b/c111 0 2024-04-24T22:45:50.161 INFO:tasks.workunit.client.1.smithi192.stdout:4/893: stat d2/db/d75/da1/d53/lbc 0 2024-04-24T22:45:50.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/937: dread d3/d27/d28/d31/dbc/fd8 [0,4194304] 0 2024-04-24T22:45:50.163 INFO:tasks.workunit.client.1.smithi192.stdout:5/792: chown d0/d13/d19/l63 1122232 1 2024-04-24T22:45:50.163 INFO:tasks.workunit.client.1.smithi192.stdout:4/894: chown d2/db/d75/da1/d36/d70/ld9 13500775 1 2024-04-24T22:45:50.165 INFO:tasks.workunit.client.1.smithi192.stdout:5/793: write d0/d13/d19/d64/d4f/d7c/f109 [247045,51193] 0 2024-04-24T22:45:50.165 INFO:tasks.workunit.client.1.smithi192.stdout:0/938: rmdir d3/d27/d28/d2e/dc7/dd4 39 2024-04-24T22:45:50.167 INFO:tasks.workunit.client.1.smithi192.stdout:0/939: fsync d3/d27/d28/d2e/dc7/d124/f128 0 2024-04-24T22:45:50.167 INFO:tasks.workunit.client.1.smithi192.stdout:4/895: symlink d2/db/d75/da1/dc9/df0/d135/l139 0 2024-04-24T22:45:50.168 INFO:tasks.workunit.client.1.smithi192.stdout:0/940: chown d3/d27/d28/dcc/d5c/f10e 35 1 2024-04-24T22:45:50.172 INFO:tasks.workunit.client.1.smithi192.stdout:0/941: creat d3/d40/dc2/f137 x:0 0 0 2024-04-24T22:45:50.176 INFO:tasks.workunit.client.1.smithi192.stdout:0/942: truncate d3/d27/d28/dcc/d5c/d8a/fad 356577 0 2024-04-24T22:45:50.179 INFO:tasks.workunit.client.1.smithi192.stdout:0/943: creat d3/d27/d28/d2e/d3f/d12d/f138 x:0 0 0 2024-04-24T22:45:50.184 INFO:tasks.workunit.client.1.smithi192.stdout:0/944: truncate d3/d27/d28/d31/dbc/dea/f127 2546689 0 2024-04-24T22:45:50.185 INFO:tasks.workunit.client.1.smithi192.stdout:3/996: mknod d2/c132 0 2024-04-24T22:45:50.194 INFO:tasks.workunit.client.1.smithi192.stdout:3/997: dwrite d2/d106/f129 [0,4194304] 0 2024-04-24T22:45:50.195 INFO:tasks.workunit.client.1.smithi192.stdout:8/959: write db/dc/d22/f23 [4893866,60697] 0 2024-04-24T22:45:50.197 INFO:tasks.workunit.client.1.smithi192.stdout:3/998: chown d2/d4/d12/d39/ca7 4 1 2024-04-24T22:45:50.202 INFO:tasks.workunit.client.1.smithi192.stdout:8/960: mknod db/d43/d7c/d85/dcd/c12c 0 2024-04-24T22:45:50.203 INFO:tasks.workunit.client.1.smithi192.stdout:8/961: write f8 [4228420,21409] 0 2024-04-24T22:45:50.204 INFO:tasks.workunit.client.1.smithi192.stdout:6/898: truncate d0/d8/d3d/d3e/fea 1566093 0 2024-04-24T22:45:50.205 INFO:tasks.workunit.client.1.smithi192.stdout:8/962: chown db/dc/d22/d31/d33 5 1 2024-04-24T22:45:50.205 INFO:tasks.workunit.client.1.smithi192.stdout:2/986: write d4/d9/d10/d6c/d6f/da8/fac [844893,101226] 0 2024-04-24T22:45:50.209 INFO:tasks.workunit.client.1.smithi192.stdout:8/963: symlink db/d5d/l12d 0 2024-04-24T22:45:50.211 INFO:tasks.workunit.client.1.smithi192.stdout:6/899: mkdir d0/d8/d3d/d9d/da5/dc8/d110/d146 0 2024-04-24T22:45:50.214 INFO:tasks.workunit.client.1.smithi192.stdout:8/964: creat db/dc/d22/dca/f12e x:0 0 0 2024-04-24T22:45:50.215 INFO:tasks.workunit.client.1.smithi192.stdout:6/900: symlink d0/d8/d3d/d3e/d40/d44/l147 0 2024-04-24T22:45:50.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/965: readlink db/dc/d22/d31/l7a 0 2024-04-24T22:45:50.219 INFO:tasks.workunit.client.1.smithi192.stdout:4/896: dwrite d2/d5c/f86 [0,4194304] 0 2024-04-24T22:45:50.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/901: readlink d0/d8/d3d/d3e/d40/l65 0 2024-04-24T22:45:50.225 INFO:tasks.workunit.client.1.smithi192.stdout:0/945: write d3/d6d/ff2 [736134,55985] 0 2024-04-24T22:45:50.228 INFO:tasks.workunit.client.1.smithi192.stdout:8/966: truncate db/dc/d22/d31/fc1 1968560 0 2024-04-24T22:45:50.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/987: dread d4/d9/d37/f4e [0,4194304] 0 2024-04-24T22:45:50.233 INFO:tasks.workunit.client.1.smithi192.stdout:4/897: unlink d2/l69 0 2024-04-24T22:45:50.241 INFO:tasks.workunit.client.1.smithi192.stdout:8/967: mkdir db/dc/d22/d31/d33/db7/d12f 0 2024-04-24T22:45:50.242 INFO:tasks.workunit.client.1.smithi192.stdout:2/988: stat d4/d9/d10/d6c/d8c/da5/de2/f106 0 2024-04-24T22:45:50.249 INFO:tasks.workunit.client.1.smithi192.stdout:6/902: creat d0/d8/d3d/d9d/dd6/f148 x:0 0 0 2024-04-24T22:45:50.250 INFO:tasks.workunit.client.1.smithi192.stdout:6/903: readlink d0/d8/d18/d2e/la4 0 2024-04-24T22:45:50.250 INFO:tasks.workunit.client.1.smithi192.stdout:2/989: truncate d4/d9/f4c 4756837 0 2024-04-24T22:45:50.251 INFO:tasks.workunit.client.1.smithi192.stdout:6/904: chown d0/d8/d29/d5a/f10e 59173835 1 2024-04-24T22:45:50.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/968: symlink db/d43/d7c/l130 0 2024-04-24T22:45:50.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/969: symlink db/dc/d22/d31/d33/d81/d8a/l131 2 2024-04-24T22:45:50.255 INFO:tasks.workunit.client.1.smithi192.stdout:4/898: link d2/d9/d6b/cf3 d2/db/d75/c13a 0 2024-04-24T22:45:50.256 INFO:tasks.workunit.client.1.smithi192.stdout:4/899: write d2/d5c/d107/f127 [581468,11392] 0 2024-04-24T22:45:50.257 INFO:tasks.workunit.client.1.smithi192.stdout:6/905: creat d0/d8/d3d/d9d/da5/dc8/d110/f149 x:0 0 0 2024-04-24T22:45:50.259 INFO:tasks.workunit.client.1.smithi192.stdout:8/970: creat db/d43/d7c/ded/d10d/f132 x:0 0 0 2024-04-24T22:45:50.261 INFO:tasks.workunit.client.1.smithi192.stdout:8/971: chown db/dc/d29/d9f/lb6 186 1 2024-04-24T22:45:50.263 INFO:tasks.workunit.client.1.smithi192.stdout:4/900: creat d2/d5c/dbb/f13b x:0 0 0 2024-04-24T22:45:50.264 INFO:tasks.workunit.client.1.smithi192.stdout:2/990: link d4/d9/f90 d4/d9/d37/de5/d12b/f13f 0 2024-04-24T22:45:50.265 INFO:tasks.workunit.client.1.smithi192.stdout:6/906: dwrite d0/d8/d3d/d3e/d40/d44/f73 [4194304,4194304] 0 2024-04-24T22:45:50.266 INFO:tasks.workunit.client.1.smithi192.stdout:4/901: write d2/db/d75/da1/d36/d5b/d28/d2f/fe6 [738883,48430] 0 2024-04-24T22:45:50.269 INFO:tasks.workunit.client.1.smithi192.stdout:8/972: creat db/dc/d29/f133 x:0 0 0 2024-04-24T22:45:50.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/991: fdatasync d4/d29/fe9 0 2024-04-24T22:45:50.272 INFO:tasks.workunit.client.1.smithi192.stdout:6/907: read d0/d8/d29/d4c/d4d/d6a/ff9 [1585704,23759] 0 2024-04-24T22:45:50.284 INFO:tasks.workunit.client.1.smithi192.stdout:6/908: rename d0/d8/d18/d2e/c45 to d0/d8/d29/d4c/d4d/d83/c14a 0 2024-04-24T22:45:50.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/973: mknod db/dc/d22/d31/d33/db7/d12f/c134 0 2024-04-24T22:45:50.320 INFO:tasks.workunit.client.1.smithi192.stdout:4/902: creat d2/db/d75/da1/d7d/f13c x:0 0 0 2024-04-24T22:45:50.321 INFO:tasks.workunit.client.1.smithi192.stdout:6/909: mkdir d0/d8/d3d/d9d/d14b 0 2024-04-24T22:45:50.322 INFO:tasks.workunit.client.1.smithi192.stdout:8/974: mkdir db/d43/d71/d135 0 2024-04-24T22:45:50.327 INFO:tasks.workunit.client.1.smithi192.stdout:4/903: mkdir d2/db/d5d/d77/d82/d13d 0 2024-04-24T22:45:50.329 INFO:tasks.workunit.client.1.smithi192.stdout:0/946: write d3/d27/d28/dcc/d5c/d8a/fad [42027,110305] 0 2024-04-24T22:45:50.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/947: stat d3/d27/d28/d31/fb5 0 2024-04-24T22:45:50.335 INFO:tasks.workunit.client.1.smithi192.stdout:4/904: rename d2/d5c/dce to d2/db/d75/da1/d36/d5b/d28/d13e 0 2024-04-24T22:45:50.338 INFO:tasks.workunit.client.1.smithi192.stdout:4/905: chown d2/db/d75/da1/d36/d5b/d28/fec 132237951 1 2024-04-24T22:45:50.345 INFO:tasks.workunit.client.1.smithi192.stdout:4/906: mknod d2/d5c/dbb/de8/c13f 0 2024-04-24T22:45:50.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/975: dread db/dc/d29/f89 [0,4194304] 0 2024-04-24T22:45:50.345 INFO:tasks.workunit.client.1.smithi192.stdout:6/910: dread d0/d8/d3d/f5f [0,4194304] 0 2024-04-24T22:45:50.347 INFO:tasks.workunit.client.1.smithi192.stdout:6/911: stat d0/d8/d29/d4c/d4d/d83/dde/l10c 0 2024-04-24T22:45:50.360 INFO:tasks.workunit.client.1.smithi192.stdout:0/948: link d3/d27/d28/d2e/d3f/c58 d3/d6d/d79/c139 0 2024-04-24T22:45:50.360 INFO:tasks.workunit.client.1.smithi192.stdout:0/949: dread - d3/d27/d3c/d64/f130 zero size 2024-04-24T22:45:50.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/976: dwrite db/dc/d29/f47 [0,4194304] 0 2024-04-24T22:45:50.364 INFO:tasks.workunit.client.1.smithi192.stdout:8/977: stat db/dc/d22/d31/d33/d81/fb1 0 2024-04-24T22:45:50.364 INFO:tasks.workunit.client.1.smithi192.stdout:6/912: dwrite d0/d8/d3d/d3e/d40/d44/f104 [0,4194304] 0 2024-04-24T22:45:50.371 INFO:tasks.workunit.client.1.smithi192.stdout:0/950: write d3/d27/d28/d31/dbc/fd8 [4581554,39963] 0 2024-04-24T22:45:50.372 INFO:tasks.workunit.client.1.smithi192.stdout:0/951: chown d3/d40/f9b 9017856 1 2024-04-24T22:45:50.381 INFO:tasks.workunit.client.1.smithi192.stdout:0/952: dwrite d3/d27/d28/dcc/d5c/d80/d94/fbe [0,4194304] 0 2024-04-24T22:45:50.395 INFO:tasks.workunit.client.1.smithi192.stdout:2/992: write d4/d9/f8b [1662372,13195] 0 2024-04-24T22:45:50.397 INFO:tasks.workunit.client.1.smithi192.stdout:0/953: dwrite d3/d27/d28/f56 [0,4194304] 0 2024-04-24T22:45:50.400 INFO:tasks.workunit.client.1.smithi192.stdout:8/978: creat db/dc/d22/d31/f136 x:0 0 0 2024-04-24T22:45:50.406 INFO:tasks.workunit.client.1.smithi192.stdout:2/993: dread - d4/d9/d10/d6c/f113 zero size 2024-04-24T22:45:50.409 INFO:tasks.workunit.client.1.smithi192.stdout:8/979: read db/d43/d76/d92/f9b [1474034,60619] 0 2024-04-24T22:45:50.410 INFO:tasks.workunit.client.1.smithi192.stdout:0/954: creat d3/d27/d28/d2e/dc7/d124/f13a x:0 0 0 2024-04-24T22:45:50.421 INFO:tasks.workunit.client.1.smithi192.stdout:8/980: symlink db/ddf/l137 0 2024-04-24T22:45:50.421 INFO:tasks.workunit.client.1.smithi192.stdout:2/994: dwrite d4/d9/d10/d6c/d95/db7/dbf/f12a [0,4194304] 0 2024-04-24T22:45:50.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/955: unlink d3/f1c 0 2024-04-24T22:45:50.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/956: chown d3/d40/dc2 98 1 2024-04-24T22:45:50.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: Adjusting osd_memory_target on smithi192 to 2581M 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:50.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:50.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:45:50.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:45:50.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:50 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:45:50.588 INFO:tasks.workunit.client.1.smithi192.stdout:8/981: mkdir db/d5d/dcb/d138 0 2024-04-24T22:45:50.595 INFO:tasks.workunit.client.1.smithi192.stdout:0/957: link d3/d27/d28/f20 d3/d27/d3c/d6c/db3/df0/f13b 0 2024-04-24T22:45:50.600 INFO:tasks.workunit.client.1.smithi192.stdout:0/958: rename d3/d27/d3c/d6c/db3/fe9 to d3/d27/d28/d2e/dc7/d124/f13c 0 2024-04-24T22:45:50.605 INFO:tasks.workunit.client.1.smithi192.stdout:4/907: dwrite d2/d9/fdb [0,4194304] 0 2024-04-24T22:45:50.606 INFO:tasks.workunit.client.1.smithi192.stdout:0/959: rename d3/d27/d28/d2e/f4c to d3/d27/d28/dcc/d5c/d11b/f13d 0 2024-04-24T22:45:50.613 INFO:tasks.workunit.client.1.smithi192.stdout:0/960: rmdir d3/d27/d3c/d3b 39 2024-04-24T22:45:50.617 INFO:tasks.workunit.client.1.smithi192.stdout:4/908: symlink d2/d9/l140 0 2024-04-24T22:45:50.620 INFO:tasks.workunit.client.1.smithi192.stdout:0/961: symlink d3/d27/d28/dcc/d5c/d80/dac/l13e 0 2024-04-24T22:45:50.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/913: dwrite d0/d8/d29/d58/f9f [0,4194304] 0 2024-04-24T22:45:50.629 INFO:tasks.workunit.client.1.smithi192.stdout:0/962: dwrite d3/d27/d3c/fab [4194304,4194304] 0 2024-04-24T22:45:50.634 INFO:tasks.workunit.client.1.smithi192.stdout:6/914: rmdir d0/d8/d3d 39 2024-04-24T22:45:50.639 INFO:tasks.workunit.client.1.smithi192.stdout:6/915: rmdir d0/d8/d18/df1 39 2024-04-24T22:45:50.646 INFO:tasks.workunit.client.1.smithi192.stdout:0/963: fsync d3/d27/d3c/fab 0 2024-04-24T22:45:50.647 INFO:tasks.workunit.client.1.smithi192.stdout:6/916: creat d0/d8/d18/d10b/f14c x:0 0 0 2024-04-24T22:45:50.648 INFO:tasks.workunit.client.1.smithi192.stdout:6/917: fdatasync d0/d8/d29/d5a/f10e 0 2024-04-24T22:45:50.649 INFO:tasks.workunit.client.1.smithi192.stdout:8/982: write db/dc/d22/f5e [3421062,70150] 0 2024-04-24T22:45:50.650 INFO:tasks.workunit.client.1.smithi192.stdout:8/983: chown db/d43/d76/d92/cc2 0 1 2024-04-24T22:45:50.651 INFO:tasks.workunit.client.1.smithi192.stdout:0/964: creat d3/d27/d3c/d6c/db3/df0/f13f x:0 0 0 2024-04-24T22:45:50.660 INFO:tasks.workunit.client.1.smithi192.stdout:6/918: rename d0/d8/d18/d2e/c4a to d0/d8/d18/dff/c14d 0 2024-04-24T22:45:50.662 INFO:tasks.workunit.client.1.smithi192.stdout:8/984: creat db/d5d/d113/f139 x:0 0 0 2024-04-24T22:45:50.663 INFO:tasks.workunit.client.1.smithi192.stdout:8/985: write db/dc/f17 [2391196,21921] 0 2024-04-24T22:45:50.667 INFO:tasks.workunit.client.1.smithi192.stdout:6/919: dread d0/d8/d3d/f62 [0,4194304] 0 2024-04-24T22:45:50.668 INFO:tasks.workunit.client.1.smithi192.stdout:6/920: dread - d0/d8/d3d/d3e/d40/f141 zero size 2024-04-24T22:45:50.670 INFO:tasks.workunit.client.1.smithi192.stdout:8/986: rename ca to db/d5d/d74/c13a 0 2024-04-24T22:45:50.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/987: truncate db/d43/d7c/ded/f10a 476402 0 2024-04-24T22:45:50.674 INFO:tasks.workunit.client.1.smithi192.stdout:6/921: dread d0/d8/d29/d4c/f74 [0,4194304] 0 2024-04-24T22:45:50.675 INFO:tasks.workunit.client.1.smithi192.stdout:1/774: sync 2024-04-24T22:45:50.676 INFO:tasks.workunit.client.1.smithi192.stdout:0/965: getdents d3/d27 0 2024-04-24T22:45:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: Adjusting osd_memory_target on smithi192 to 2581M 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:45:50.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:50 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:45:50.679 INFO:tasks.workunit.client.1.smithi192.stdout:1/775: write d9/d15/d2d/d63/f79 [4732303,77692] 0 2024-04-24T22:45:50.682 INFO:tasks.workunit.client.1.smithi192.stdout:0/966: dread d3/d27/d28/dcc/f4d [8388608,4194304] 0 2024-04-24T22:45:50.686 INFO:tasks.workunit.client.1.smithi192.stdout:4/909: dwrite d2/db/d75/da1/d36/d5b/d28/d13e/fd2 [0,4194304] 0 2024-04-24T22:45:50.686 INFO:tasks.workunit.client.1.smithi192.stdout:8/988: mkdir db/d43/d7c/d85/df7/d13b 0 2024-04-24T22:45:50.688 INFO:tasks.workunit.client.1.smithi192.stdout:6/922: dwrite d0/d8/d29/d5a/d63/f117 [0,4194304] 0 2024-04-24T22:45:50.690 INFO:tasks.workunit.client.1.smithi192.stdout:8/989: write db/d5d/f10f [656175,97827] 0 2024-04-24T22:45:50.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/923: chown d0/d8/d3d/d3e/d40/d44/f126 24879375 1 2024-04-24T22:45:50.696 INFO:tasks.workunit.client.1.smithi192.stdout:4/910: truncate d2/db/d5d/f115 4706809 0 2024-04-24T22:45:50.697 INFO:tasks.workunit.client.1.smithi192.stdout:1/776: mknod d9/dd/c101 0 2024-04-24T22:45:50.698 INFO:tasks.workunit.client.1.smithi192.stdout:6/924: readlink d0/d8/d3d/la2 0 2024-04-24T22:45:50.701 INFO:tasks.workunit.client.1.smithi192.stdout:8/990: creat db/dc/d22/d31/d33/d8f/f13c x:0 0 0 2024-04-24T22:45:50.701 INFO:tasks.workunit.client.1.smithi192.stdout:4/911: rmdir d2/d5c 39 2024-04-24T22:45:50.704 INFO:tasks.workunit.client.1.smithi192.stdout:1/777: fdatasync d9/d15/d2d/d48/d93/f98 0 2024-04-24T22:45:50.709 INFO:tasks.workunit.client.1.smithi192.stdout:8/991: creat db/d43/d7c/d85/db4/f13d x:0 0 0 2024-04-24T22:45:50.711 INFO:tasks.workunit.client.1.smithi192.stdout:8/992: dread - db/d43/d7c/ded/d10d/f117 zero size 2024-04-24T22:45:50.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/925: write d0/d8/d3d/f49 [2421929,30598] 0 2024-04-24T22:45:50.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/912: dwrite d2/d5c/dbb/f13b [0,4194304] 0 2024-04-24T22:45:50.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/913: stat d2/db 0 2024-04-24T22:45:50.723 INFO:tasks.workunit.client.1.smithi192.stdout:8/993: fsync db/dc/f1b 0 2024-04-24T22:45:50.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/926: mknod d0/d8/d3d/d9d/d14b/c14e 0 2024-04-24T22:45:50.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/927: chown d0/d8/d29/d4c/d4d/d83/dde 17 1 2024-04-24T22:45:50.728 INFO:tasks.workunit.client.1.smithi192.stdout:8/994: chown db/dc/d29/c61 11513136 1 2024-04-24T22:45:50.731 INFO:tasks.workunit.client.1.smithi192.stdout:8/995: mknod db/d5d/d113/c13e 0 2024-04-24T22:45:50.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/778: dwrite d9/d15/f1b [0,4194304] 0 2024-04-24T22:45:50.741 INFO:tasks.workunit.client.1.smithi192.stdout:6/928: write d0/d8/d29/d5a/d63/f81 [4602894,96403] 0 2024-04-24T22:45:50.742 INFO:tasks.workunit.client.1.smithi192.stdout:1/779: dread - d9/dd/d74/dcd/ff7 zero size 2024-04-24T22:45:50.743 INFO:tasks.workunit.client.1.smithi192.stdout:8/996: write db/dc/d22/f35 [3663902,84025] 0 2024-04-24T22:45:50.744 INFO:tasks.workunit.client.1.smithi192.stdout:6/929: symlink d0/d8/d22/l14f 0 2024-04-24T22:45:50.746 INFO:tasks.workunit.client.1.smithi192.stdout:1/780: read - d9/d15/d2d/d4a/f8b zero size 2024-04-24T22:45:50.747 INFO:tasks.workunit.client.1.smithi192.stdout:6/930: mkdir d0/d8/d3d/d3e/d40/d150 0 2024-04-24T22:45:50.747 INFO:tasks.workunit.client.1.smithi192.stdout:1/781: stat d9/d15/d2d/d31/c27 0 2024-04-24T22:45:50.750 INFO:tasks.workunit.client.1.smithi192.stdout:6/931: mknod d0/d8/d22/dcb/de0/c151 0 2024-04-24T22:45:50.753 INFO:tasks.workunit.client.1.smithi192.stdout:6/932: fdatasync d0/d8/d29/d4c/d4d/d83/fd7 0 2024-04-24T22:45:50.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/933: readlink d0/d8/d29/d5a/l11b 0 2024-04-24T22:45:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:8/997: dread db/dc/d22/f5e [0,4194304] 0 2024-04-24T22:45:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:6/934: creat d0/d8/d18/d143/f152 x:0 0 0 2024-04-24T22:45:50.758 INFO:tasks.workunit.client.1.smithi192.stdout:1/782: read d9/d15/d2d/d63/d68/fe8 [231670,101944] 0 2024-04-24T22:45:50.760 INFO:tasks.workunit.client.1.smithi192.stdout:8/998: rename db/d43/d7c/ded/d10d/f117 to db/d43/d7c/d85/f13f 0 2024-04-24T22:45:50.775 INFO:tasks.workunit.client.1.smithi192.stdout:8/999: write db/dc/d22/dca/ff6 [96620,92224] 0 2024-04-24T22:45:50.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/999: sync 2024-04-24T22:45:50.882 INFO:tasks.workunit.client.1.smithi192.stdout:5/794: sync 2024-04-24T22:45:50.884 INFO:tasks.workunit.client.1.smithi192.stdout:5/795: truncate d0/d1b/d62/d10f/d2c/d3c/d91/ff4 1566198 0 2024-04-24T22:45:50.887 INFO:tasks.workunit.client.1.smithi192.stdout:5/796: write d0/d5b/f75 [1195386,108728] 0 2024-04-24T22:45:50.975 INFO:tasks.workunit.client.1.smithi192.stdout:5/797: truncate d0/d13/d19/d64/d4f/d7c/ff2 1309479 0 2024-04-24T22:45:51.018 INFO:tasks.workunit.client.1.smithi192.stdout:2/995: sync 2024-04-24T22:45:51.066 INFO:tasks.workunit.client.1.smithi192.stdout:2/996: mknod d4/d9/d9f/d126/c140 0 2024-04-24T22:45:51.072 INFO:tasks.workunit.client.1.smithi192.stdout:5/798: getdents d0/d1b/d50 0 2024-04-24T22:45:51.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/997: write d4/d29/fe9 [4169388,60290] 0 2024-04-24T22:45:51.092 INFO:tasks.workunit.client.1.smithi192.stdout:5/799: write d0/d1b/f65 [4220899,117050] 0 2024-04-24T22:45:51.097 INFO:tasks.workunit.client.1.smithi192.stdout:5/800: rename d0/d5b to d0/d1b/d62/d10f/d2c/d3c/d91/d112 0 2024-04-24T22:45:51.097 INFO:tasks.workunit.client.1.smithi192.stdout:2/998: write d4/d29/d60/d9a/f111 [1603104,74838] 0 2024-04-24T22:45:51.101 INFO:tasks.workunit.client.1.smithi192.stdout:5/801: creat d0/d1b/d62/d10f/d2c/d3c/d91/f113 x:0 0 0 2024-04-24T22:45:51.106 INFO:tasks.workunit.client.1.smithi192.stdout:2/999: write d4/d29/d60/d9a/db3/f129 [66235,83111] 0 2024-04-24T22:45:51.110 INFO:tasks.workunit.client.1.smithi192.stdout:5/802: dwrite d0/d13/dd1/d108/f46 [0,4194304] 0 2024-04-24T22:45:51.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/803: write d0/d1b/f24 [344533,68220] 0 2024-04-24T22:45:51.122 INFO:tasks.workunit.client.1.smithi192.stdout:5/804: rmdir d0/d13/d19/d64 39 2024-04-24T22:45:51.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/805: dwrite d0/d13/d19/d83/dad/fe9 [0,4194304] 0 2024-04-24T22:45:51.138 INFO:tasks.workunit.client.1.smithi192.stdout:5/806: fdatasync d0/d1b/f6d 0 2024-04-24T22:45:51.138 INFO:tasks.workunit.client.1.smithi192.stdout:0/967: sync 2024-04-24T22:45:51.141 INFO:tasks.workunit.client.1.smithi192.stdout:5/807: creat d0/d1b/d62/dc1/dc2/f114 x:0 0 0 2024-04-24T22:45:51.145 INFO:tasks.workunit.client.1.smithi192.stdout:0/968: read d3/d27/d28/f8e [1946184,122047] 0 2024-04-24T22:45:51.148 INFO:tasks.workunit.client.1.smithi192.stdout:0/969: chown d3/d27/d28/d31/dbc/dea/f127 11813390 1 2024-04-24T22:45:51.149 INFO:tasks.workunit.client.1.smithi192.stdout:5/808: creat d0/d13/d19/f115 x:0 0 0 2024-04-24T22:45:51.151 INFO:tasks.workunit.client.1.smithi192.stdout:0/970: getdents d3/d27/d28/d2e/d3f 0 2024-04-24T22:45:51.154 INFO:tasks.workunit.client.1.smithi192.stdout:5/809: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:51.155 INFO:tasks.workunit.client.1.smithi192.stdout:0/971: symlink d3/d27/d28/dcc/l140 0 2024-04-24T22:45:51.156 INFO:tasks.workunit.client.1.smithi192.stdout:0/972: dread - d3/d27/d28/d2e/dc7/f11d zero size 2024-04-24T22:45:51.156 INFO:tasks.workunit.client.1.smithi192.stdout:5/810: dread - d0/d13/d19/d64/de7/f10d zero size 2024-04-24T22:45:51.157 INFO:tasks.workunit.client.1.smithi192.stdout:5/811: read - d0/d1b/d62/fa7 zero size 2024-04-24T22:45:51.161 INFO:tasks.workunit.client.1.smithi192.stdout:5/812: mkdir d0/d1b/d50/d116 0 2024-04-24T22:45:51.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/813: dread - d0/d13/d19/d71/d88/f107 zero size 2024-04-24T22:45:51.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/814: rmdir d0/d13/d19/d71/d7f 39 2024-04-24T22:45:51.168 INFO:tasks.workunit.client.1.smithi192.stdout:0/973: dwrite d3/d27/d28/f8e [0,4194304] 0 2024-04-24T22:45:51.168 INFO:tasks.workunit.client.1.smithi192.stdout:5/815: truncate d0/d13/d3b/f55 4383867 0 2024-04-24T22:45:51.177 INFO:tasks.workunit.client.1.smithi192.stdout:4/914: sync 2024-04-24T22:45:51.177 INFO:tasks.workunit.client.1.smithi192.stdout:0/974: dwrite d3/d27/d28/d2e/dc7/fd0 [0,4194304] 0 2024-04-24T22:45:51.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/915: chown d2/d5c/dbb/de8/d133 460606 1 2024-04-24T22:45:51.183 INFO:tasks.workunit.client.1.smithi192.stdout:4/916: mkdir d2/db/d75/da1/d36/d5b/d28/d13e/d141 0 2024-04-24T22:45:51.187 INFO:tasks.workunit.client.1.smithi192.stdout:5/816: dwrite d0/d13/d19/f1a [0,4194304] 0 2024-04-24T22:45:51.188 INFO:tasks.workunit.client.1.smithi192.stdout:4/917: rename d2/db/d75/d10d/l112 to d2/db/d75/da1/d36/d5b/d28/d2f/d7e/l142 0 2024-04-24T22:45:51.194 INFO:tasks.workunit.client.1.smithi192.stdout:5/817: getdents d0/d1b/d62/d10f/d2c/d3c/d91/d112/d105 0 2024-04-24T22:45:51.196 INFO:tasks.workunit.client.1.smithi192.stdout:0/975: dread d3/d27/d28/dcc/f72 [0,4194304] 0 2024-04-24T22:45:51.199 INFO:tasks.workunit.client.1.smithi192.stdout:5/818: dread d0/d13/d19/d64/d4f/d6c/ff6 [0,4194304] 0 2024-04-24T22:45:51.199 INFO:tasks.workunit.client.1.smithi192.stdout:0/976: mknod d3/d27/d28/dcc/d5c/d80/dac/c141 0 2024-04-24T22:45:51.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/977: fsync d3/d6d/ff2 0 2024-04-24T22:45:51.200 INFO:tasks.workunit.client.1.smithi192.stdout:5/819: readlink d0/d13/d19/d71/dbf/l102 0 2024-04-24T22:45:51.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/978: readlink d3/d27/d3c/d6c/l70 0 2024-04-24T22:45:51.211 INFO:tasks.workunit.client.1.smithi192.stdout:4/918: write d2/db/d75/da1/d36/d5b/f106 [26852,98950] 0 2024-04-24T22:45:51.213 INFO:tasks.workunit.client.1.smithi192.stdout:4/919: symlink d2/d5c/d107/l143 0 2024-04-24T22:45:51.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/979: getdents d3/d27/d28/dcc/da8/d111 0 2024-04-24T22:45:51.216 INFO:tasks.workunit.client.1.smithi192.stdout:4/920: rmdir d2/db/d75/da1/d7d 39 2024-04-24T22:45:51.221 INFO:tasks.workunit.client.1.smithi192.stdout:4/921: fsync d2/db/d75/da1/d53/fcc 0 2024-04-24T22:45:51.223 INFO:tasks.workunit.client.1.smithi192.stdout:5/820: dread d0/d13/f3f [0,4194304] 0 2024-04-24T22:45:51.225 INFO:tasks.workunit.client.1.smithi192.stdout:1/783: sync 2024-04-24T22:45:51.225 INFO:tasks.workunit.client.1.smithi192.stdout:6/935: sync 2024-04-24T22:45:51.226 INFO:tasks.workunit.client.1.smithi192.stdout:1/784: readlink d9/lf9 0 2024-04-24T22:45:51.226 INFO:tasks.workunit.client.1.smithi192.stdout:6/936: chown d0/f82 17762535 1 2024-04-24T22:45:51.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/980: rename d3/d6d/d79/l100 to d3/l142 0 2024-04-24T22:45:51.230 INFO:tasks.workunit.client.1.smithi192.stdout:4/922: mkdir d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/d144 0 2024-04-24T22:45:51.232 INFO:tasks.workunit.client.1.smithi192.stdout:1/785: mkdir d9/d15/d5e/d90/da9/d102 0 2024-04-24T22:45:51.234 INFO:tasks.workunit.client.1.smithi192.stdout:0/981: creat d3/d27/d28/dcc/da8/d111/f143 x:0 0 0 2024-04-24T22:45:51.236 INFO:tasks.workunit.client.1.smithi192.stdout:4/923: chown d2/db/d5d/dfc 4 1 2024-04-24T22:45:51.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/982: rmdir d3/d27/d28/dcc/d5c/d8a 39 2024-04-24T22:45:51.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/821: creat d0/d13/f117 x:0 0 0 2024-04-24T22:45:51.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/822: chown d0/d13/c7b 50714753 1 2024-04-24T22:45:51.261 INFO:tasks.workunit.client.1.smithi192.stdout:5/823: rename d0/d13/d19/d83/dad/fd2 to d0/f118 0 2024-04-24T22:45:51.269 INFO:tasks.workunit.client.1.smithi192.stdout:6/937: truncate d0/d8/d18/d2e/de8/f124 461691 0 2024-04-24T22:45:51.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/786: write d9/f42 [1542729,80107] 0 2024-04-24T22:45:51.276 INFO:tasks.workunit.client.1.smithi192.stdout:1/787: chown d9/dd/d74/caa 0 1 2024-04-24T22:45:51.277 INFO:tasks.workunit.client.1.smithi192.stdout:4/924: write d2/db/d5d/d108/fc3 [4571287,82250] 0 2024-04-24T22:45:51.279 INFO:tasks.workunit.client.1.smithi192.stdout:4/925: truncate d2/db/d10/f126 630511 0 2024-04-24T22:45:51.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/938: dread d0/d8/d18/f3f [0,4194304] 0 2024-04-24T22:45:51.285 INFO:tasks.workunit.client.1.smithi192.stdout:6/939: dread - d0/d8/f131 zero size 2024-04-24T22:45:51.290 INFO:tasks.workunit.client.1.smithi192.stdout:0/983: dwrite d3/d27/d3c/d3b/f65 [0,4194304] 0 2024-04-24T22:45:51.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/940: mkdir d0/d8/d3d/d3e/df3/d153 0 2024-04-24T22:45:51.298 INFO:tasks.workunit.client.1.smithi192.stdout:1/788: dread d9/d15/d2d/d31/fab [0,4194304] 0 2024-04-24T22:45:51.302 INFO:tasks.workunit.client.1.smithi192.stdout:5/824: write d0/d13/d19/d71/dbf/fec [549354,121991] 0 2024-04-24T22:45:51.303 INFO:tasks.workunit.client.1.smithi192.stdout:1/789: symlink d9/dd/d74/dcd/l103 0 2024-04-24T22:45:51.306 INFO:tasks.workunit.client.1.smithi192.stdout:0/984: unlink d3/d27/d28/c1e 0 2024-04-24T22:45:51.306 INFO:tasks.workunit.client.1.smithi192.stdout:6/941: symlink d0/d8/d3d/d9d/dd6/l154 0 2024-04-24T22:45:51.311 INFO:tasks.workunit.client.1.smithi192.stdout:1/790: mknod d9/d15/d5e/c104 0 2024-04-24T22:45:51.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/791: truncate d9/fa 8455716 0 2024-04-24T22:45:51.316 INFO:tasks.workunit.client.1.smithi192.stdout:6/942: mkdir d0/d8/d3d/d3e/d40/d155 0 2024-04-24T22:45:51.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/825: dread d0/d13/d19/fe4 [0,4194304] 0 2024-04-24T22:45:51.321 INFO:tasks.workunit.client.1.smithi192.stdout:1/792: dwrite d9/d15/d5e/ffe [0,4194304] 0 2024-04-24T22:45:51.328 INFO:tasks.workunit.client.1.smithi192.stdout:6/943: truncate d0/d8/d29/f7c 1097485 0 2024-04-24T22:45:51.331 INFO:tasks.workunit.client.1.smithi192.stdout:5/826: creat d0/d13/d19/d64/d4f/d52/f119 x:0 0 0 2024-04-24T22:45:51.331 INFO:tasks.workunit.client.1.smithi192.stdout:5/827: readlink d0/l98 0 2024-04-24T22:45:51.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/985: getdents d3/d27/d3c 0 2024-04-24T22:45:51.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/828: dread - d0/d13/d3b/fda zero size 2024-04-24T22:45:51.340 INFO:tasks.workunit.client.1.smithi192.stdout:0/986: mkdir d3/d27/d3c/d6c/d144 0 2024-04-24T22:45:51.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/829: read d0/d1b/d62/d10f/d2c/f5c [2204795,128239] 0 2024-04-24T22:45:51.351 INFO:tasks.workunit.client.1.smithi192.stdout:5/830: fdatasync d0/d1b/d62/d10f/d2c/d3c/f5a 0 2024-04-24T22:45:51.353 INFO:tasks.workunit.client.1.smithi192.stdout:5/831: write d0/d1b/d62/d10f/d2c/d3c/d91/f113 [925210,82174] 0 2024-04-24T22:45:51.354 INFO:tasks.workunit.client.1.smithi192.stdout:0/987: rmdir d3/d27/d3c/d6c/d144 0 2024-04-24T22:45:51.354 INFO:tasks.workunit.client.1.smithi192.stdout:5/832: readlink d0/d13/d19/d71/l74 0 2024-04-24T22:45:51.360 INFO:tasks.workunit.client.1.smithi192.stdout:1/793: write d9/dd/f41 [3712506,115839] 0 2024-04-24T22:45:51.365 INFO:tasks.workunit.client.1.smithi192.stdout:0/988: rename d3/d27/d3c/d64/f84 to d3/d27/d28/dcc/d5c/d80/d83/dd9/f145 0 2024-04-24T22:45:51.365 INFO:tasks.workunit.client.1.smithi192.stdout:6/944: truncate d0/d8/d29/d5a/d63/f81 2760313 0 2024-04-24T22:45:51.373 INFO:tasks.workunit.client.1.smithi192.stdout:1/794: fsync d9/d15/d2d/d31/f62 0 2024-04-24T22:45:51.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/833: link d0/d1b/d62/d10f/d2c/d3c/d91/d112/f7a d0/d1b/d62/d10f/d2c/d3c/d91/d112/d105/f11a 0 2024-04-24T22:45:51.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/834: write d0/d1b/f24 [2518308,75466] 0 2024-04-24T22:45:51.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/989: dread d3/d27/d3c/d6c/db3/df0/f13b [4194304,4194304] 0 2024-04-24T22:45:51.389 INFO:tasks.workunit.client.1.smithi192.stdout:1/795: symlink d9/d15/d2d/l105 0 2024-04-24T22:45:51.390 INFO:tasks.workunit.client.1.smithi192.stdout:6/945: write d0/d8/d29/d58/fbe [348317,63195] 0 2024-04-24T22:45:51.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/835: mknod d0/d13/d19/da9/c11b 0 2024-04-24T22:45:51.399 INFO:tasks.workunit.client.1.smithi192.stdout:5/836: unlink d0/d1b/d62/d10f/d2c/d3c/f77 0 2024-04-24T22:45:51.401 INFO:tasks.workunit.client.1.smithi192.stdout:0/990: rename d3/d27/d28/d2e/d3f/c57 to d3/d27/d28/d2e/dc7/dd4/c146 0 2024-04-24T22:45:51.406 INFO:tasks.workunit.client.1.smithi192.stdout:5/837: fdatasync d0/d1b/d62/d10f/d2c/f3d 0 2024-04-24T22:45:51.413 INFO:tasks.workunit.client.1.smithi192.stdout:5/838: dwrite d0/d13/d19/d83/dad/fd8 [0,4194304] 0 2024-04-24T22:45:51.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/796: write d9/d15/d2d/d4a/fd0 [795945,22258] 0 2024-04-24T22:45:51.414 INFO:tasks.workunit.client.1.smithi192.stdout:5/839: chown d0/d13/d19/d71/d88/f107 1325983 1 2024-04-24T22:45:51.420 INFO:tasks.workunit.client.1.smithi192.stdout:6/946: link d0/c5 d0/d8/d3d/dc6/d127/d138/c156 0 2024-04-24T22:45:51.424 INFO:tasks.workunit.client.1.smithi192.stdout:5/840: unlink d0/d13/d19/d71/d88/f107 0 2024-04-24T22:45:51.426 INFO:tasks.workunit.client.1.smithi192.stdout:1/797: mkdir d9/dd/d106 0 2024-04-24T22:45:51.427 INFO:tasks.workunit.client.1.smithi192.stdout:0/991: dwrite d3/d27/d28/dcc/f6b [0,4194304] 0 2024-04-24T22:45:51.428 INFO:tasks.workunit.client.1.smithi192.stdout:1/798: stat d9/d15/d2d/d48/d93/dfc/dac/lc2 0 2024-04-24T22:45:51.431 INFO:tasks.workunit.client.1.smithi192.stdout:5/841: read - d0/d13/dd1/d108/f96 zero size 2024-04-24T22:45:51.434 INFO:tasks.workunit.client.1.smithi192.stdout:6/947: creat d0/d8/d18/f157 x:0 0 0 2024-04-24T22:45:51.435 INFO:tasks.workunit.client.1.smithi192.stdout:5/842: fsync d0/d1b/d62/d10f/d2c/f44 0 2024-04-24T22:45:51.437 INFO:tasks.workunit.client.1.smithi192.stdout:1/799: dwrite d9/d15/d5e/d7c/fe7 [0,4194304] 0 2024-04-24T22:45:51.439 INFO:tasks.workunit.client.1.smithi192.stdout:5/843: mkdir d0/d90/d11c 0 2024-04-24T22:45:51.440 INFO:tasks.workunit.client.1.smithi192.stdout:6/948: mkdir d0/d8/d22/dcb/d158 0 2024-04-24T22:45:51.441 INFO:tasks.workunit.client.1.smithi192.stdout:6/949: fdatasync d0/d8/d18/d143/f152 0 2024-04-24T22:45:51.444 INFO:tasks.workunit.client.1.smithi192.stdout:6/950: unlink d0/d8/d3d/d3e/d40/c8f 0 2024-04-24T22:45:51.446 INFO:tasks.workunit.client.1.smithi192.stdout:1/800: dread d9/d15/f47 [0,4194304] 0 2024-04-24T22:45:51.448 INFO:tasks.workunit.client.1.smithi192.stdout:6/951: creat d0/d8/d29/d5a/ddf/f159 x:0 0 0 2024-04-24T22:45:51.454 INFO:tasks.workunit.client.1.smithi192.stdout:0/992: dread d3/f10 [0,4194304] 0 2024-04-24T22:45:51.456 INFO:tasks.workunit.client.1.smithi192.stdout:6/952: rename d0/d8/d29/d4c/d4d/d83/lcd to d0/d8/d3d/d9d/da5/l15a 0 2024-04-24T22:45:51.457 INFO:tasks.workunit.client.1.smithi192.stdout:6/953: dread - d0/d8/d29/d5a/ddf/f159 zero size 2024-04-24T22:45:51.458 INFO:tasks.workunit.client.1.smithi192.stdout:0/993: truncate d3/d27/d28/dcc/f4d 539083 0 2024-04-24T22:45:51.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/954: rename d0/d8/d3d/d9d/da5 to d0/d8/d18/df1/df2/d15b 0 2024-04-24T22:45:51.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/801: dwrite f3 [0,4194304] 0 2024-04-24T22:45:51.480 INFO:tasks.workunit.client.1.smithi192.stdout:6/955: truncate d0/d8/d18/df1/df2/d15b/f108 917944 0 2024-04-24T22:45:51.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/956: write d0/d8/d18/df1/df2/f13c [1022718,66659] 0 2024-04-24T22:45:51.488 INFO:tasks.workunit.client.1.smithi192.stdout:6/957: creat d0/f15c x:0 0 0 2024-04-24T22:45:51.498 INFO:tasks.workunit.client.1.smithi192.stdout:6/958: truncate d0/d8/f6f 2729201 0 2024-04-24T22:45:51.502 INFO:tasks.workunit.client.1.smithi192.stdout:6/959: read d0/d8/f2b [997240,113767] 0 2024-04-24T22:45:51.513 INFO:tasks.workunit.client.1.smithi192.stdout:6/960: dwrite d0/d8/d29/d4c/d4d/d6a/ff9 [4194304,4194304] 0 2024-04-24T22:45:51.517 INFO:tasks.workunit.client.1.smithi192.stdout:6/961: truncate d0/d8/d29/d4c/fce 140584 0 2024-04-24T22:45:51.518 INFO:tasks.workunit.client.1.smithi192.stdout:6/962: dread - d0/d8/d3d/d3e/d7e/f120 zero size 2024-04-24T22:45:51.521 INFO:tasks.workunit.client.1.smithi192.stdout:6/963: creat d0/d8/d29/d4c/d4d/d6a/f15d x:0 0 0 2024-04-24T22:45:51.525 INFO:tasks.workunit.client.1.smithi192.stdout:6/964: creat d0/d8/d3d/d3e/d40/d44/f15e x:0 0 0 2024-04-24T22:45:51.528 INFO:tasks.workunit.client.1.smithi192.stdout:6/965: mknod d0/d8/d3d/dc6/d127/d138/c15f 0 2024-04-24T22:45:51.532 INFO:tasks.workunit.client.1.smithi192.stdout:6/966: unlink d0/d8/d18/df1/df2/d15b/cfa 0 2024-04-24T22:45:51.534 INFO:tasks.workunit.client.1.smithi192.stdout:6/967: fdatasync d0/d8/d29/d4c/d4d/f72 0 2024-04-24T22:45:51.548 INFO:tasks.workunit.client.1.smithi192.stdout:6/968: dread d0/d8/f91 [0,4194304] 0 2024-04-24T22:45:51.552 INFO:tasks.workunit.client.1.smithi192.stdout:6/969: rename d0/d8/d3d/d3e/d40/d155 to d0/d8/d3d/d3e/d40/d44/dc7/d160 0 2024-04-24T22:45:51.555 INFO:tasks.workunit.client.1.smithi192.stdout:6/970: unlink d0/d8/d18/c4f 0 2024-04-24T22:45:51.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/971: dwrite d0/d8/d3d/d3e/d40/d44/f15e [0,4194304] 0 2024-04-24T22:45:51.566 INFO:tasks.workunit.client.1.smithi192.stdout:6/972: mkdir d0/d8/d29/d5a/d63/d161 0 2024-04-24T22:45:51.568 INFO:tasks.workunit.client.1.smithi192.stdout:6/973: creat d0/d8/d18/df1/df2/d103/f162 x:0 0 0 2024-04-24T22:45:51.569 INFO:tasks.workunit.client.1.smithi192.stdout:6/974: write d0/d8/d3d/d3e/fac [332968,12377] 0 2024-04-24T22:45:51.573 INFO:tasks.workunit.client.1.smithi192.stdout:6/975: truncate d0/d8/fe 573944 0 2024-04-24T22:45:51.574 INFO:tasks.workunit.client.1.smithi192.stdout:6/976: dread - d0/d8/d3d/d3e/d7e/f125 zero size 2024-04-24T22:45:51.575 INFO:tasks.workunit.client.1.smithi192.stdout:6/977: fsync d0/d8/d29/d4c/d4d/f111 0 2024-04-24T22:45:51.576 INFO:tasks.workunit.client.1.smithi192.stdout:6/978: fsync d0/d8/d18/df1/df2/d15b/dc8/d110/f149 0 2024-04-24T22:45:51.583 INFO:tasks.workunit.client.1.smithi192.stdout:6/979: dwrite d0/d8/d3d/d3e/d7e/f125 [0,4194304] 0 2024-04-24T22:45:51.586 INFO:tasks.workunit.client.1.smithi192.stdout:6/980: fsync d0/fc2 0 2024-04-24T22:45:51.594 INFO:tasks.workunit.client.1.smithi192.stdout:6/981: rename d0/d8/d29/d4c/d4d/d83/dde/c11c to d0/d8/d3d/d3e/d40/d150/c163 0 2024-04-24T22:45:51.595 INFO:tasks.workunit.client.1.smithi192.stdout:6/982: stat d0/d8/d3d/d9d/ce2 0 2024-04-24T22:45:51.596 INFO:tasks.workunit.client.1.smithi192.stdout:6/983: read d0/d8/d3d/d3e/d40/d44/fe3 [360479,117649] 0 2024-04-24T22:45:51.597 INFO:tasks.workunit.client.1.smithi192.stdout:6/984: fdatasync d0/d8/d29/d5a/ddf/f159 0 2024-04-24T22:45:51.602 INFO:tasks.workunit.client.1.smithi192.stdout:6/985: creat d0/d8/d22/dcb/d158/f164 x:0 0 0 2024-04-24T22:45:51.604 INFO:tasks.workunit.client.1.smithi192.stdout:6/986: creat d0/d8/d22/dcb/de0/d10d/f165 x:0 0 0 2024-04-24T22:45:51.605 INFO:tasks.workunit.client.1.smithi192.stdout:6/987: fdatasync d0/d8/d29/d58/fbe 0 2024-04-24T22:45:51.614 INFO:tasks.workunit.client.1.smithi192.stdout:6/988: write d0/d8/d22/f145 [839489,15210] 0 2024-04-24T22:45:51.615 INFO:tasks.workunit.client.1.smithi192.stdout:4/926: sync 2024-04-24T22:45:51.617 INFO:tasks.workunit.client.1.smithi192.stdout:6/989: creat d0/d8/f166 x:0 0 0 2024-04-24T22:45:51.618 INFO:tasks.workunit.client.1.smithi192.stdout:4/927: rmdir d2/db/d75/da1/d36/d5b/d28/d13e 39 2024-04-24T22:45:51.619 INFO:tasks.workunit.client.1.smithi192.stdout:6/990: creat d0/d8/d29/d4c/f167 x:0 0 0 2024-04-24T22:45:51.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/928: creat d2/db/d75/da1/d36/d5b/d28/df6/f145 x:0 0 0 2024-04-24T22:45:51.622 INFO:tasks.workunit.client.1.smithi192.stdout:6/991: unlink d0/d8/f34 0 2024-04-24T22:45:51.625 INFO:tasks.workunit.client.1.smithi192.stdout:6/992: creat d0/d8/d3d/d3e/d40/d44/f168 x:0 0 0 2024-04-24T22:45:51.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/929: dwrite d2/d9/f8a [0,4194304] 0 2024-04-24T22:45:51.628 INFO:tasks.workunit.client.1.smithi192.stdout:4/930: readlink d2/db/d75/da1/dc9/df0/d135/l139 0 2024-04-24T22:45:51.633 INFO:tasks.workunit.client.1.smithi192.stdout:4/931: getdents d2/db/d75/da1/d36/d5b/d28/d9f/df2 0 2024-04-24T22:45:51.645 INFO:tasks.workunit.client.1.smithi192.stdout:6/993: write d0/d8/d29/f122 [1775628,22697] 0 2024-04-24T22:45:51.645 INFO:tasks.workunit.client.1.smithi192.stdout:4/932: write d2/fd6 [869183,76043] 0 2024-04-24T22:45:51.647 INFO:tasks.workunit.client.1.smithi192.stdout:6/994: write d0/d8/d18/d2e/fa0 [1064694,31767] 0 2024-04-24T22:45:51.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/933: chown d2/db/d75/da1/d7d/d9c/cab 2240220 1 2024-04-24T22:45:51.650 INFO:tasks.workunit.client.1.smithi192.stdout:6/995: creat d0/d8/d29/d4c/d4d/d83/f169 x:0 0 0 2024-04-24T22:45:51.654 INFO:tasks.workunit.client.1.smithi192.stdout:4/934: creat d2/db/d75/da1/d36/d5b/f146 x:0 0 0 2024-04-24T22:45:51.656 INFO:tasks.workunit.client.1.smithi192.stdout:6/996: dwrite d0/d8/d18/f157 [0,4194304] 0 2024-04-24T22:45:51.658 INFO:tasks.workunit.client.1.smithi192.stdout:6/997: write d0/d8/d22/f145 [1241444,11975] 0 2024-04-24T22:45:51.664 INFO:tasks.workunit.client.1.smithi192.stdout:4/935: write d2/db/d10/f4f [5108991,6171] 0 2024-04-24T22:45:51.665 INFO:tasks.workunit.client.1.smithi192.stdout:6/998: truncate d0/d8/f2b 6723935 0 2024-04-24T22:45:51.666 INFO:tasks.workunit.client.1.smithi192.stdout:4/936: write d2/db/d5d/f115 [162717,106232] 0 2024-04-24T22:45:51.667 INFO:tasks.workunit.client.1.smithi192.stdout:4/937: read - d2/db/d75/da1/d36/d5b/d28/df6/f145 zero size 2024-04-24T22:45:51.670 INFO:tasks.workunit.client.1.smithi192.stdout:6/999: rename d0/d8/d3d/d3e/d40/d44/l147 to d0/d8/d3d/d3e/de1/l16a 0 2024-04-24T22:45:51.682 INFO:tasks.workunit.client.1.smithi192.stdout:4/938: dwrite d2/db/d75/da1/d53/fcc [0,4194304] 0 2024-04-24T22:45:51.686 INFO:tasks.workunit.client.1.smithi192.stdout:4/939: unlink d2/d5c/d107/f11f 0 2024-04-24T22:45:51.688 INFO:tasks.workunit.client.1.smithi192.stdout:4/940: creat d2/db/d75/da1/d7d/d9c/f147 x:0 0 0 2024-04-24T22:45:51.691 INFO:tasks.workunit.client.1.smithi192.stdout:4/941: rename d2/db/d75/cb8 to d2/c148 0 2024-04-24T22:45:51.700 INFO:tasks.workunit.client.1.smithi192.stdout:4/942: truncate d2/db/d5d/d77/f92 453301 0 2024-04-24T22:45:51.706 INFO:tasks.workunit.client.1.smithi192.stdout:4/943: dwrite d2/db/d75/da1/d36/d5b/f106 [0,4194304] 0 2024-04-24T22:45:51.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/944: creat d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/d144/f149 x:0 0 0 2024-04-24T22:45:51.715 INFO:tasks.workunit.client.1.smithi192.stdout:4/945: unlink d2/db/d5d/d77/d82/fa3 0 2024-04-24T22:45:51.721 INFO:tasks.workunit.client.1.smithi192.stdout:4/946: creat d2/d9/d25/d2e/f14a x:0 0 0 2024-04-24T22:45:51.722 INFO:tasks.workunit.client.1.smithi192.stdout:4/947: stat d2/db/d75/da1/dc9/df0/d135 0 2024-04-24T22:45:51.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/948: rename d2/db/d75/da1/d36/c11b to d2/db/d75/da1/ddf/c14b 0 2024-04-24T22:45:51.731 INFO:tasks.workunit.client.1.smithi192.stdout:4/949: rename d2/d9/d6b/l6d to d2/db/d75/da1/d36/d5b/l14c 0 2024-04-24T22:45:51.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/950: symlink d2/db/d75/da1/d36/d5b/d28/d2f/l14d 0 2024-04-24T22:45:51.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/951: readlink d2/db/d75/da1/d36/d5b/d28/l56 0 2024-04-24T22:45:51.740 INFO:tasks.workunit.client.1.smithi192.stdout:4/952: write d2/db/d75/da1/d53/fc5 [117761,32983] 0 2024-04-24T22:45:51.742 INFO:tasks.workunit.client.1.smithi192.stdout:4/953: creat d2/db/d75/da1/d36/dfe/f14e x:0 0 0 2024-04-24T22:45:51.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/954: getdents d2 0 2024-04-24T22:45:51.752 INFO:tasks.workunit.client.1.smithi192.stdout:4/955: dread d2/d5c/dbb/f13b [0,4194304] 0 2024-04-24T22:45:51.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/956: dread - d2/d9/d25/f71 zero size 2024-04-24T22:45:51.754 INFO:tasks.workunit.client.1.smithi192.stdout:4/957: fdatasync d2/db/d10/f123 0 2024-04-24T22:45:51.757 INFO:tasks.workunit.client.1.smithi192.stdout:4/958: mkdir d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/d144/d14f 0 2024-04-24T22:45:51.758 INFO:tasks.workunit.client.1.smithi192.stdout:4/959: creat d2/d9/d6b/f150 x:0 0 0 2024-04-24T22:45:51.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/960: dread d2/db/d75/da1/d36/d5b/d28/d9f/fa0 [0,4194304] 0 2024-04-24T22:45:51.765 INFO:tasks.workunit.client.1.smithi192.stdout:4/961: mkdir d2/db/d151 0 2024-04-24T22:45:51.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/962: rename d2/db/d75/da1/d36/d5b/d28/df6 to d2/db/d152 0 2024-04-24T22:45:51.773 INFO:tasks.workunit.client.1.smithi192.stdout:4/963: rename d2/db/d152/d110 to d2/db/d75/da1/d36/d5b/d153 0 2024-04-24T22:45:51.775 INFO:tasks.workunit.client.1.smithi192.stdout:4/964: symlink d2/d5c/dbb/l154 0 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: pgmap v23: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 82 MiB/s rd, 188 MiB/s wr, 433 op/s 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.778 INFO:tasks.workunit.client.1.smithi192.stdout:4/965: dread d2/db/d75/da1/d36/d5b/d28/d9f/fa0 [0,4194304] 0 2024-04-24T22:45:51.783 INFO:tasks.workunit.client.1.smithi192.stdout:4/966: dwrite d2/db/d75/da1/d36/d5b/f37 [0,4194304] 0 2024-04-24T22:45:51.786 INFO:tasks.workunit.client.1.smithi192.stdout:5/844: sync 2024-04-24T22:45:51.786 INFO:tasks.workunit.client.1.smithi192.stdout:4/967: read - d2/db/d75/da1/d36/d5b/d28/d9f/f10b zero size 2024-04-24T22:45:51.790 INFO:tasks.workunit.client.1.smithi192.stdout:4/968: rmdir d2/db/d75/da1/dc9/df0/d135 39 2024-04-24T22:45:51.793 INFO:tasks.workunit.client.1.smithi192.stdout:5/845: write d0/d13/d19/d64/fa8 [50328,36031] 0 2024-04-24T22:45:51.794 INFO:tasks.workunit.client.1.smithi192.stdout:5/846: readlink d0/d1b/d50/l6b 0 2024-04-24T22:45:51.797 INFO:tasks.workunit.client.1.smithi192.stdout:4/969: dwrite d2/d9/d6b/f150 [0,4194304] 0 2024-04-24T22:45:51.798 INFO:tasks.workunit.client.1.smithi192.stdout:5/847: symlink d0/d1b/d62/dc1/dc2/l11d 0 2024-04-24T22:45:51.802 INFO:tasks.workunit.client.1.smithi192.stdout:4/970: creat d2/f155 x:0 0 0 2024-04-24T22:45:51.803 INFO:tasks.workunit.client.1.smithi192.stdout:5/848: truncate d0/d1b/d62/fa7 702311 0 2024-04-24T22:45:51.804 INFO:tasks.workunit.client.1.smithi192.stdout:5/849: write d0/d13/d19/d64/fa8 [1049538,67797] 0 2024-04-24T22:45:51.805 INFO:tasks.workunit.client.1.smithi192.stdout:5/850: write d0/d13/f117 [1030897,118881] 0 2024-04-24T22:45:51.810 INFO:tasks.workunit.client.1.smithi192.stdout:5/851: mkdir d0/d13/d19/d71/d11e 0 2024-04-24T22:45:51.812 INFO:tasks.workunit.client.1.smithi192.stdout:5/852: creat d0/d90/f11f x:0 0 0 2024-04-24T22:45:51.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/971: rename d2/da5/cad to d2/db/d75/da1/d36/d5b/c156 0 2024-04-24T22:45:51.823 INFO:tasks.workunit.client.1.smithi192.stdout:4/972: symlink d2/db/d75/da1/d36/d5b/d28/d9f/l157 0 2024-04-24T22:45:51.823 INFO:tasks.workunit.client.1.smithi192.stdout:4/973: fsync d2/db/d75/da1/d36/d5b/f146 0 2024-04-24T22:45:51.824 INFO:tasks.workunit.client.1.smithi192.stdout:4/974: chown d2/db/d75/da1/d53/fc5 3 1 2024-04-24T22:45:51.827 INFO:tasks.workunit.client.1.smithi192.stdout:4/975: read d2/db/d75/da1/f4c [3601507,2584] 0 2024-04-24T22:45:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: pgmap v23: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 82 MiB/s rd, 188 MiB/s wr, 433 op/s 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:51.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/994: sync 2024-04-24T22:45:51.992 INFO:tasks.workunit.client.1.smithi192.stdout:1/802: sync 2024-04-24T22:45:52.010 INFO:tasks.workunit.client.1.smithi192.stdout:0/995: fdatasync d3/d6d/f108 0 2024-04-24T22:45:52.012 INFO:tasks.workunit.client.1.smithi192.stdout:0/996: fsync d3/f48 0 2024-04-24T22:45:52.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/997: write d3/d40/f9b [2251928,19601] 0 2024-04-24T22:45:52.018 INFO:tasks.workunit.client.1.smithi192.stdout:1/803: getdents d9/dd/d74/d83 0 2024-04-24T22:45:52.019 INFO:tasks.workunit.client.1.smithi192.stdout:0/998: creat d3/d27/d28/dcc/df3/f147 x:0 0 0 2024-04-24T22:45:52.021 INFO:tasks.workunit.client.1.smithi192.stdout:1/804: symlink d9/d15/d2d/d63/l107 0 2024-04-24T22:45:52.023 INFO:tasks.workunit.client.1.smithi192.stdout:0/999: mknod d3/d27/d3c/d6c/c148 0 2024-04-24T22:45:52.024 INFO:tasks.workunit.client.1.smithi192.stdout:1/805: mkdir d9/d15/d2d/d63/d68/d9d/d108 0 2024-04-24T22:45:52.025 INFO:tasks.workunit.client.1.smithi192.stdout:1/806: mknod d9/c109 0 2024-04-24T22:45:52.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/807: mkdir d9/d15/d2d/d48/d93/dc5/dd6/d10a 0 2024-04-24T22:45:52.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/808: dwrite d9/d15/d2d/d63/d68/d9d/ff5 [0,4194304] 0 2024-04-24T22:45:52.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/809: dwrite d9/d15/d2d/d4a/fce [0,4194304] 0 2024-04-24T22:45:52.066 INFO:tasks.workunit.client.1.smithi192.stdout:1/810: dread d9/d15/d2d/d4a/fce [0,4194304] 0 2024-04-24T22:45:52.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/811: creat d9/d51/f10b x:0 0 0 2024-04-24T22:45:52.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/812: chown d9/d15/d2d/d31/d2f/fc0 1225723 1 2024-04-24T22:45:52.073 INFO:tasks.workunit.client.1.smithi192.stdout:1/813: creat d9/d15/d2d/d63/d68/d9d/d108/f10c x:0 0 0 2024-04-24T22:45:52.075 INFO:tasks.workunit.client.1.smithi192.stdout:1/814: creat d9/d15/d5e/d90/da9/ddd/f10d x:0 0 0 2024-04-24T22:45:52.080 INFO:tasks.workunit.client.1.smithi192.stdout:1/815: dwrite d9/d15/d2d/d63/d68/f76 [4194304,4194304] 0 2024-04-24T22:45:52.082 INFO:tasks.workunit.client.1.smithi192.stdout:1/816: write d9/dd/d74/d83/fb7 [6021528,26513] 0 2024-04-24T22:45:52.082 INFO:tasks.workunit.client.1.smithi192.stdout:1/817: chown d9/dd/d74/dcd 0 1 2024-04-24T22:45:52.086 INFO:tasks.workunit.client.1.smithi192.stdout:1/818: symlink d9/db8/l10e 0 2024-04-24T22:45:52.087 INFO:tasks.workunit.client.1.smithi192.stdout:1/819: write d9/d15/d5e/d90/da9/fc1 [1195889,80759] 0 2024-04-24T22:45:52.094 INFO:tasks.workunit.client.1.smithi192.stdout:1/820: dwrite d9/d15/d5e/d90/da9/ddd/f10d [0,4194304] 0 2024-04-24T22:45:52.103 INFO:tasks.workunit.client.1.smithi192.stdout:1/821: write d9/d15/d2d/d31/f62 [402809,9598] 0 2024-04-24T22:45:52.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/853: sync 2024-04-24T22:45:52.105 INFO:tasks.workunit.client.1.smithi192.stdout:5/854: mknod d0/d13/d19/da9/d10b/c120 0 2024-04-24T22:45:52.108 INFO:tasks.workunit.client.1.smithi192.stdout:5/855: chown d0/d13/d19/d83/f87 17630098 1 2024-04-24T22:45:52.110 INFO:tasks.workunit.client.1.smithi192.stdout:1/822: dwrite d9/d15/d2d/d63/d68/d9d/fee [0,4194304] 0 2024-04-24T22:45:52.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/856: truncate d0/f23 1427271 0 2024-04-24T22:45:52.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/857: dread d0/d13/d19/d64/d4f/d7c/ff2 [0,4194304] 0 2024-04-24T22:45:52.123 INFO:tasks.workunit.client.1.smithi192.stdout:1/823: write d9/dd/d74/f7a [742384,116603] 0 2024-04-24T22:45:52.130 INFO:tasks.workunit.client.1.smithi192.stdout:5/858: getdents d0/d13/d19/d71/dcf 0 2024-04-24T22:45:52.137 INFO:tasks.workunit.client.1.smithi192.stdout:1/824: dwrite d9/d15/d2d/d31/d2f/fc0 [0,4194304] 0 2024-04-24T22:45:52.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/825: mknod d9/d51/c10f 0 2024-04-24T22:45:52.143 INFO:tasks.workunit.client.1.smithi192.stdout:5/859: dwrite d0/d1b/d62/d10f/d2c/fa3 [0,4194304] 0 2024-04-24T22:45:52.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/860: write d0/d1b/d50/f7d [5240362,50510] 0 2024-04-24T22:45:52.154 INFO:tasks.workunit.client.1.smithi192.stdout:1/826: link d9/dd/d74/d83/fb7 d9/d15/d2d/d63/d68/f110 0 2024-04-24T22:45:52.155 INFO:tasks.workunit.client.1.smithi192.stdout:1/827: read d9/d15/d2d/d63/d68/d9d/fee [3870438,99901] 0 2024-04-24T22:45:52.157 INFO:tasks.workunit.client.1.smithi192.stdout:1/828: rmdir d9/d15/d2d/d48 39 2024-04-24T22:45:52.161 INFO:tasks.workunit.client.1.smithi192.stdout:1/829: symlink d9/d15/d2d/d63/d68/l111 0 2024-04-24T22:45:52.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/861: dwrite d0/d13/d3b/d79/fe0 [0,4194304] 0 2024-04-24T22:45:52.163 INFO:tasks.workunit.client.1.smithi192.stdout:5/862: chown d0/d13/f3f 27 1 2024-04-24T22:45:52.166 INFO:tasks.workunit.client.1.smithi192.stdout:4/976: sync 2024-04-24T22:45:52.168 INFO:tasks.workunit.client.1.smithi192.stdout:1/830: dread d9/d15/d5e/d90/da9/ddd/f10d [0,4194304] 0 2024-04-24T22:45:52.174 INFO:tasks.workunit.client.1.smithi192.stdout:4/977: creat d2/db/d75/da1/d36/d5b/d28/deb/d12d/d136/f158 x:0 0 0 2024-04-24T22:45:52.177 INFO:tasks.workunit.client.1.smithi192.stdout:4/978: dread - d2/db/d5d/d77/d82/d8d/ffa zero size 2024-04-24T22:45:52.181 INFO:tasks.workunit.client.1.smithi192.stdout:1/831: creat d9/d51/f112 x:0 0 0 2024-04-24T22:45:52.186 INFO:tasks.workunit.client.1.smithi192.stdout:4/979: unlink d2/db/d75/da1/d36/c3a 0 2024-04-24T22:45:52.189 INFO:tasks.workunit.client.1.smithi192.stdout:1/832: dwrite d9/d15/d5e/feb [0,4194304] 0 2024-04-24T22:45:52.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/863: dread d0/d1b/d62/d10f/f20 [0,4194304] 0 2024-04-24T22:45:52.200 INFO:tasks.workunit.client.1.smithi192.stdout:4/980: mkdir d2/db/d5d/d77/d82/d13d/d159 0 2024-04-24T22:45:52.203 INFO:tasks.workunit.client.1.smithi192.stdout:4/981: mkdir d2/db/d151/d15a 0 2024-04-24T22:45:52.203 INFO:tasks.workunit.client.1.smithi192.stdout:4/982: stat d2/db/d75/da1/d7d/f10f 0 2024-04-24T22:45:52.205 INFO:tasks.workunit.client.1.smithi192.stdout:4/983: fdatasync d2/db/d75/da1/d36/d5b/d28/deb/f125 0 2024-04-24T22:45:52.211 INFO:tasks.workunit.client.1.smithi192.stdout:1/833: write d9/f14 [4977290,53458] 0 2024-04-24T22:45:52.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/834: write d9/d15/d2d/d48/d93/da4/dd2/fbd [1886819,7640] 0 2024-04-24T22:45:52.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/984: dwrite d2/d9/fdb [0,4194304] 0 2024-04-24T22:45:52.220 INFO:tasks.workunit.client.1.smithi192.stdout:5/864: dwrite d0/d13/fde [0,4194304] 0 2024-04-24T22:45:52.221 INFO:tasks.workunit.client.1.smithi192.stdout:1/835: stat d9/d15/f2e 0 2024-04-24T22:45:52.222 INFO:tasks.workunit.client.1.smithi192.stdout:5/865: chown d0/d13/d19/d83/dad/ld0 6408516 1 2024-04-24T22:45:52.224 INFO:tasks.workunit.client.1.smithi192.stdout:1/836: fdatasync d9/d15/d5e/d90/da9/ddd/fe0 0 2024-04-24T22:45:52.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/866: fsync d0/d1b/f69 0 2024-04-24T22:45:52.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/837: unlink d9/d15/d2d/l5f 0 2024-04-24T22:45:52.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/838: mkdir d9/d15/d2d/d48/d93/da4/dd2/dbf/d113 0 2024-04-24T22:45:52.245 INFO:tasks.workunit.client.1.smithi192.stdout:1/839: link d9/dd/d74/caa d9/dd/d74/d83/c114 0 2024-04-24T22:45:52.246 INFO:tasks.workunit.client.1.smithi192.stdout:4/985: write d2/db/d75/da1/d36/d5b/d28/d2f/f34 [2075207,10511] 0 2024-04-24T22:45:52.248 INFO:tasks.workunit.client.1.smithi192.stdout:1/840: mkdir d9/d15/d2d/d48/d93/dfc/dac/d115 0 2024-04-24T22:45:52.249 INFO:tasks.workunit.client.1.smithi192.stdout:1/841: chown d9/d15/d2d/ccb 21170 1 2024-04-24T22:45:52.250 INFO:tasks.workunit.client.1.smithi192.stdout:1/842: chown d9/d15/d2d/d63/fb2 3 1 2024-04-24T22:45:52.251 INFO:tasks.workunit.client.1.smithi192.stdout:4/986: creat d2/db/d75/da1/d36/d5b/d28/d2f/d7e/f15b x:0 0 0 2024-04-24T22:45:52.256 INFO:tasks.workunit.client.1.smithi192.stdout:1/843: creat d9/d51/f116 x:0 0 0 2024-04-24T22:45:52.260 INFO:tasks.workunit.client.1.smithi192.stdout:4/987: dwrite d2/db/d75/da1/d36/d5b/d28/d13e/fd8 [0,4194304] 0 2024-04-24T22:45:52.260 INFO:tasks.workunit.client.1.smithi192.stdout:5/867: dread d0/d13/d19/da9/fab [0,4194304] 0 2024-04-24T22:45:52.261 INFO:tasks.workunit.client.1.smithi192.stdout:4/988: stat d2/d9/d25/c93 0 2024-04-24T22:45:52.262 INFO:tasks.workunit.client.1.smithi192.stdout:5/868: chown d0/d1b/d62/d10f/d2c/ldd 5870 1 2024-04-24T22:45:52.263 INFO:tasks.workunit.client.1.smithi192.stdout:1/844: read d9/dd/d74/dcd/fcf [3140067,84357] 0 2024-04-24T22:45:52.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/869: mknod d0/d13/d19/d71/d88/c121 0 2024-04-24T22:45:52.267 INFO:tasks.workunit.client.1.smithi192.stdout:1/845: mkdir d9/d15/d5e/d117 0 2024-04-24T22:45:52.270 INFO:tasks.workunit.client.1.smithi192.stdout:4/989: dwrite d2/db/d75/da1/d36/d5b/f106 [0,4194304] 0 2024-04-24T22:45:52.270 INFO:tasks.workunit.client.1.smithi192.stdout:1/846: chown d9/d15/d2d/d48/d93/lc7 33494573 1 2024-04-24T22:45:52.272 INFO:tasks.workunit.client.1.smithi192.stdout:5/870: write d0/d1b/d62/dc1/dc2/f114 [879432,19270] 0 2024-04-24T22:45:52.273 INFO:tasks.workunit.client.1.smithi192.stdout:1/847: write d9/d15/d2d/d63/f79 [2222218,108434] 0 2024-04-24T22:45:52.278 INFO:tasks.workunit.client.1.smithi192.stdout:5/871: creat d0/d1b/d50/f122 x:0 0 0 2024-04-24T22:45:52.280 INFO:tasks.workunit.client.1.smithi192.stdout:5/872: chown d0/d13/d19/d83/dad/fd8 213701709 1 2024-04-24T22:45:52.281 INFO:tasks.workunit.client.1.smithi192.stdout:5/873: stat d0/d1b/d62/d10f/d2c/d3c/fc3 0 2024-04-24T22:45:52.282 INFO:tasks.workunit.client.1.smithi192.stdout:4/990: symlink d2/db/d75/da1/d36/d5b/d28/d2f/d7e/l15c 0 2024-04-24T22:45:52.284 INFO:tasks.workunit.client.1.smithi192.stdout:1/848: creat d9/d15/d2d/d48/d93/db4/f118 x:0 0 0 2024-04-24T22:45:52.289 INFO:tasks.workunit.client.1.smithi192.stdout:1/849: unlink d9/d15/d2d/c53 0 2024-04-24T22:45:52.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/850: dread - d9/d51/f112 zero size 2024-04-24T22:45:52.299 INFO:tasks.workunit.client.1.smithi192.stdout:1/851: rename d9/d15/d2d/d4a/l58 to d9/d15/d2d/d63/d68/l119 0 2024-04-24T22:45:52.300 INFO:tasks.workunit.client.1.smithi192.stdout:5/874: link d0/d13/d19/d71/d88/c121 d0/d13/d19/d64/d4f/d7c/c123 0 2024-04-24T22:45:52.301 INFO:tasks.workunit.client.1.smithi192.stdout:1/852: fdatasync d9/d15/d2d/d63/d68/f76 0 2024-04-24T22:45:52.305 INFO:tasks.workunit.client.1.smithi192.stdout:5/875: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:52.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/853: unlink d9/dd/d74/d83/ca1 0 2024-04-24T22:45:52.311 INFO:tasks.workunit.client.1.smithi192.stdout:1/854: chown d9/d51/l66 7 1 2024-04-24T22:45:52.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/855: stat d9/d51/d70 0 2024-04-24T22:45:52.314 INFO:tasks.workunit.client.1.smithi192.stdout:5/876: creat d0/d13/dd1/df1/f124 x:0 0 0 2024-04-24T22:45:52.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/856: creat d9/d15/d2d/d63/d68/dd5/f11a x:0 0 0 2024-04-24T22:45:52.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/877: dread - d0/d1b/d62/d10f/d2c/fb0 zero size 2024-04-24T22:45:52.317 INFO:tasks.workunit.client.1.smithi192.stdout:4/991: dread d2/da5/fcb [0,4194304] 0 2024-04-24T22:45:52.319 INFO:tasks.workunit.client.1.smithi192.stdout:1/857: chown d9/d15/d5e/d90/da9/ddd/ce2 37365613 1 2024-04-24T22:45:52.320 INFO:tasks.workunit.client.1.smithi192.stdout:1/858: chown d9/d15/l7d 206 1 2024-04-24T22:45:52.321 INFO:tasks.workunit.client.1.smithi192.stdout:5/878: mkdir d0/d1b/d62/d10f/d66/d125 0 2024-04-24T22:45:52.323 INFO:tasks.workunit.client.1.smithi192.stdout:1/859: mkdir d9/dd/d74/dcd/d11b 0 2024-04-24T22:45:52.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/879: creat d0/d13/d19/d64/d4f/d52/f126 x:0 0 0 2024-04-24T22:45:52.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/880: dread - d0/d13/d19/d71/d7f/d9a/fea zero size 2024-04-24T22:45:52.330 INFO:tasks.workunit.client.1.smithi192.stdout:5/881: creat d0/d1b/d62/d10f/d2c/d3c/d91/f127 x:0 0 0 2024-04-24T22:45:52.331 INFO:tasks.workunit.client.1.smithi192.stdout:5/882: readlink d0/d13/d19/da9/le3 0 2024-04-24T22:45:52.334 INFO:tasks.workunit.client.1.smithi192.stdout:5/883: truncate d0/d1b/d62/d10f/f72 1816042 0 2024-04-24T22:45:52.334 INFO:tasks.workunit.client.1.smithi192.stdout:4/992: dread d2/db/d10/f4f [0,4194304] 0 2024-04-24T22:45:52.339 INFO:tasks.workunit.client.1.smithi192.stdout:5/884: mknod d0/d1b/d62/d10f/d2c/d3c/d91/c128 0 2024-04-24T22:45:52.340 INFO:tasks.workunit.client.1.smithi192.stdout:4/993: truncate d2/d5c/fbf 1578625 0 2024-04-24T22:45:52.348 INFO:tasks.workunit.client.1.smithi192.stdout:4/994: dwrite d2/db/d75/da1/d36/d5b/d28/d2f/d7e/d120/d144/f149 [0,4194304] 0 2024-04-24T22:45:52.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/860: rmdir d9/dd/d74/dcd 39 2024-04-24T22:45:52.353 INFO:tasks.workunit.client.1.smithi192.stdout:1/861: fdatasync d9/d15/d5e/d90/da9/fc1 0 2024-04-24T22:45:52.354 INFO:tasks.workunit.client.1.smithi192.stdout:1/862: chown d9/d15/f1b 124 1 2024-04-24T22:45:52.355 INFO:tasks.workunit.client.1.smithi192.stdout:4/995: mknod d2/c15d 0 2024-04-24T22:45:52.359 INFO:tasks.workunit.client.1.smithi192.stdout:1/863: symlink d9/d6f/l11c 0 2024-04-24T22:45:52.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/885: write d0/d1b/d62/d10f/d2c/d3c/d91/d112/f7a [259787,46903] 0 2024-04-24T22:45:52.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/996: write d2/db/d75/da1/d36/d5b/f146 [583101,130224] 0 2024-04-24T22:45:52.367 INFO:tasks.workunit.client.1.smithi192.stdout:5/886: write d0/d1b/d62/d10f/d2c/d3c/d91/d112/f7a [1702029,112885] 0 2024-04-24T22:45:52.367 INFO:tasks.workunit.client.1.smithi192.stdout:1/864: rename d9/d15/d5e/l72 to d9/d15/d2d/d48/d93/db4/l11d 0 2024-04-24T22:45:52.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/887: rename d0/d1b/d50/c78 to d0/d13/d19/d64/c129 0 2024-04-24T22:45:52.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/888: readlink d0/d13/d53/l85 0 2024-04-24T22:45:52.377 INFO:tasks.workunit.client.1.smithi192.stdout:5/889: read d0/d1b/d62/d10f/f27 [128316,23974] 0 2024-04-24T22:45:52.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/997: read d2/d9/fc8 [127728,75717] 0 2024-04-24T22:45:52.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/865: write d9/d15/d2d/d31/d2f/f3a [5616145,24971] 0 2024-04-24T22:45:52.389 INFO:tasks.workunit.client.1.smithi192.stdout:1/866: rmdir d9/d15/d2d/d31 39 2024-04-24T22:45:52.390 INFO:tasks.workunit.client.1.smithi192.stdout:5/890: creat d0/d10c/f12a x:0 0 0 2024-04-24T22:45:52.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/867: mkdir d9/d15/d2d/d48/d93/dfc/dac/d11e 0 2024-04-24T22:45:52.397 INFO:tasks.workunit.client.1.smithi192.stdout:1/868: rmdir d9/d15/d5e/d7c 39 2024-04-24T22:45:52.399 INFO:tasks.workunit.client.1.smithi192.stdout:1/869: read d9/dd/d74/d83/fb7 [2483117,6524] 0 2024-04-24T22:45:52.402 INFO:tasks.workunit.client.1.smithi192.stdout:5/891: rmdir d0/d1b/d50/d116 0 2024-04-24T22:45:52.404 INFO:tasks.workunit.client.1.smithi192.stdout:1/870: rmdir d9/d15/d2d/d48/d93/dfc 39 2024-04-24T22:45:52.404 INFO:tasks.workunit.client.1.smithi192.stdout:4/998: dwrite d2/fd0 [0,4194304] 0 2024-04-24T22:45:52.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/871: fdatasync d9/d15/d2d/d63/f79 0 2024-04-24T22:45:52.406 INFO:tasks.workunit.client.1.smithi192.stdout:5/892: mknod d0/d13/d19/d64/c12b 0 2024-04-24T22:45:52.410 INFO:tasks.workunit.client.1.smithi192.stdout:5/893: creat d0/d13/d19/d64/d4f/d6c/f12c x:0 0 0 2024-04-24T22:45:52.411 INFO:tasks.workunit.client.1.smithi192.stdout:4/999: mknod d2/da5/c15e 0 2024-04-24T22:45:52.415 INFO:tasks.workunit.client.1.smithi192.stdout:5/894: unlink d0/d1b/d62/d10f/f20 0 2024-04-24T22:45:52.417 INFO:tasks.workunit.client.1.smithi192.stdout:1/872: creat d9/d15/d5e/d90/f11f x:0 0 0 2024-04-24T22:45:52.418 INFO:tasks.workunit.client.1.smithi192.stdout:5/895: unlink d0/d13/dd1/d108/f46 0 2024-04-24T22:45:52.421 INFO:tasks.workunit.client.1.smithi192.stdout:1/873: truncate d9/d6f/f9b 1150279 0 2024-04-24T22:45:52.423 INFO:tasks.workunit.client.1.smithi192.stdout:5/896: fdatasync d0/f4a 0 2024-04-24T22:45:52.424 INFO:tasks.workunit.client.1.smithi192.stdout:5/897: readlink d0/d13/d53/l85 0 2024-04-24T22:45:52.426 INFO:tasks.workunit.client.1.smithi192.stdout:1/874: rename d9/d15/d2d/d63/d68/dd5/f11a to d9/d15/d2d/d48/d93/db4/f120 0 2024-04-24T22:45:52.428 INFO:tasks.workunit.client.1.smithi192.stdout:5/898: creat d0/d13/d19/d71/dbf/f12d x:0 0 0 2024-04-24T22:45:52.430 INFO:tasks.workunit.client.1.smithi192.stdout:1/875: rename d9/d15/d5e/d67/f71 to d9/d15/d2d/d48/d93/da4/dd2/dbf/de9/f121 0 2024-04-24T22:45:52.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/876: stat d9/lf9 0 2024-04-24T22:45:52.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/877: chown d9/f2c 168745 1 2024-04-24T22:45:52.437 INFO:tasks.workunit.client.1.smithi192.stdout:1/878: rename d9/f2c to d9/d15/d2d/d48/d93/dc5/dd6/d10a/f122 0 2024-04-24T22:45:52.449 INFO:tasks.workunit.client.1.smithi192.stdout:1/879: symlink d9/dd/d106/l123 0 2024-04-24T22:45:52.450 INFO:tasks.workunit.client.1.smithi192.stdout:5/899: rename d0/d13/d3b/c111 to d0/d13/d19/c12e 0 2024-04-24T22:45:52.452 INFO:tasks.workunit.client.1.smithi192.stdout:5/900: chown d0/d13/d19/d83/dad/ld0 5043950 1 2024-04-24T22:45:52.455 INFO:tasks.workunit.client.1.smithi192.stdout:1/880: rename d9/db8/l10e to d9/d15/d2d/d48/d93/dc5/l124 0 2024-04-24T22:45:52.455 INFO:tasks.workunit.client.1.smithi192.stdout:5/901: dread d0/d13/d3b/f55 [0,4194304] 0 2024-04-24T22:45:52.462 INFO:tasks.workunit.client.1.smithi192.stdout:1/881: dwrite d9/d15/d2d/d48/d93/da4/fc9 [0,4194304] 0 2024-04-24T22:45:52.465 INFO:tasks.workunit.client.1.smithi192.stdout:5/902: write d0/d13/d3b/f3a [1266545,120630] 0 2024-04-24T22:45:52.469 INFO:tasks.workunit.client.1.smithi192.stdout:1/882: dwrite d9/d15/f1b [0,4194304] 0 2024-04-24T22:45:52.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/883: symlink d9/d15/d5e/d117/l125 0 2024-04-24T22:45:52.473 INFO:tasks.workunit.client.1.smithi192.stdout:1/884: fsync d9/d15/d2d/d63/d68/f110 0 2024-04-24T22:45:52.475 INFO:tasks.workunit.client.1.smithi192.stdout:1/885: truncate d9/d15/d5e/d90/da9/ddd/fe0 587635 0 2024-04-24T22:45:52.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/886: chown d9/d15/d2d/d63 725605 1 2024-04-24T22:45:52.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/887: readlink d9/dd/l5a 0 2024-04-24T22:45:52.483 INFO:tasks.workunit.client.1.smithi192.stdout:1/888: write d9/d15/d2d/d48/f8d [779827,127033] 0 2024-04-24T22:45:52.484 INFO:tasks.workunit.client.1.smithi192.stdout:1/889: dread - d9/d51/f116 zero size 2024-04-24T22:45:52.486 INFO:tasks.workunit.client.1.smithi192.stdout:1/890: write d9/d15/d2d/d63/d68/d9d/d108/f10c [929954,119715] 0 2024-04-24T22:45:52.504 INFO:tasks.workunit.client.1.smithi192.stdout:1/891: dwrite d9/f3c [0,4194304] 0 2024-04-24T22:45:52.505 INFO:tasks.workunit.client.1.smithi192.stdout:5/903: dread d0/d1b/d62/d10f/d2c/d3c/f5a [0,4194304] 0 2024-04-24T22:45:52.505 INFO:tasks.workunit.client.1.smithi192.stdout:5/904: dread - d0/d1b/d50/f122 zero size 2024-04-24T22:45:52.510 INFO:tasks.workunit.client.1.smithi192.stdout:1/892: symlink d9/dd/d74/dcd/d11b/l126 0 2024-04-24T22:45:52.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/905: dwrite d0/d13/d19/da9/fab [0,4194304] 0 2024-04-24T22:45:52.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/893: getdents d9/d15/d2d/d4a 0 2024-04-24T22:45:52.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/894: stat d9/dd/c1e 0 2024-04-24T22:45:52.520 INFO:tasks.workunit.client.1.smithi192.stdout:1/895: chown d9/f1a 1214 1 2024-04-24T22:45:52.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/906: rmdir d0/d1b/d62 39 2024-04-24T22:45:52.522 INFO:tasks.workunit.client.1.smithi192.stdout:1/896: write d9/d15/f49 [4590210,41878] 0 2024-04-24T22:45:52.524 INFO:tasks.workunit.client.1.smithi192.stdout:5/907: mknod d0/d13/d19/d83/c12f 0 2024-04-24T22:45:52.525 INFO:tasks.workunit.client.1.smithi192.stdout:5/908: write d0/d13/d19/d83/dad/fd8 [898950,43706] 0 2024-04-24T22:45:52.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/909: truncate d0/d13/d19/d64/d4f/d6c/fb5 3247316 0 2024-04-24T22:45:52.530 INFO:tasks.workunit.client.1.smithi192.stdout:1/897: dread d9/d6f/f86 [0,4194304] 0 2024-04-24T22:45:52.532 INFO:tasks.workunit.client.1.smithi192.stdout:5/910: fsync d0/d13/d19/d71/ffd 0 2024-04-24T22:45:52.540 INFO:tasks.workunit.client.1.smithi192.stdout:5/911: rename d0/d1b/f69 to d0/d13/d3b/dd7/f130 0 2024-04-24T22:45:52.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/898: truncate d9/f3c 7645069 0 2024-04-24T22:45:52.545 INFO:tasks.workunit.client.1.smithi192.stdout:1/899: truncate d9/d15/f49 4749486 0 2024-04-24T22:45:52.546 INFO:tasks.workunit.client.1.smithi192.stdout:1/900: write d9/d15/d5e/ffe [2061389,95647] 0 2024-04-24T22:45:52.553 INFO:tasks.workunit.client.1.smithi192.stdout:1/901: dwrite d9/f42 [0,4194304] 0 2024-04-24T22:45:52.554 INFO:tasks.workunit.client.1.smithi192.stdout:1/902: dread - d9/d15/d2d/d48/d93/db4/f120 zero size 2024-04-24T22:45:52.557 INFO:tasks.workunit.client.1.smithi192.stdout:1/903: fsync d9/d15/d2d/d31/f62 0 2024-04-24T22:45:52.559 INFO:tasks.workunit.client.1.smithi192.stdout:5/912: dwrite d0/d13/d19/d64/d4f/d6c/fae [0,4194304] 0 2024-04-24T22:45:52.560 INFO:tasks.workunit.client.1.smithi192.stdout:5/913: chown d0/d13/d3b/d79 28 1 2024-04-24T22:45:52.569 INFO:tasks.workunit.client.1.smithi192.stdout:1/904: mkdir d9/d15/d2d/d48/d127 0 2024-04-24T22:45:52.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/905: fsync d9/d15/d2d/d4a/f5b 0 2024-04-24T22:45:52.572 INFO:tasks.workunit.client.1.smithi192.stdout:5/914: creat d0/f131 x:0 0 0 2024-04-24T22:45:52.576 INFO:tasks.workunit.client.1.smithi192.stdout:1/906: stat d9/c40 0 2024-04-24T22:45:52.580 INFO:tasks.workunit.client.1.smithi192.stdout:5/915: write d0/d1b/d50/fb1 [1056093,116287] 0 2024-04-24T22:45:52.583 INFO:tasks.workunit.client.1.smithi192.stdout:1/907: truncate d9/d15/d5e/d7c/fd3 366112 0 2024-04-24T22:45:52.586 INFO:tasks.workunit.client.1.smithi192.stdout:1/908: mkdir d9/dd/d106/d128 0 2024-04-24T22:45:52.595 INFO:tasks.workunit.client.1.smithi192.stdout:5/916: dwrite d0/d1b/d62/d10f/d2c/d3c/d91/d112/f7a [0,4194304] 0 2024-04-24T22:45:52.596 INFO:tasks.workunit.client.1.smithi192.stdout:1/909: dwrite d9/d15/d2d/d48/d93/db4/f120 [0,4194304] 0 2024-04-24T22:45:52.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/917: write d0/d13/d19/d71/dbf/f12d [933714,94542] 0 2024-04-24T22:45:52.605 INFO:tasks.workunit.client.1.smithi192.stdout:1/910: dwrite d9/d15/d2d/d48/d93/da4/db5/ff8 [0,4194304] 0 2024-04-24T22:45:52.613 INFO:tasks.workunit.client.1.smithi192.stdout:1/911: write d9/d15/d2d/d63/d68/d9d/fee [5046198,24808] 0 2024-04-24T22:45:52.614 INFO:tasks.workunit.client.1.smithi192.stdout:5/918: dwrite d0/d13/d3b/fda [0,4194304] 0 2024-04-24T22:45:52.618 INFO:tasks.workunit.client.1.smithi192.stdout:1/912: symlink d9/d15/d2d/d63/d68/d9d/l129 0 2024-04-24T22:45:52.619 INFO:tasks.workunit.client.1.smithi192.stdout:1/913: readlink d9/d15/d2d/d4a/l69 0 2024-04-24T22:45:52.628 INFO:tasks.workunit.client.1.smithi192.stdout:1/914: dwrite d9/d15/d2d/d48/d93/da4/fc9 [0,4194304] 0 2024-04-24T22:45:52.629 INFO:tasks.workunit.client.1.smithi192.stdout:1/915: chown d9/dd/d106/l123 1202 1 2024-04-24T22:45:52.632 INFO:tasks.workunit.client.1.smithi192.stdout:1/916: creat d9/d15/d2d/d48/d93/da4/dd2/dbf/f12a x:0 0 0 2024-04-24T22:45:52.635 INFO:tasks.workunit.client.1.smithi192.stdout:1/917: mkdir d9/d15/d5e/d90/da9/ddd/def/d12b 0 2024-04-24T22:45:52.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/919: write d0/d13/d19/d71/ffd [1092997,24725] 0 2024-04-24T22:45:52.643 INFO:tasks.workunit.client.1.smithi192.stdout:1/918: dwrite d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:52.649 INFO:tasks.workunit.client.1.smithi192.stdout:5/920: rename d0/d1b/d62/d10f/d2c/fff to d0/d13/f132 0 2024-04-24T22:45:52.654 INFO:tasks.workunit.client.1.smithi192.stdout:1/919: dwrite d9/d15/d2d/d4a/f8b [0,4194304] 0 2024-04-24T22:45:52.660 INFO:tasks.workunit.client.1.smithi192.stdout:5/921: rename d0/d90 to d0/d13/d133 0 2024-04-24T22:45:52.663 INFO:tasks.workunit.client.1.smithi192.stdout:5/922: readlink d0/d13/l9d 0 2024-04-24T22:45:52.663 INFO:tasks.workunit.client.1.smithi192.stdout:1/920: link d9/dd/c101 d9/d51/d70/c12c 0 2024-04-24T22:45:52.664 INFO:tasks.workunit.client.1.smithi192.stdout:5/923: readlink d0/d13/d19/d64/d4f/lb6 0 2024-04-24T22:45:52.670 INFO:tasks.workunit.client.1.smithi192.stdout:1/921: write d9/d6f/f86 [43037,59306] 0 2024-04-24T22:45:52.670 INFO:tasks.workunit.client.1.smithi192.stdout:5/924: mknod d0/d13/dd1/c134 0 2024-04-24T22:45:52.672 INFO:tasks.workunit.client.1.smithi192.stdout:1/922: truncate d9/d15/d2d/d31/f62 1213389 0 2024-04-24T22:45:52.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/923: mkdir d9/d15/d2d/d63/d68/d9d/d108/d12d 0 2024-04-24T22:45:52.676 INFO:tasks.workunit.client.1.smithi192.stdout:1/924: mkdir d9/d15/d5e/d7c/d12e 0 2024-04-24T22:45:52.682 INFO:tasks.workunit.client.1.smithi192.stdout:1/925: dwrite d9/d15/d2d/d31/f62 [0,4194304] 0 2024-04-24T22:45:52.685 INFO:tasks.workunit.client.1.smithi192.stdout:1/926: creat d9/d15/d2d/d48/d93/da4/db5/f12f x:0 0 0 2024-04-24T22:45:52.686 INFO:tasks.workunit.client.1.smithi192.stdout:1/927: write d9/d15/d2d/d4a/f8b [1945463,3555] 0 2024-04-24T22:45:52.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:52 smithi063 ceph-mon[26912]: Reconfiguring node-exporter.smithi063 (dependencies changed)... 2024-04-24T22:45:52.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:52 smithi063 ceph-mon[26912]: Reconfiguring daemon node-exporter.smithi063 on smithi063 2024-04-24T22:45:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:52 smithi192 ceph-mon[33991]: Reconfiguring node-exporter.smithi063 (dependencies changed)... 2024-04-24T22:45:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:52 smithi192 ceph-mon[33991]: Reconfiguring daemon node-exporter.smithi063 on smithi063 2024-04-24T22:45:53.273 INFO:tasks.workunit.client.1.smithi192.stdout:1/928: sync 2024-04-24T22:45:53.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/925: sync 2024-04-24T22:45:53.274 INFO:tasks.workunit.client.1.smithi192.stdout:5/926: write d0/d13/f117 [648387,93036] 0 2024-04-24T22:45:53.275 INFO:tasks.workunit.client.1.smithi192.stdout:1/929: read d9/d15/d5e/d7c/fe7 [59710,69694] 0 2024-04-24T22:45:53.278 INFO:tasks.workunit.client.1.smithi192.stdout:5/927: rename d0/d1b/d50/l6b to d0/d13/d19/d71/dcf/l135 0 2024-04-24T22:45:53.280 INFO:tasks.workunit.client.1.smithi192.stdout:5/928: creat d0/d13/d19/d71/dcf/f136 x:0 0 0 2024-04-24T22:45:53.283 INFO:tasks.workunit.client.1.smithi192.stdout:5/929: creat d0/d13/d19/d64/d4f/d6c/f137 x:0 0 0 2024-04-24T22:45:53.285 INFO:tasks.workunit.client.1.smithi192.stdout:5/930: mknod d0/d13/d19/d64/de7/c138 0 2024-04-24T22:45:53.288 INFO:tasks.workunit.client.1.smithi192.stdout:5/931: rename d0/d1b/f37 to d0/d13/d19/da9/f139 0 2024-04-24T22:45:53.289 INFO:tasks.workunit.client.1.smithi192.stdout:1/930: dread d9/d15/f49 [0,4194304] 0 2024-04-24T22:45:53.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/932: chown d0/f8b 8505900 1 2024-04-24T22:45:53.292 INFO:tasks.workunit.client.1.smithi192.stdout:5/933: write d0/d13/d3b/fda [2526810,96965] 0 2024-04-24T22:45:53.293 INFO:tasks.workunit.client.1.smithi192.stdout:5/934: readlink d0/d13/d19/da9/le3 0 2024-04-24T22:45:53.294 INFO:tasks.workunit.client.1.smithi192.stdout:1/931: dread f3 [0,4194304] 0 2024-04-24T22:45:53.294 INFO:tasks.workunit.client.1.smithi192.stdout:1/932: readlink d9/d6f/l11c 0 2024-04-24T22:45:53.299 INFO:tasks.workunit.client.1.smithi192.stdout:1/933: write d9/d6f/f77 [4790393,5760] 0 2024-04-24T22:45:53.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/934: chown d9/d15/d2d/d63/d68/c7b 38 1 2024-04-24T22:45:53.301 INFO:tasks.workunit.client.1.smithi192.stdout:5/935: creat d0/d13/d19/d71/f13a x:0 0 0 2024-04-24T22:45:53.303 INFO:tasks.workunit.client.1.smithi192.stdout:1/935: creat d9/d15/d2d/d4a/f130 x:0 0 0 2024-04-24T22:45:53.305 INFO:tasks.workunit.client.1.smithi192.stdout:1/936: fdatasync d9/d15/d5e/d90/da9/ddd/f10d 0 2024-04-24T22:45:53.305 INFO:tasks.workunit.client.1.smithi192.stdout:1/937: read - d9/d15/d5e/d90/f11f zero size 2024-04-24T22:45:53.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/936: rename d0/d1b/d62/dc1 to d0/d13/d13b 0 2024-04-24T22:45:53.307 INFO:tasks.workunit.client.1.smithi192.stdout:5/937: chown d0/d13/d53/l85 3426140 1 2024-04-24T22:45:53.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/938: chown d0/d1b/d62/fa7 5516 1 2024-04-24T22:45:53.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/938: write d9/d15/f47 [1603118,58405] 0 2024-04-24T22:45:53.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/939: truncate d0/d13/d3b/f55 3497965 0 2024-04-24T22:45:53.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/940: dread - d0/d13/f132 zero size 2024-04-24T22:45:53.323 INFO:tasks.workunit.client.1.smithi192.stdout:1/939: dwrite d9/d15/d2d/d4a/f96 [0,4194304] 0 2024-04-24T22:45:53.326 INFO:tasks.workunit.client.1.smithi192.stdout:1/940: truncate d9/dd/ff 4818358 0 2024-04-24T22:45:53.331 INFO:tasks.workunit.client.1.smithi192.stdout:5/941: dwrite d0/f8b [0,4194304] 0 2024-04-24T22:45:53.332 INFO:tasks.workunit.client.1.smithi192.stdout:1/941: rename d9/d15/d2d/l105 to d9/d15/d2d/d48/d93/l131 0 2024-04-24T22:45:53.334 INFO:tasks.workunit.client.1.smithi192.stdout:5/942: symlink d0/d13/d13b/dc2/l13c 0 2024-04-24T22:45:53.341 INFO:tasks.workunit.client.1.smithi192.stdout:1/942: dwrite d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:53.341 INFO:tasks.workunit.client.1.smithi192.stdout:5/943: rename d0/d1b to d0/d13/dd1/df1/d13d 0 2024-04-24T22:45:53.342 INFO:tasks.workunit.client.1.smithi192.stdout:5/944: chown d0/d13/dd1 28201 1 2024-04-24T22:45:53.344 INFO:tasks.workunit.client.1.smithi192.stdout:5/945: chown d0/d13/d3b/d79/le8 3246503 1 2024-04-24T22:45:53.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/943: symlink d9/d15/d2d/d48/d93/da4/db5/l132 0 2024-04-24T22:45:53.356 INFO:tasks.workunit.client.1.smithi192.stdout:5/946: link d0/d13/d19/d71/c104 d0/d13/d19/d71/c13e 0 2024-04-24T22:45:53.362 INFO:tasks.workunit.client.1.smithi192.stdout:1/944: write d9/d15/d2d/d48/d93/da4/dd2/dbf/de9/f121 [4106614,116988] 0 2024-04-24T22:45:53.364 INFO:tasks.workunit.client.1.smithi192.stdout:1/945: readlink d9/d15/d2d/d48/d93/db4/l11d 0 2024-04-24T22:45:53.369 INFO:tasks.workunit.client.1.smithi192.stdout:1/946: dread d9/d15/d2d/d31/d2f/f73 [0,4194304] 0 2024-04-24T22:45:53.370 INFO:tasks.workunit.client.1.smithi192.stdout:5/947: truncate d0/d13/dd1/df1/d13d/f24 1395227 0 2024-04-24T22:45:53.373 INFO:tasks.workunit.client.1.smithi192.stdout:5/948: write d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/f127 [58916,120615] 0 2024-04-24T22:45:53.375 INFO:tasks.workunit.client.1.smithi192.stdout:1/947: dwrite d9/d6f/f77 [0,4194304] 0 2024-04-24T22:45:53.376 INFO:tasks.workunit.client.1.smithi192.stdout:1/948: chown d9/d15/d2d/d63/f79 54 1 2024-04-24T22:45:53.376 INFO:tasks.workunit.client.1.smithi192.stdout:5/949: truncate d0/d13/d3b/f55 3292633 0 2024-04-24T22:45:53.377 INFO:tasks.workunit.client.1.smithi192.stdout:5/950: fsync d0/d13/d19/f60 0 2024-04-24T22:45:53.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/949: fsync d9/d15/d2d/d63/d68/d9d/fda 0 2024-04-24T22:45:53.383 INFO:tasks.workunit.client.1.smithi192.stdout:5/951: truncate d0/d13/d19/d64/d4f/d6c/ff6 614868 0 2024-04-24T22:45:53.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/950: symlink d9/d15/d2d/d63/d68/d9d/l133 0 2024-04-24T22:45:53.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/952: link d0/d13/dd1/df1/d13d/d62/d10f/d66/ldb d0/d13/d13b/l13f 0 2024-04-24T22:45:53.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/953: chown d0/d13/d19/cac 353362 1 2024-04-24T22:45:53.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/954: chown d0/d13/dd1/df1/d13d/d50/f7d 2209674 1 2024-04-24T22:45:53.395 INFO:tasks.workunit.client.1.smithi192.stdout:1/951: link d9/d15/d5e/d90/da9/ddd/ce2 d9/d51/c134 0 2024-04-24T22:45:53.401 INFO:tasks.workunit.client.1.smithi192.stdout:5/955: fsync d0/d13/dd1/df1/d13d/f24 0 2024-04-24T22:45:53.403 INFO:tasks.workunit.client.1.smithi192.stdout:1/952: mkdir d9/dd/d106/d128/d135 0 2024-04-24T22:45:53.409 INFO:tasks.workunit.client.1.smithi192.stdout:1/953: mkdir d9/d15/d2d/d48/d93/dfc/dac/d136 0 2024-04-24T22:45:53.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/954: mkdir d9/dd/d74/dcd/d11b/d137 0 2024-04-24T22:45:53.414 INFO:tasks.workunit.client.1.smithi192.stdout:5/956: dwrite d0/d13/dd1/d108/f96 [0,4194304] 0 2024-04-24T22:45:53.419 INFO:tasks.workunit.client.1.smithi192.stdout:5/957: mkdir d0/d140 0 2024-04-24T22:45:53.422 INFO:tasks.workunit.client.1.smithi192.stdout:5/958: read d0/d13/d19/d64/d4f/d52/fbb [3129407,94631] 0 2024-04-24T22:45:53.423 INFO:tasks.workunit.client.1.smithi192.stdout:5/959: chown d0/d13/d19/d64/d4f/d52/f119 105 1 2024-04-24T22:45:53.424 INFO:tasks.workunit.client.1.smithi192.stdout:5/960: dread - d0/d13/d19/d64/de7/f10d zero size 2024-04-24T22:45:53.425 INFO:tasks.workunit.client.1.smithi192.stdout:5/961: symlink d0/d140/l141 0 2024-04-24T22:45:53.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/962: mkdir d0/d13/d19/d71/dcf/d142 0 2024-04-24T22:45:53.428 INFO:tasks.workunit.client.1.smithi192.stdout:5/963: truncate d0/d10c/f12a 310528 0 2024-04-24T22:45:53.430 INFO:tasks.workunit.client.1.smithi192.stdout:5/964: mknod d0/d13/d3b/c143 0 2024-04-24T22:45:53.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/955: truncate d9/d15/d5e/d90/da9/ddd/fe0 150719 0 2024-04-24T22:45:53.433 INFO:tasks.workunit.client.1.smithi192.stdout:1/956: read d9/d15/d2d/d4a/f5b [2228870,24118] 0 2024-04-24T22:45:53.433 INFO:tasks.workunit.client.1.smithi192.stdout:1/957: chown d9/dd/le 0 1 2024-04-24T22:45:53.441 INFO:tasks.workunit.client.1.smithi192.stdout:5/965: write d0/d13/d3b/fbd [1493079,21514] 0 2024-04-24T22:45:53.441 INFO:tasks.workunit.client.1.smithi192.stdout:1/958: link d9/d15/d2d/d63/f79 d9/d15/f138 0 2024-04-24T22:45:53.444 INFO:tasks.workunit.client.1.smithi192.stdout:1/959: mknod d9/d15/d2d/d48/d93/da4/dd2/dbf/c139 0 2024-04-24T22:45:53.450 INFO:tasks.workunit.client.1.smithi192.stdout:1/960: write d9/d15/d2d/d4a/fce [4506846,58777] 0 2024-04-24T22:45:53.450 INFO:tasks.workunit.client.1.smithi192.stdout:5/966: write d0/d13/dd1/d108/ff0 [451700,125831] 0 2024-04-24T22:45:53.451 INFO:tasks.workunit.client.1.smithi192.stdout:1/961: chown d9/d6f 0 1 2024-04-24T22:45:53.455 INFO:tasks.workunit.client.1.smithi192.stdout:5/967: mkdir d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/d112/d105/d144 0 2024-04-24T22:45:53.457 INFO:tasks.workunit.client.1.smithi192.stdout:1/962: dwrite d9/d51/f10b [0,4194304] 0 2024-04-24T22:45:53.460 INFO:tasks.workunit.client.1.smithi192.stdout:5/968: fdatasync d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/fc3 0 2024-04-24T22:45:53.463 INFO:tasks.workunit.client.1.smithi192.stdout:1/963: symlink d9/d15/d5e/d90/da9/ddd/def/d12b/l13a 0 2024-04-24T22:45:53.471 INFO:tasks.workunit.client.1.smithi192.stdout:1/964: dwrite d9/d15/d5e/d90/da9/fc1 [0,4194304] 0 2024-04-24T22:45:53.473 INFO:tasks.workunit.client.1.smithi192.stdout:1/965: readlink d9/d15/d2d/d31/l56 0 2024-04-24T22:45:53.475 INFO:tasks.workunit.client.1.smithi192.stdout:5/969: dread d0/d13/d3b/d79/fdf [0,4194304] 0 2024-04-24T22:45:53.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/966: creat d9/d15/d2d/d48/d93/dc5/dd6/d10a/f13b x:0 0 0 2024-04-24T22:45:53.482 INFO:tasks.workunit.client.1.smithi192.stdout:5/970: rename d0/d13/d19/d71/dcf/l135 to d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/l145 0 2024-04-24T22:45:53.484 INFO:tasks.workunit.client.1.smithi192.stdout:1/967: getdents d9/dd/d74/dcd/d11b 0 2024-04-24T22:45:53.487 INFO:tasks.workunit.client.1.smithi192.stdout:1/968: creat d9/d15/d2d/d48/d93/da4/db5/f13c x:0 0 0 2024-04-24T22:45:53.488 INFO:tasks.workunit.client.1.smithi192.stdout:1/969: chown d9/dd 3724 1 2024-04-24T22:45:53.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/971: dwrite d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/d112/f7a [0,4194304] 0 2024-04-24T22:45:53.491 INFO:tasks.workunit.client.1.smithi192.stdout:1/970: write d9/d15/d2d/d31/d2f/f3a [1875261,16422] 0 2024-04-24T22:45:53.496 INFO:tasks.workunit.client.1.smithi192.stdout:5/972: mkdir d0/d13/d19/d71/d11e/d146 0 2024-04-24T22:45:53.497 INFO:tasks.workunit.client.1.smithi192.stdout:1/971: mkdir d9/d15/d2d/d48/d93/dfc/dac/d115/d13d 0 2024-04-24T22:45:53.498 INFO:tasks.workunit.client.1.smithi192.stdout:5/973: creat d0/d13/d19/da9/f147 x:0 0 0 2024-04-24T22:45:53.508 INFO:tasks.workunit.client.1.smithi192.stdout:5/974: dread d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/ff4 [0,4194304] 0 2024-04-24T22:45:53.509 INFO:tasks.workunit.client.1.smithi192.stdout:5/975: write d0/d13/d19/da9/f147 [366859,81858] 0 2024-04-24T22:45:53.512 INFO:tasks.workunit.client.1.smithi192.stdout:5/976: dread - d0/d13/d19/d83/dad/ff9 zero size 2024-04-24T22:45:53.514 INFO:tasks.workunit.client.1.smithi192.stdout:5/977: symlink d0/d13/d19/d83/dad/l148 0 2024-04-24T22:45:53.515 INFO:tasks.workunit.client.1.smithi192.stdout:5/978: fsync d0/d13/dd1/d108/ff0 0 2024-04-24T22:45:53.516 INFO:tasks.workunit.client.1.smithi192.stdout:5/979: unlink d0/d13/d19/d71/d88/la4 0 2024-04-24T22:45:53.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/980: rename d0/d13/d19/d71/daf/cb2 to d0/d13/d19/d64/d4f/c149 0 2024-04-24T22:45:53.533 INFO:tasks.workunit.client.1.smithi192.stdout:5/981: write d0/d13/f43 [969692,101092] 0 2024-04-24T22:45:53.539 INFO:tasks.workunit.client.1.smithi192.stdout:5/982: dwrite d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/d112/f7a [0,4194304] 0 2024-04-24T22:45:53.543 INFO:tasks.workunit.client.1.smithi192.stdout:5/983: mknod d0/d13/dd1/df1/c14a 0 2024-04-24T22:45:53.546 INFO:tasks.workunit.client.1.smithi192.stdout:5/984: rename d0/d13/d133/f11f to d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/d112/d105/d144/f14b 0 2024-04-24T22:45:53.549 INFO:tasks.workunit.client.1.smithi192.stdout:5/985: creat d0/d13/dd1/df1/d13d/d50/f14c x:0 0 0 2024-04-24T22:45:53.553 INFO:tasks.workunit.client.1.smithi192.stdout:5/986: rename d0/d13/d19/d83/dad to d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d14d 0 2024-04-24T22:45:53.555 INFO:tasks.workunit.client.1.smithi192.stdout:5/987: creat d0/d13/dd1/de1/f14e x:0 0 0 2024-04-24T22:45:53.570 INFO:tasks.workunit.client.1.smithi192.stdout:5/988: dwrite d0/d13/dd1/df1/d13d/f89 [0,4194304] 0 2024-04-24T22:45:53.572 INFO:tasks.workunit.client.1.smithi192.stdout:5/989: rmdir d0/d13/d53 39 2024-04-24T22:45:53.577 INFO:tasks.workunit.client.1.smithi192.stdout:5/990: rename d0/d13/d19/da9/f139 to d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/d112/d105/f14f 0 2024-04-24T22:45:53.578 INFO:tasks.workunit.client.1.smithi192.stdout:5/991: dread - d0/d13/dd1/df1/d13d/d50/f14c zero size 2024-04-24T22:45:53.582 INFO:tasks.workunit.client.1.smithi192.stdout:5/992: symlink d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/l150 0 2024-04-24T22:45:53.582 INFO:tasks.workunit.client.1.smithi192.stdout:5/993: write d0/d13/dd1/df1/d13d/d62/d10f/d2c/fa3 [1845089,96136] 0 2024-04-24T22:45:53.585 INFO:tasks.workunit.client.1.smithi192.stdout:5/994: read d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/fc3 [1346646,51359] 0 2024-04-24T22:45:53.586 INFO:tasks.workunit.client.1.smithi192.stdout:5/995: chown d0/d13/d19/l63 0 1 2024-04-24T22:45:53.591 INFO:tasks.workunit.client.1.smithi192.stdout:5/996: fdatasync d0/f73 0 2024-04-24T22:45:53.594 INFO:tasks.workunit.client.1.smithi192.stdout:5/997: read d0/d13/dd1/df1/d13d/f25 [277076,91810] 0 2024-04-24T22:45:53.597 INFO:tasks.workunit.client.1.smithi192.stdout:5/998: symlink d0/d13/dd1/df1/d13d/d62/d10f/d2c/d3c/d91/l151 0 2024-04-24T22:45:53.601 INFO:tasks.workunit.client.1.smithi192.stdout:5/999: link d0/d13/d19/d64/d4f/d52/fbb d0/d13/dd1/df1/d13d/d62/d10f/d66/d125/f152 0 2024-04-24T22:45:53.788 INFO:tasks.workunit.client.1.smithi192.stdout:1/972: sync 2024-04-24T22:45:53.791 INFO:tasks.workunit.client.1.smithi192.stdout:1/973: mknod d9/d15/d5e/d90/da9/ddd/def/d12b/c13e 0 2024-04-24T22:45:53.793 INFO:tasks.workunit.client.1.smithi192.stdout:1/974: creat d9/d15/d2d/d48/d93/dfc/dac/d11e/f13f x:0 0 0 2024-04-24T22:45:53.800 INFO:tasks.workunit.client.1.smithi192.stdout:1/975: dwrite d9/d15/d2d/d48/d93/da4/dd2/dbf/f12a [0,4194304] 0 2024-04-24T22:45:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:53 smithi063 ceph-mon[26912]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 102 MiB/s rd, 238 MiB/s wr, 573 op/s 2024-04-24T22:45:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:53 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:53 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:53.804 INFO:tasks.workunit.client.1.smithi192.stdout:1/976: creat d9/f140 x:0 0 0 2024-04-24T22:45:53.816 INFO:tasks.workunit.client.1.smithi192.stdout:1/977: write d9/d15/d2d/d31/d2f/f85 [4544735,85251] 0 2024-04-24T22:45:53.817 INFO:tasks.workunit.client.1.smithi192.stdout:1/978: stat d9/d15/d2d/d4a/fd0 0 2024-04-24T22:45:53.817 INFO:tasks.workunit.client.1.smithi192.stdout:1/979: stat d9/d15/d2d/d48/d93/lc7 0 2024-04-24T22:45:53.823 INFO:tasks.workunit.client.1.smithi192.stdout:1/980: dwrite d9/d51/d70/f7f [0,4194304] 0 2024-04-24T22:45:53.824 INFO:tasks.workunit.client.1.smithi192.stdout:1/981: stat d9/d51/d70 0 2024-04-24T22:45:53.825 INFO:tasks.workunit.client.1.smithi192.stdout:1/982: readlink d9/dd/l4d 0 2024-04-24T22:45:53.828 INFO:tasks.workunit.client.1.smithi192.stdout:1/983: getdents d9/d15/d2d/d48/d127 0 2024-04-24T22:45:53.830 INFO:tasks.workunit.client.1.smithi192.stdout:1/984: symlink d9/dd/d106/l141 0 2024-04-24T22:45:53.833 INFO:tasks.workunit.client.1.smithi192.stdout:1/985: mkdir d9/d15/d2d/d142 0 2024-04-24T22:45:53.837 INFO:tasks.workunit.client.1.smithi192.stdout:1/986: rename d9/d15/d2d/d48/d93/dfc/dac/d115/d13d to d9/d15/d2d/d63/d68/dd5/d143 0 2024-04-24T22:45:53.839 INFO:tasks.workunit.client.1.smithi192.stdout:1/987: fsync d9/d15/f138 0 2024-04-24T22:45:53.840 INFO:tasks.workunit.client.1.smithi192.stdout:1/988: dread - d9/d15/d2d/d48/d93/f98 zero size 2024-04-24T22:45:53.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/989: fsync d9/d15/d2d/d48/d93/dc5/dd6/d10a/f13b 0 2024-04-24T22:45:53.843 INFO:tasks.workunit.client.1.smithi192.stdout:1/990: mknod d9/c144 0 2024-04-24T22:45:53.845 INFO:tasks.workunit.client.1.smithi192.stdout:1/991: mknod d9/d15/d2d/d48/d93/da4/db5/c145 0 2024-04-24T22:45:53.847 INFO:tasks.workunit.client.1.smithi192.stdout:1/992: mknod d9/d51/d70/c146 0 2024-04-24T22:45:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:53 smithi192 ceph-mon[33991]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 102 MiB/s rd, 238 MiB/s wr, 573 op/s 2024-04-24T22:45:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:53 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:53 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:54.058 INFO:tasks.workunit.client.1.smithi192.stdout:1/993: sync 2024-04-24T22:45:54.062 INFO:tasks.workunit.client.1.smithi192.stdout:1/994: link d9/d15/d2d/d48/d93/lc7 d9/d15/d2d/l147 0 2024-04-24T22:45:54.064 INFO:tasks.workunit.client.1.smithi192.stdout:1/995: dread d9/f38 [0,4194304] 0 2024-04-24T22:45:54.067 INFO:tasks.workunit.client.1.smithi192.stdout:1/996: mknod d9/d15/d2d/d31/c148 0 2024-04-24T22:45:54.282 INFO:tasks.workunit.client.1.smithi192.stdout:1/997: sync 2024-04-24T22:45:54.283 INFO:tasks.workunit.client.1.smithi192.stdout:1/998: fsync d9/d15/d2d/d48/d93/da4/db5/f13c 0 2024-04-24T22:45:54.286 INFO:tasks.workunit.client.1.smithi192.stdout:1/999: creat d9/d15/d2d/d48/d93/da4/db5/f149 x:0 0 0 2024-04-24T22:45:54.289 INFO:tasks.workunit.client.1.smithi192.stderr:+ rm -rf -- ./tmp.gixOwSWXYz 2024-04-24T22:45:54.746 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:54 smithi063 ceph-mon[26912]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-04-24T22:45:54.746 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:54 smithi063 ceph-mon[26912]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-04-24T22:45:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:54 smithi192 ceph-mon[33991]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-04-24T22:45:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:54 smithi192 ceph-mon[33991]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-04-24T22:45:55.763 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:55 smithi063 ceph-mon[26912]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 59 MiB/s rd, 144 MiB/s wr, 352 op/s 2024-04-24T22:45:55.763 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:55 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:55 smithi192 ceph-mon[33991]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 59 MiB/s rd, 144 MiB/s wr, 352 op/s 2024-04-24T22:45:55.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:55 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:45:57.678 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:57 smithi063 ceph-mon[26912]: pgmap v26: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 60 MiB/s rd, 151 MiB/s wr, 386 op/s 2024-04-24T22:45:57.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:57 smithi192 ceph-mon[33991]: pgmap v26: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 60 MiB/s rd, 151 MiB/s wr, 386 op/s 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.694+0000 7f683f79e700 1 -- 172.21.15.63:0/3348150261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fcf10 msgr2=0x7f68380fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.694+0000 7f683f79e700 1 --2- 172.21.15.63:0/3348150261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fcf10 0x7f68380fd320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6834004940 tx=0x7f6834009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- 172.21.15.63:0/3348150261 shutdown_connections 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 --2- 172.21.15.63:0/3348150261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fe080 0x7f68380fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 --2- 172.21.15.63:0/3348150261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fcf10 0x7f68380fd320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- 172.21.15.63:0/3348150261 >> 172.21.15.63:0/3348150261 conn(0x7f68380f8560 msgr2=0x7f68380fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- 172.21.15.63:0/3348150261 shutdown_connections 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- 172.21.15.63:0/3348150261 wait complete. 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 Processor -- start 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- start start 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fcf10 0x7f6838071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.695+0000 7f683f79e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838070860 con 0x7f68380fe080 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683f79e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380709d0 con 0x7f68380fcf10 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42684/0 (socket says 172.21.15.63:42684) 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 -- 172.21.15.63:0/1498660772 learned_addr learned my addr 172.21.15.63:0/1498660772 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683d53a700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fcf10 0x7f6838071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:58.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 -- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fcf10 msgr2=0x7f6838071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fcf10 0x7f6838071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 -- 172.21.15.63:0/1498660772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834005040 con 0x7f68380fe080 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683cd39700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f682800eb40 tx=0x7f682800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6828007cc0 con 0x7f68380fe080 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6838070c50 con 0x7f68380fe080 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.696+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f682800f040 con 0x7f68380fe080 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.697+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682800ca10 con 0x7f68380fe080 2024-04-24T22:45:58.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.697+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838115990 con 0x7f68380fe080 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.698+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683805f5c0 con 0x7f68380fe080 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.698+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f682800cc80 con 0x7f68380fe080 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.699+0000 7f682dffb700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 0x7f682406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.699+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f682808a040 con 0x7f68380fe080 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.699+0000 7f683d53a700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 0x7f682406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:58.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.699+0000 7f683d53a700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 0x7f682406dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6834004890 tx=0x7f6834009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:58.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.704+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6828050100 con 0x7f68380fe080 2024-04-24T22:45:58.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.946+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6838102320 con 0x7f682406b790 2024-04-24T22:45:58.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.948+0000 7f682dffb700 1 -- 172.21.15.63:0/1498660772 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f6838102320 con 0x7f682406b790 2024-04-24T22:45:58.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 msgr2=0x7f682406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:58.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 0x7f682406dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6834004890 tx=0x7f6834009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:58.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 msgr2=0x7f68380702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:58.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f682800eb40 tx=0x7f682800b040 comp rx=0 tx=0).stop 2024-04-24T22:45:58.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 shutdown_connections 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f682406b790 0x7f682406dc40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68380fcf10 0x7f6838071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 --2- 172.21.15.63:0/1498660772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68380fe080 0x7f68380702b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 >> 172.21.15.63:0/1498660772 conn(0x7f68380f8560 msgr2=0x7f68380f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 shutdown_connections 2024-04-24T22:45:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:58.950+0000 7f683f79e700 1 -- 172.21.15.63:0/1498660772 wait complete. 2024-04-24T22:45:58.962 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:45:59.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/966372002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fb420 msgr2=0x7f6f780fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/966372002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fb420 0x7f6f780fb830 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f68004940 tx=0x7f6f68009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/966372002 shutdown_connections 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/966372002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fc6d0 0x7f6f780fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/966372002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fb420 0x7f6f780fb830 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/966372002 >> 172.21.15.63:0/966372002 conn(0x7f6f780f6b80 msgr2=0x7f6f780f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/966372002 shutdown_connections 2024-04-24T22:45:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.066+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/966372002 wait complete. 2024-04-24T22:45:59.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 Processor -- start 2024-04-24T22:45:59.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 -- start start 2024-04-24T22:45:59.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 0x7f6f7810fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78110f60 con 0x7f6f780fc6d0 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.067+0000 7f6f7d2cb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781110d0 con 0x7f6f780fb420 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42700/0 (socket says 172.21.15.63:42700) 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 -- 172.21.15.63:0/2782902581 learned_addr learned my addr 172.21.15.63:0/2782902581 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f76ffd700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 0x7f6f7810fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 -- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 msgr2=0x7f6f7810fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 0x7f6f7810fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 -- 172.21.15.63:0/2782902581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68005040 con 0x7f6f780fc6d0 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f76ffd700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 0x7f6f7810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:45:59.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f767fc700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b580 tx=0x7f6f6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c00f6f0 con 0x7f6f780fc6d0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f781113b0 con 0x7f6f780fc6d0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f6c00fd20 con 0x7f6f780fc6d0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.068+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f78111970 con 0x7f6f780fc6d0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.069+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c01a400 con 0x7f6f780fc6d0 2024-04-24T22:45:59.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.069+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7805f5c0 con 0x7f6f780fc6d0 2024-04-24T22:45:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.070+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f6f6c00f850 con 0x7f6f780fc6d0 2024-04-24T22:45:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.071+0000 7f6f5f7fe700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 0x7f6f6006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.071+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6f6c08a000 con 0x7f6f780fc6d0 2024-04-24T22:45:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.071+0000 7f6f76ffd700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 0x7f6f6006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.071+0000 7f6f76ffd700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 0x7f6f6006dc30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800b3b0 tx=0x7f6f68009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.076+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6f6c057780 con 0x7f6f780fc6d0 2024-04-24T22:45:59.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.329+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f78103790 con 0x7f6f6006b780 2024-04-24T22:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.335+0000 7f6f5f7fe700 1 -- 172.21.15.63:0/2782902581 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f6f78103790 con 0x7f6f6006b780 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 msgr2=0x7f6f6006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 0x7f6f6006dc30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800b3b0 tx=0x7f6f68009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 msgr2=0x7f6f78110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b580 tx=0x7f6f6c0091a0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 shutdown_connections 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6f6006b780 0x7f6f6006dc30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780fb420 0x7f6f7810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 --2- 172.21.15.63:0/2782902581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f780fc6d0 0x7f6f78110250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 >> 172.21.15.63:0/2782902581 conn(0x7f6f780f6b80 msgr2=0x7f6f780f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 shutdown_connections 2024-04-24T22:45:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.337+0000 7f6f7d2cb700 1 -- 172.21.15.63:0/2782902581 wait complete. 2024-04-24T22:45:59.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 -- 172.21.15.63:0/1639658120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 msgr2=0x7f8ca40fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/1639658120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca40fb5f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8c94004940 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:59.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 -- 172.21.15.63:0/1639658120 shutdown_connections 2024-04-24T22:45:59.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/1639658120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 0x7f8ca4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/1639658120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca40fb5f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 -- 172.21.15.63:0/1639658120 >> 172.21.15.63:0/1639658120 conn(0x7f8ca40f6b80 msgr2=0x7f8ca40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 -- 172.21.15.63:0/1639658120 shutdown_connections 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.497+0000 7f8ca8e18700 1 -- 172.21.15.63:0/1639658120 wait complete. 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 Processor -- start 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 -- start start 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 0x7f8ca410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca410e4c0 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca8e18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca410ed10 con 0x7f8ca40fbb30 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42712/0 (socket says 172.21.15.63:42712) 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca259c700 1 -- 172.21.15.63:0/3834303038 learned_addr learned my addr 172.21.15.63:0/3834303038 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.498+0000 7f8ca1d9b700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 0x7f8ca410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca259c700 1 -- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 msgr2=0x7f8ca410dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca259c700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 0x7f8ca410dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca259c700 1 -- 172.21.15.63:0/3834303038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94005040 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca259c700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c94013870 tx=0x7f8c94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9400e070 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca410ef90 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca410f470 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c9401c400 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.499+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9400b650 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.500+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca405f5c0 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.501+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f8c9401ca50 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.502+0000 7f8c92ffd700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 0x7f8c8c072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.502+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8c9408bac0 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.502+0000 7f8ca1d9b700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 0x7f8c8c072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.502+0000 7f8ca1d9b700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 0x7f8c8c072050 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009b80 tx=0x7f8c9800b040 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.506+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8c94059130 con 0x7f8ca40fb1e0 2024-04-24T22:45:59.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.770+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ca405f5c0 con 0x7f8c8c06fba0 2024-04-24T22:45:59.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:59 smithi063 ceph-mon[26912]: pgmap v27: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 49 MiB/s rd, 133 MiB/s wr, 327 op/s 2024-04-24T22:45:59.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:59 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:59.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:45:59 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.793+0000 7f8c92ffd700 1 -- 172.21.15.63:0/3834303038 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f8ca405f5c0 con 0x7f8c8c06fba0 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 starting - - - - 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 31s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (8m) 29s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 31s ago 8m 85.3M - 9.4.7 954c08fa6188 f74efe3bd239 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (3m) 31s ago 3m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (3m) 31s ago 3m 11.7M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:45:59.795 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (3m) 29s ago 3m 205M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (3m) 29s ago 3m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (10m) 31s ago 10m 411M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (72s) 29s ago 7m 491M - 19.0.0-2445-gd2c45d97 f68e37b807ef a965183fccc3 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 31s ago 10m 49.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (7m) 29s ago 7m 41.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 starting - - - - 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 29s ago 8m 15.9M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 31s ago 7m 110M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 31s ago 6m 123M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 31s ago 6m 100M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (5m) 29s ago 5m 145M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (5m) 29s ago 5m 132M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:45:59.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (5m) 29s ago 5m 115M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 31s ago 8m 56.1M - 2.43.0 a07b618ecd1d 35c4cee45df7 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 msgr2=0x7f8c8c072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 0x7f8c8c072050 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009b80 tx=0x7f8c9800b040 comp rx=0 tx=0).stop 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 msgr2=0x7f8ca410d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8c94013870 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 shutdown_connections 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f8c8c06fba0 0x7f8c8c072050 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ca40fb1e0 0x7f8ca410d980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 --2- 172.21.15.63:0/3834303038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ca40fbb30 0x7f8ca410dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 >> 172.21.15.63:0/3834303038 conn(0x7f8ca40f6b80 msgr2=0x7f8ca41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 shutdown_connections 2024-04-24T22:45:59.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.795+0000 7f8ca8e18700 1 -- 172.21.15.63:0/3834303038 wait complete. 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 -- 172.21.15.63:0/478184187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47980fdbc0 msgr2=0x7f47980fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 --2- 172.21.15.63:0/478184187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47980fdbc0 0x7f47980fbfe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f478c004940 tx=0x7f478c009e30 comp rx=0 tx=0).stop 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 -- 172.21.15.63:0/478184187 shutdown_connections 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 --2- 172.21.15.63:0/478184187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47980fdbc0 0x7f47980fbfe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 --2- 172.21.15.63:0/478184187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 0x7f47980fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 -- 172.21.15.63:0/478184187 >> 172.21.15.63:0/478184187 conn(0x7f47980f6b80 msgr2=0x7f47980f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 -- 172.21.15.63:0/478184187 shutdown_connections 2024-04-24T22:45:59.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.911+0000 7f479d346700 1 -- 172.21.15.63:0/478184187 wait complete. 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 Processor -- start 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 -- start start 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 0x7f479810fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981107f0 con 0x7f47981102a0 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f479d346700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981114f0 con 0x7f47980fd270 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42736/0 (socket says 172.21.15.63:42736) 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 -- 172.21.15.63:0/4147005031 learned_addr learned my addr 172.21.15.63:0/4147005031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f4796ffd700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 0x7f479810fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 -- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 msgr2=0x7f479810fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:45:59.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 0x7f479810fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:45:59.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.912+0000 7f47967fc700 1 -- 172.21.15.63:0/4147005031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c005040 con 0x7f47981102a0 2024-04-24T22:45:59.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.913+0000 7f47967fc700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f478c008d50 tx=0x7f478c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.913+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c00e070 con 0x7f47981102a0 2024-04-24T22:45:59.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.913+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4798111770 con 0x7f47981102a0 2024-04-24T22:45:59.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.913+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f478c00bbf0 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.914+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c013e80 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.914+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798111cd0 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.915+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f478c01c4d0 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.915+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479805f5c0 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.915+0000 7f477f7fe700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 0x7f478006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.915+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f478c08bc70 con 0x7f47981102a0 2024-04-24T22:45:59.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.915+0000 7f4796ffd700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 0x7f478006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:45:59.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.916+0000 7f4796ffd700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 0x7f478006dc50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47980650b0 tx=0x7f47880090f0 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:45:59.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:45:59.920+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f478c0593a0 con 0x7f47981102a0 2024-04-24T22:45:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:59 smithi192 ceph-mon[33991]: pgmap v27: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 49 MiB/s rd, 133 MiB/s wr, 327 op/s 2024-04-24T22:45:59.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:59 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:45:59.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:45:59 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:00.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.251+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4798107c80 con 0x7f47981102a0 2024-04-24T22:46:00.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.251+0000 7f477f7fe700 1 -- 172.21.15.63:0/4147005031 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f478c058f30 con 0x7f47981102a0 2024-04-24T22:46:00.253 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:46:00.253 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:46:00.253 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:00.254 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 msgr2=0x7f478006dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 0x7f478006dc50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47980650b0 tx=0x7f47880090f0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 msgr2=0x7f4798110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f478c008d50 tx=0x7f478c009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 shutdown_connections 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f478006b7a0 0x7f478006dc50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47980fd270 0x7f479810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 --2- 172.21.15.63:0/4147005031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47981102a0 0x7f4798110fb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.253+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 >> 172.21.15.63:0/4147005031 conn(0x7f47980f6b80 msgr2=0x7f47980f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.254+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 shutdown_connections 2024-04-24T22:46:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.254+0000 7f479d346700 1 -- 172.21.15.63:0/4147005031 wait complete. 2024-04-24T22:46:00.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/3659399362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0ffee0 msgr2=0x7f6ebc102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/3659399362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc102300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8004940 tx=0x7f6ea8009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/3659399362 shutdown_connections 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/3659399362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc102300 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/3659399362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/3659399362 >> 172.21.15.63:0/3659399362 conn(0x7f6ebc0f7430 msgr2=0x7f6ebc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/3659399362 shutdown_connections 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.368+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/3659399362 wait complete. 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 Processor -- start 2024-04-24T22:46:00.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 -- start start 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0fbe50 con 0x7f6ebc0662c0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec2e2b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0fbfc0 con 0x7f6ebc0ffee0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ec0bc7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:54382/0 (socket says 172.21.15.63:54382) 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.369+0000 7f6ebbfff700 1 -- 172.21.15.63:0/1974539187 learned_addr learned my addr 172.21.15.63:0/1974539187 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec0bc7700 1 -- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 msgr2=0x7f6ebc0fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec0bc7700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc0fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec0bc7700 1 -- 172.21.15.63:0/1974539187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea8005040 con 0x7f6ebc0662c0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec0bc7700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0fd3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc066000 tx=0x7f6eb0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb00106f0 con 0x7f6ebc0662c0 2024-04-24T22:46:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb0010d20 con 0x7f6ebc0662c0 2024-04-24T22:46:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc0fc2a0 con 0x7f6ebc0662c0 2024-04-24T22:46:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb00113a0 con 0x7f6ebc0662c0 2024-04-24T22:46:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.370+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc115a70 con 0x7f6ebc0662c0 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.371+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc05f5c0 con 0x7f6ebc0662c0 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.372+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f6eb0010850 con 0x7f6ebc0662c0 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.373+0000 7f6eb97fa700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 0x7f6eac06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.373+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6eb008a2e0 con 0x7f6ebc0662c0 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.373+0000 7f6ebbfff700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 0x7f6eac06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.374+0000 7f6ebbfff700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 0x7f6eac06dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8004f90 tx=0x7f6ea8009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:00.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.378+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6eb0053f30 con 0x7f6ebc0662c0 2024-04-24T22:46:00.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.691+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ebc0f8f20 con 0x7f6eac06b790 2024-04-24T22:46:00.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.693+0000 7f6eb97fa700 1 -- 172.21.15.63:0/1974539187 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f6ebc0f8f20 con 0x7f6eac06b790 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:46:00.695 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 msgr2=0x7f6eac06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 0x7f6eac06dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8004f90 tx=0x7f6ea8009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 msgr2=0x7f6ebc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0fd3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc066000 tx=0x7f6eb0004430 comp rx=0 tx=0).stop 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 shutdown_connections 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f6eac06b790 0x7f6eac06dc40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc0662c0 0x7f6ebc0fd3d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 --2- 172.21.15.63:0/1974539187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc0ffee0 0x7f6ebc0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:00.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 >> 172.21.15.63:0/1974539187 conn(0x7f6ebc0f7430 msgr2=0x7f6ebc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:00.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 shutdown_connections 2024-04-24T22:46:00.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:00.695+0000 7f6ec2e2b700 1 -- 172.21.15.63:0/1974539187 wait complete. 2024-04-24T22:46:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:00 smithi063 ceph-mon[26912]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:00 smithi063 ceph-mon[26912]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-04-24T22:46:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:00 smithi063 ceph-mon[26912]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-04-24T22:46:00.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:00 smithi063 ceph-mon[26912]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:00.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:00 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4147005031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:46:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:00 smithi192 ceph-mon[33991]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:00 smithi192 ceph-mon[33991]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-04-24T22:46:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:00 smithi192 ceph-mon[33991]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-04-24T22:46:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:00 smithi192 ceph-mon[33991]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:00.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:00 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4147005031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:46:01.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:01 smithi063 ceph-mon[26912]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:01.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:01 smithi063 ceph-mon[26912]: pgmap v28: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 25 MiB/s rd, 72 MiB/s wr, 237 op/s 2024-04-24T22:46:01.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:01 smithi192 ceph-mon[33991]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:01.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:01 smithi192 ceph-mon[33991]: pgmap v28: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 25 MiB/s rd, 72 MiB/s wr, 237 op/s 2024-04-24T22:46:02.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:02 smithi063 ceph-mon[26912]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:02.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:02 smithi192 ceph-mon[33991]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:03 smithi063 ceph-mon[26912]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 25 MiB/s rd, 72 MiB/s wr, 317 op/s 2024-04-24T22:46:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:03 smithi192 ceph-mon[33991]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 25 MiB/s rd, 72 MiB/s wr, 317 op/s 2024-04-24T22:46:05.553 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T22:46:05.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T22:46:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:05 smithi063 ceph-mon[26912]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 176 op/s 2024-04-24T22:46:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:05 smithi192 ceph-mon[33991]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 176 op/s 2024-04-24T22:46:07.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:07 smithi063 ceph-mon[26912]: pgmap v31: 65 pgs: 65 active+clean; 2.0 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 195 op/s 2024-04-24T22:46:07.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:07 smithi192 ceph-mon[33991]: pgmap v31: 65 pgs: 65 active+clean; 2.0 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 195 op/s 2024-04-24T22:46:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:09 smithi063 ceph-mon[26912]: pgmap v32: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 221 op/s 2024-04-24T22:46:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:09 smithi192 ceph-mon[33991]: pgmap v32: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 221 op/s 2024-04-24T22:46:10.240 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T22:46:10.241 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-24T22:46:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:10 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:10.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:10 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:11 smithi063 ceph-mon[26912]: pgmap v33: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-24T22:46:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:11 smithi192 ceph-mon[33991]: pgmap v33: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-24T22:46:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:13 smithi192 ceph-mon[33991]: pgmap v34: 65 pgs: 65 active+clean; 437 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-04-24T22:46:14.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:13 smithi063 ceph-mon[26912]: pgmap v34: 65 pgs: 65 active+clean; 437 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-04-24T22:46:15.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:15 smithi063 ceph-mon[26912]: pgmap v35: 65 pgs: 65 active+clean; 437 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-24T22:46:15.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:15 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:15.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:15 smithi192 ceph-mon[33991]: pgmap v35: 65 pgs: 65 active+clean; 437 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-24T22:46:15.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:15 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:16 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:16.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:16 smithi063 ceph-mon[26912]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:46:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:16 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:16 smithi192 ceph-mon[33991]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:46:17.764 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:17 smithi063 ceph-mon[26912]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:46:17.764 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:17 smithi063 ceph-mon[26912]: pgmap v36: 65 pgs: 65 active+clean; 358 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-04-24T22:46:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:17 smithi192 ceph-mon[33991]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:46:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:17 smithi192 ceph-mon[33991]: pgmap v36: 65 pgs: 65 active+clean; 358 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-04-24T22:46:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:19 smithi192 ceph-mon[33991]: pgmap v37: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-24T22:46:20.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:19 smithi063 ceph-mon[26912]: pgmap v37: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-24T22:46:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:21 smithi192 ceph-mon[33991]: pgmap v38: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-24T22:46:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:21 smithi063 ceph-mon[26912]: pgmap v38: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-24T22:46:23.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:22 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:23.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:22 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:23.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:22 smithi063 ceph-mon[26912]: Reconfiguring node-exporter.smithi192 (dependencies changed)... 2024-04-24T22:46:23.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:22 smithi063 ceph-mon[26912]: Reconfiguring daemon node-exporter.smithi192 on smithi192 2024-04-24T22:46:23.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:22 smithi063 ceph-mon[26912]: pgmap v39: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-04-24T22:46:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:22 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:23.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:22 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:23.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:22 smithi192 ceph-mon[33991]: Reconfiguring node-exporter.smithi192 (dependencies changed)... 2024-04-24T22:46:23.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:22 smithi192 ceph-mon[33991]: Reconfiguring daemon node-exporter.smithi192 on smithi192 2024-04-24T22:46:23.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:22 smithi192 ceph-mon[33991]: pgmap v39: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-04-24T22:46:25.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:24 smithi192 ceph-mon[33991]: pgmap v40: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2024-04-24T22:46:25.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:24 smithi063 ceph-mon[26912]: pgmap v40: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 98 op/s 2024-04-24T22:46:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:25 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:26.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:25 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:27.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:26 smithi192 ceph-mon[33991]: pgmap v41: 65 pgs: 65 active+clean; 304 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.4 MiB/s wr, 130 op/s 2024-04-24T22:46:27.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:26 smithi063 ceph-mon[26912]: pgmap v41: 65 pgs: 65 active+clean; 304 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.4 MiB/s wr, 130 op/s 2024-04-24T22:46:29.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:28 smithi192 ceph-mon[33991]: pgmap v42: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 109 op/s 2024-04-24T22:46:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:28 smithi063 ceph-mon[26912]: pgmap v42: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 109 op/s 2024-04-24T22:46:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 -- 172.21.15.63:0/2565371769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 msgr2=0x7f0fdc0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 --2- 172.21.15.63:0/2565371769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc0fd3e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4004940 tx=0x7f0fc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 -- 172.21.15.63:0/2565371769 shutdown_connections 2024-04-24T22:46:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 --2- 172.21.15.63:0/2565371769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc0fd3e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 --2- 172.21.15.63:0/2565371769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 -- 172.21.15.63:0/2565371769 >> 172.21.15.63:0/2565371769 conn(0x7f0fdc0f7440 msgr2=0x7f0fdc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 -- 172.21.15.63:0/2565371769 shutdown_connections 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.805+0000 7f0fe292e700 1 -- 172.21.15.63:0/2565371769 wait complete. 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 Processor -- start 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 -- start start 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc110690 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fe292e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc110800 con 0x7f0fdc0fbcc0 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fdb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fdb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43564/0 (socket says 172.21.15.63:43564) 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fdb7fe700 1 -- 172.21.15.63:0/1570256194 learned_addr learned my addr 172.21.15.63:0/1570256194 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.806+0000 7f0fdbfff700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fdb7fe700 1 -- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 msgr2=0x7f0fdc10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fdb7fe700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fdb7fe700 1 -- 172.21.15.63:0/1570256194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4005040 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fdbfff700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fdb7fe700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b3b0 tx=0x7f0fc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc400e040 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fc400bcc0 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc401c630 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc115200 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.807+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc115770 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.808+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc101900 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.809+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f0fc4013980 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.809+0000 7f0fd8ff9700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 0x7f0fc806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.809+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0fc408b1d0 con 0x7f0fdc0fcf70 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.809+0000 7f0fdbfff700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 0x7f0fc806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:30.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.810+0000 7f0fdbfff700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 0x7f0fc806dc50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009fd0 tx=0x7f0fcc009ee0 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:30.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:30.814+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0fc4058860 con 0x7f0fdc0fcf70 2024-04-24T22:46:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.057+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fdc02b340 con 0x7f0fc806b7a0 2024-04-24T22:46:31.059 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:30 smithi063 ceph-mon[26912]: pgmap v43: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-04-24T22:46:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.058+0000 7f0fd8ff9700 1 -- 172.21.15.63:0/1570256194 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0fdc02b340 con 0x7f0fc806b7a0 2024-04-24T22:46:31.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 msgr2=0x7f0fc806dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 0x7f0fc806dc50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009fd0 tx=0x7f0fcc009ee0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 msgr2=0x7f0fdc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b3b0 tx=0x7f0fc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 shutdown_connections 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0fc806b7a0 0x7f0fc806dc50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.060+0000 7f0fe292e700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fdc0fbcc0 0x7f0fdc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.061+0000 7f0fe292e700 1 --2- 172.21.15.63:0/1570256194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc0fcf70 0x7f0fdc110090 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.061+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 >> 172.21.15.63:0/1570256194 conn(0x7f0fdc0f7440 msgr2=0x7f0fdc066810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.061+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 shutdown_connections 2024-04-24T22:46:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.061+0000 7f0fe292e700 1 -- 172.21.15.63:0/1570256194 wait complete. 2024-04-24T22:46:31.073 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:46:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:30 smithi192 ceph-mon[33991]: pgmap v43: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-04-24T22:46:31.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 -- 172.21.15.63:0/1120480475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 msgr2=0x7f30200fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 --2- 172.21.15.63:0/1120480475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f30200fd270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f301c004940 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 -- 172.21.15.63:0/1120480475 shutdown_connections 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 --2- 172.21.15.63:0/1120480475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 0x7f30200fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 --2- 172.21.15.63:0/1120480475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f30200fd270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 -- 172.21.15.63:0/1120480475 >> 172.21.15.63:0/1120480475 conn(0x7f30200f8560 msgr2=0x7f30200fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 -- 172.21.15.63:0/1120480475 shutdown_connections 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 -- 172.21.15.63:0/1120480475 wait complete. 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.217+0000 7f30280ed700 1 Processor -- start 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f30280ed700 1 -- start start 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f30280ed700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f30280ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 0x7f30200702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f30280ed700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020070840 con 0x7f30200fce60 2024-04-24T22:46:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f30280ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020070980 con 0x7f30200fdfd0 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43594/0 (socket says 172.21.15.63:43594) 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 -- 172.21.15.63:0/2225612578 learned_addr learned my addr 172.21.15.63:0/2225612578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025688700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 0x7f30200702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 -- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 msgr2=0x7f30200702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 0x7f30200702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 -- 172.21.15.63:0/2225612578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c005040 con 0x7f30200fce60 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.218+0000 7f3025e89700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f301c009150 tx=0x7f301c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.219+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c00e040 con 0x7f30200fce60 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.219+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301c004290 con 0x7f30200fce60 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.219+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3020070bd0 con 0x7f30200fce60 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.219+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c01c630 con 0x7f30200fce60 2024-04-24T22:46:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.219+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30201159b0 con 0x7f30200fce60 2024-04-24T22:46:31.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.220+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302005f5c0 con 0x7f30200fce60 2024-04-24T22:46:31.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.220+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f301c00bbf0 con 0x7f30200fce60 2024-04-24T22:46:31.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.221+0000 7f30167fc700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 0x7f300c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.221+0000 7f3025688700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 0x7f300c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.221+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f301c08aee0 con 0x7f30200fce60 2024-04-24T22:46:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.222+0000 7f3025688700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 0x7f300c06dc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3010009fd0 tx=0x7f301000b040 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.226+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f301c054c00 con 0x7f30200fce60 2024-04-24T22:46:31.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.464+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30201022f0 con 0x7f300c06b790 2024-04-24T22:46:31.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.466+0000 7f30167fc700 1 -- 172.21.15.63:0/2225612578 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f30201022f0 con 0x7f300c06b790 2024-04-24T22:46:31.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 msgr2=0x7f300c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 0x7f300c06dc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3010009fd0 tx=0x7f301000b040 comp rx=0 tx=0).stop 2024-04-24T22:46:31.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 msgr2=0x7f3020071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f301c009150 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 shutdown_connections 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f300c06b790 0x7f300c06dc40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30200fce60 0x7f3020071d70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 --2- 172.21.15.63:0/2225612578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30200fdfd0 0x7f30200702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 >> 172.21.15.63:0/2225612578 conn(0x7f30200f8560 msgr2=0x7f30200f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 shutdown_connections 2024-04-24T22:46:31.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.468+0000 7f30280ed700 1 -- 172.21.15.63:0/2225612578 wait complete. 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 -- 172.21.15.63:0/3583985039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fdfd0 msgr2=0x7f15c80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 --2- 172.21.15.63:0/3583985039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fdfd0 0x7f15c80fe440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f15b0004940 tx=0x7f15b0009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 -- 172.21.15.63:0/3583985039 shutdown_connections 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 --2- 172.21.15.63:0/3583985039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fdfd0 0x7f15c80fe440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 --2- 172.21.15.63:0/3583985039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fce60 0x7f15c80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 -- 172.21.15.63:0/3583985039 >> 172.21.15.63:0/3583985039 conn(0x7f15c80f8540 msgr2=0x7f15c80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 -- 172.21.15.63:0/3583985039 shutdown_connections 2024-04-24T22:46:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.579+0000 7f15cde45700 1 -- 172.21.15.63:0/3583985039 wait complete. 2024-04-24T22:46:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 Processor -- start 2024-04-24T22:46:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 -- start start 2024-04-24T22:46:31.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fdfd0 0x7f15c8110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8110610 con 0x7f15c80fce60 2024-04-24T22:46:31.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15cde45700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8110780 con 0x7f15c80fdfd0 2024-04-24T22:46:31.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.580+0000 7f15c77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43604/0 (socket says 172.21.15.63:43604) 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 -- 172.21.15.63:0/749263013 learned_addr learned my addr 172.21.15.63:0/749263013 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c6ffd700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fdfd0 0x7f15c8110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 -- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fdfd0 msgr2=0x7f15c8110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fdfd0 0x7f15c8110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 -- 172.21.15.63:0/749263013 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0005040 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15c77fe700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f15c0001db0 tx=0x7f15c0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c000ed50 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c81151e0 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15c0007df0 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c0014640 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.581+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c81157a0 con 0x7f15c80fce60 2024-04-24T22:46:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.582+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c8101960 con 0x7f15c80fce60 2024-04-24T22:46:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.583+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f15c0007920 con 0x7f15c80fce60 2024-04-24T22:46:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.583+0000 7f15bffff700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 0x7f15b406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.583+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f15c008a0f0 con 0x7f15c80fce60 2024-04-24T22:46:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.583+0000 7f15c6ffd700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 0x7f15b406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.584+0000 7f15c6ffd700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 0x7f15b406dc50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f15b000b970 tx=0x7f15b0009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.588+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f15c0050100 con 0x7f15c80fce60 2024-04-24T22:46:31.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.813+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15c802b340 con 0x7f15b406b7a0 2024-04-24T22:46:31.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.823+0000 7f15bffff700 1 -- 172.21.15.63:0/749263013 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f15c802b340 con 0x7f15b406b7a0 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 starting - - - - 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 63s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (8m) 61s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 starting - - - - 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (3m) 63s ago 3m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (4m) 63s ago 4m 11.7M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:46:31.825 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (4m) 61s ago 4m 205M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (3m) 61s ago 3m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:9283 running (11m) 63s ago 11m 411M - 17.2.7-902-gc61472c8 0de407c1b3af 6371f13c8008 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (104s) 61s ago 8m 491M - 19.0.0-2445-gd2c45d97 f68e37b807ef a965183fccc3 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 63s ago 11m 49.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (8m) 61s ago 8m 41.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 starting - - - - 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 61s ago 8m 15.9M - 1.5.0 0da6a335fe13 2055b1ac5d67 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 63s ago 7m 110M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 63s ago 7m 123M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 63s ago 6m 100M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (6m) 61s ago 6m 145M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:46:31.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (5m) 61s ago 5m 132M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:46:31.827 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (5m) 61s ago 5m 115M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:46:31.827 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 starting - - - - 2024-04-24T22:46:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 msgr2=0x7f15b406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 0x7f15b406dc50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f15b000b970 tx=0x7f15b0009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 msgr2=0x7f15c810fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f15c0001db0 tx=0x7f15c0009ee0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 shutdown_connections 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f15b406b7a0 0x7f15b406dc50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c80fce60 0x7f15c810fad0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 --2- 172.21.15.63:0/749263013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c80fdfd0 0x7f15c8110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 >> 172.21.15.63:0/749263013 conn(0x7f15c80f8540 msgr2=0x7f15c80f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 shutdown_connections 2024-04-24T22:46:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.825+0000 7f15cde45700 1 -- 172.21.15.63:0/749263013 wait complete. 2024-04-24T22:46:31.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 -- 172.21.15.63:0/2999253198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fdff0 msgr2=0x7f0da00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 --2- 172.21.15.63:0/2999253198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fdff0 0x7f0da00fe460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d90004940 tx=0x7f0d90009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 -- 172.21.15.63:0/2999253198 shutdown_connections 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 --2- 172.21.15.63:0/2999253198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fdff0 0x7f0da00fe460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 --2- 172.21.15.63:0/2999253198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fce80 0x7f0da00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.928+0000 7f0da7470700 1 -- 172.21.15.63:0/2999253198 >> 172.21.15.63:0/2999253198 conn(0x7f0da00f8580 msgr2=0x7f0da00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 -- 172.21.15.63:0/2999253198 shutdown_connections 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 -- 172.21.15.63:0/2999253198 wait complete. 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 Processor -- start 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 -- start start 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fdff0 0x7f0da010de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da010e440 con 0x7f0da00fce80 2024-04-24T22:46:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da7470700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da010e5b0 con 0x7f0da00fdff0 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da520c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da520c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43632/0 (socket says 172.21.15.63:43632) 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.929+0000 7f0da520c700 1 -- 172.21.15.63:0/2092575100 learned_addr learned my addr 172.21.15.63:0/2092575100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da4a0b700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fdff0 0x7f0da010de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da520c700 1 -- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fdff0 msgr2=0x7f0da010de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da520c700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fdff0 0x7f0da010de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da520c700 1 -- 172.21.15.63:0/2092575100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90005040 con 0x7f0da00fce80 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da520c700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00eb40 tx=0x7f0d9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c007cc0 con 0x7f0da00fce80 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9c00f040 con 0x7f0da00fce80 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da0112f60 con 0x7f0da00fce80 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c00ca10 con 0x7f0da00fce80 2024-04-24T22:46:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.930+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da0113410 con 0x7f0da00fce80 2024-04-24T22:46:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.931+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da004c710 con 0x7f0da00fce80 2024-04-24T22:46:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.932+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f0d9c00cc80 con 0x7f0da00fce80 2024-04-24T22:46:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.932+0000 7f0d95ffb700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 0x7f0d8c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.932+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0d9c08a000 con 0x7f0da00fce80 2024-04-24T22:46:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.932+0000 7f0da4a0b700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 0x7f0d8c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.933+0000 7f0da4a0b700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 0x7f0d8c06dc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009b30 tx=0x7f0d90009e30 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:31.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:31.937+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0d9c050100 con 0x7f0da00fce80 2024-04-24T22:46:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:31 smithi192 ceph-mon[33991]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:31 smithi192 ceph-mon[33991]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.288+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0da01138b0 con 0x7f0da00fce80 2024-04-24T22:46:32.291 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:31 smithi063 ceph-mon[26912]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:32.291 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:31 smithi063 ceph-mon[26912]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.289+0000 7f0d95ffb700 1 -- 172.21.15.63:0/2092575100 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0d9c0576a0 con 0x7f0da00fce80 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:32.291 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:46:32.292 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 msgr2=0x7f0d8c06dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 0x7f0d8c06dc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009b30 tx=0x7f0d90009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 msgr2=0x7f0da010d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00eb40 tx=0x7f0d9c00b040 comp rx=0 tx=0).stop 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 shutdown_connections 2024-04-24T22:46:32.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7f0d8c06b7a0 0x7f0d8c06dc50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da00fce80 0x7f0da010d900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 --2- 172.21.15.63:0/2092575100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00fdff0 0x7f0da010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 >> 172.21.15.63:0/2092575100 conn(0x7f0da00f8580 msgr2=0x7f0da00f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:32.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 shutdown_connections 2024-04-24T22:46:32.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.291+0000 7f0da7470700 1 -- 172.21.15.63:0/2092575100 wait complete. 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 -- 172.21.15.63:0/1847328573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 msgr2=0x7ff8040fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 --2- 172.21.15.63:0/1847328573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040fe4f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8004940 tx=0x7ff7f8009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 -- 172.21.15.63:0/1847328573 shutdown_connections 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 --2- 172.21.15.63:0/1847328573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040fe4f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 --2- 172.21.15.63:0/1847328573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 0x7ff8040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.396+0000 7ff809b68700 1 -- 172.21.15.63:0/1847328573 >> 172.21.15.63:0/1847328573 conn(0x7ff8040f8560 msgr2=0x7ff8040fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 -- 172.21.15.63:0/1847328573 shutdown_connections 2024-04-24T22:46:32.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 -- 172.21.15.63:0/1847328573 wait complete. 2024-04-24T22:46:32.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 Processor -- start 2024-04-24T22:46:32.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 -- start start 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 0x7ff804071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804070840 con 0x7ff8040fe080 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.397+0000 7ff809b68700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804070980 con 0x7ff8040fcf10 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43644/0 (socket says 172.21.15.63:43644) 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 -- 172.21.15.63:0/719278166 learned_addr learned my addr 172.21.15.63:0/719278166 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:46:32.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff8037fe700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 0x7ff804071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 -- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 msgr2=0x7ff804071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 0x7ff804071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 -- 172.21.15.63:0/719278166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8005040 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff802ffd700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8013870 tx=0x7ff7f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7f8021070 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff804070c00 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7f8009430 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7f800e070 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.398+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8041159e0 con 0x7ff8040fe080 2024-04-24T22:46:32.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.399+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80405f5c0 con 0x7ff8040fe080 2024-04-24T22:46:32.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.400+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7ff7f801c750 con 0x7ff8040fe080 2024-04-24T22:46:32.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.400+0000 7ff7ebfff700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 0x7ff7ec06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:46:32.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.400+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff7f808b5f0 con 0x7ff8040fe080 2024-04-24T22:46:32.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.400+0000 7ff8037fe700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 0x7ff7ec06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:46:32.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.401+0000 7ff8037fe700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 0x7ff7ec06dbe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff804071b20 tx=0x7ff7f400b040 comp rx=0 tx=0).ready entity=mgr.24353 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:46:32.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.405+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff7f8055250 con 0x7ff8040fe080 2024-04-24T22:46:32.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.637+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 --> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff804102350 con 0x7ff7ec06b730 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.638+0000 7ff7ebfff700 1 -- 172.21.15.63:0/719278166 <== mgr.24353 v2:172.21.15.192:6828/4284728814 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff804102350 con 0x7ff7ec06b730 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:46:32.640 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T22:46:32.641 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-04-24T22:46:32.641 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:46:32.641 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:46:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 msgr2=0x7ff7ec06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 0x7ff7ec06dbe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff804071b20 tx=0x7ff7f400b040 comp rx=0 tx=0).stop 2024-04-24T22:46:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 msgr2=0x7ff8040702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:46:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8013870 tx=0x7ff7f8009e30 comp rx=0 tx=0).stop 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 shutdown_connections 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:6828/4284728814,v1:172.21.15.192:6829/4284728814] conn(0x7ff7ec06b730 0x7ff7ec06dbe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8040fcf10 0x7ff804071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 --2- 172.21.15.63:0/719278166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8040fe080 0x7ff8040702b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 >> 172.21.15.63:0/719278166 conn(0x7ff8040f8560 msgr2=0x7ff8040f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 shutdown_connections 2024-04-24T22:46:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:46:32.640+0000 7ff809b68700 1 -- 172.21.15.63:0/719278166 wait complete. 2024-04-24T22:46:33.173 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:32 smithi192 ceph-mon[33991]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:33.173 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:32 smithi192 ceph-mon[33991]: pgmap v44: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-04-24T22:46:33.173 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:32 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2092575100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:46:33.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:32 smithi063 ceph-mon[26912]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:33.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:32 smithi063 ceph-mon[26912]: pgmap v44: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-04-24T22:46:33.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:32 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2092575100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:46:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:33 smithi192 ceph-mon[33991]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:34.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:33 smithi063 ceph-mon[26912]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: pgmap v45: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T22:46:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: Upgrade: Need to upgrade myself (mgr.smithi192.ctmwwc) 2024-04-24T22:46:35.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:34 smithi063 ceph-mon[26912]: Upgrade: Need to upgrade myself (mgr.smithi192.ctmwwc) 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: pgmap v45: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:46:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: Upgrade: Need to upgrade myself (mgr.smithi192.ctmwwc) 2024-04-24T22:46:35.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:34 smithi192 ceph-mon[33991]: Upgrade: Need to upgrade myself (mgr.smithi192.ctmwwc) 2024-04-24T22:46:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: pgmap v46: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-24T22:46:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: Upgrade: Updating mgr.smithi063.upcnvd 2024-04-24T22:46:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:37.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:46:37.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:46:37.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:37.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:37 smithi063 ceph-mon[26912]: Deploying daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:46:37.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: pgmap v46: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-24T22:46:37.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: Upgrade: Updating mgr.smithi063.upcnvd 2024-04-24T22:46:37.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:37.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:46:37.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:46:37.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:37.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:37 smithi192 ceph-mon[33991]: Deploying daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:46:39.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:39 smithi192 ceph-mon[33991]: pgmap v47: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-24T22:46:39.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:39 smithi063 ceph-mon[26912]: pgmap v47: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-24T22:46:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:40 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:40.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:40 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:41.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:41 smithi192 ceph-mon[33991]: pgmap v48: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-24T22:46:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:41 smithi063 ceph-mon[26912]: pgmap v48: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-24T22:46:43.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:43 smithi063 ceph-mon[26912]: pgmap v49: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-04-24T22:46:43.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:43 smithi063 ceph-mon[26912]: mgrmap e26: smithi192.ctmwwc(active, since 92s), standbys: smithi063.upcnvd 2024-04-24T22:46:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:43 smithi192 ceph-mon[33991]: pgmap v49: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-04-24T22:46:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:43 smithi192 ceph-mon[33991]: mgrmap e26: smithi192.ctmwwc(active, since 92s), standbys: smithi063.upcnvd 2024-04-24T22:46:45.664 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:45 smithi192 ceph-mon[33991]: pgmap v50: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-04-24T22:46:45.664 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:45 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:45.664 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:45 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:45.664 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:45 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:45 smithi063 ceph-mon[26912]: pgmap v50: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-04-24T22:46:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:45 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:45 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:45 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:46.702 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:46 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:46.703 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:46 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:46.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:46 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:46.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:46 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:47 smithi063 ceph-mon[26912]: pgmap v51: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-04-24T22:46:47.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:47 smithi192 ceph-mon[33991]: pgmap v51: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-04-24T22:46:49.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:49 smithi063 ceph-mon[26912]: pgmap v52: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.3 MiB/s wr, 343 op/s 2024-04-24T22:46:49.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:49 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:49.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:49 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:49.595 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:49 smithi192 ceph-mon[33991]: pgmap v52: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.3 MiB/s wr, 343 op/s 2024-04-24T22:46:49.595 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:49 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:49.596 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:49 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:51 smithi063 ceph-mon[26912]: pgmap v53: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.0 MiB/s wr, 320 op/s 2024-04-24T22:46:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:51 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:51 smithi192 ceph-mon[33991]: pgmap v53: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.0 MiB/s wr, 320 op/s 2024-04-24T22:46:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:51 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: pgmap v54: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.7 MiB/s wr, 445 op/s 2024-04-24T22:46:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: Standby manager daemon smithi063.upcnvd restarted 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: Standby manager daemon smithi063.upcnvd started 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/crt"}]: dispatch 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/key"}]: dispatch 2024-04-24T22:46:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:52 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:46:53.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: pgmap v54: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.7 MiB/s wr, 445 op/s 2024-04-24T22:46:53.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:53.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: Standby manager daemon smithi063.upcnvd restarted 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: Standby manager daemon smithi063.upcnvd started 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/crt"}]: dispatch 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.upcnvd/key"}]: dispatch 2024-04-24T22:46:53.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:52 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:46:54.867 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:54 smithi063 ceph-mon[26912]: mgrmap e27: smithi192.ctmwwc(active, since 103s), standbys: smithi063.upcnvd 2024-04-24T22:46:54.867 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:54 smithi063 ceph-mon[26912]: pgmap v55: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.7 MiB/s wr, 399 op/s 2024-04-24T22:46:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:54 smithi192 ceph-mon[33991]: mgrmap e27: smithi192.ctmwwc(active, since 103s), standbys: smithi063.upcnvd 2024-04-24T22:46:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:54 smithi192 ceph-mon[33991]: pgmap v55: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.7 MiB/s wr, 399 op/s 2024-04-24T22:46:55.898 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:55 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:55 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:56.938 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:56 smithi063 ceph-mon[26912]: pgmap v56: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.8 MiB/s wr, 461 op/s 2024-04-24T22:46:56.938 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:56 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:56.938 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:56 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:56.939 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:56 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:56.939 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:56 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:46:57.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:56 smithi192 ceph-mon[33991]: pgmap v56: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.8 MiB/s wr, 461 op/s 2024-04-24T22:46:57.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:56 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:57.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:56 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:57.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:56 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:46:57.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:56 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr fail", "who": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:46:58.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd='[{"prefix": "mgr fail", "who": "smithi192.ctmwwc"}]': finished 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: mgrmap e28: smithi063.upcnvd(active, starting, since 0.00613046s) 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:46:58.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:46:58.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:46:58.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:58.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:46:58.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:58.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:46:58.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:46:58.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T22:46:58.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' 2024-04-24T22:46:58.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "mgr fail", "who": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.24353 172.21.15.192:0/3668249681' entity='mgr.smithi192.ctmwwc' cmd='[{"prefix": "mgr fail", "who": "smithi192.ctmwwc"}]': finished 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: mgrmap e28: smithi063.upcnvd(active, starting, since 0.00613046s) 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:46:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:46:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:46:58.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: [24/Apr/2024:22:46:57] ENGINE Bus STARTING 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: [24/Apr/2024:22:46:57] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: [24/Apr/2024:22:46:57] ENGINE Serving on http://172.21.15.63:8765 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: [24/Apr/2024:22:46:57] ENGINE Bus STARTED 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: mgrmap e29: smithi063.upcnvd(active, since 1.00808s) 2024-04-24T22:46:58.971 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:58 smithi063 ceph-mon[26912]: pgmap v3: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:46:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: [24/Apr/2024:22:46:57] ENGINE Bus STARTING 2024-04-24T22:46:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: [24/Apr/2024:22:46:57] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:46:59.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: [24/Apr/2024:22:46:57] ENGINE Serving on http://172.21.15.63:8765 2024-04-24T22:46:59.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: [24/Apr/2024:22:46:57] ENGINE Bus STARTED 2024-04-24T22:46:59.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: mgrmap e29: smithi063.upcnvd(active, since 1.00808s) 2024-04-24T22:46:59.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:58 smithi192 ceph-mon[33991]: pgmap v3: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:00.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:59 smithi063 ceph-mon[26912]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T22:47:00.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:59 smithi063 ceph-mon[26912]: pgmap v4: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:00.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:46:59 smithi063 ceph-mon[26912]: mgrmap e30: smithi063.upcnvd(active, since 2s) 2024-04-24T22:47:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:59 smithi192 ceph-mon[33991]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T22:47:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:59 smithi192 ceph-mon[33991]: pgmap v4: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:00.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:46:59 smithi192 ceph-mon[33991]: mgrmap e30: smithi063.upcnvd(active, since 2s) 2024-04-24T22:47:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:01 smithi192 ceph-mon[33991]: pgmap v5: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:01 smithi192 ceph-mon[33991]: mgrmap e31: smithi063.upcnvd(active, since 4s) 2024-04-24T22:47:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:01 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:02.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:01 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:01 smithi063 ceph-mon[26912]: pgmap v5: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:01 smithi063 ceph-mon[26912]: mgrmap e31: smithi063.upcnvd(active, since 4s) 2024-04-24T22:47:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:01 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:01 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:02.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 -- 172.21.15.63:0/1067382122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fdfd0 msgr2=0x7f54b80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:02.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 --2- 172.21.15.63:0/1067382122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fdfd0 0x7f54b80fe440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f54ac004940 tx=0x7f54ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:02.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 -- 172.21.15.63:0/1067382122 shutdown_connections 2024-04-24T22:47:02.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 --2- 172.21.15.63:0/1067382122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fdfd0 0x7f54b80fe440 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:02.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 --2- 172.21.15.63:0/1067382122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fce60 0x7f54b80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:02.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 -- 172.21.15.63:0/1067382122 >> 172.21.15.63:0/1067382122 conn(0x7f54b80f8540 msgr2=0x7f54b80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:02.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 -- 172.21.15.63:0/1067382122 shutdown_connections 2024-04-24T22:47:02.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.763+0000 7f54bd58b700 1 -- 172.21.15.63:0/1067382122 wait complete. 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 Processor -- start 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 -- start start 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fdfd0 0x7f54b810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b810e400 con 0x7f54b80fce60 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54bd58b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b810e570 con 0x7f54b80fdfd0 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.764+0000 7f54b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b67fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fdfd0 0x7f54b810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36418/0 (socket says 172.21.15.63:36418) 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 -- 172.21.15.63:0/2868059981 learned_addr learned my addr 172.21.15.63:0/2868059981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 -- 172.21.15.63:0/2868059981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fdfd0 msgr2=0x7f54b810de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fdfd0 0x7f54b810de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 -- 172.21.15.63:0/2868059981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac005040 con 0x7f54b80fce60 2024-04-24T22:47:02.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54b6ffd700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f54a800eb40 tx=0x7f54a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:02.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a8007cc0 con 0x7f54b80fce60 2024-04-24T22:47:02.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54a800f040 con 0x7f54b80fce60 2024-04-24T22:47:02.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b8070190 con 0x7f54b80fce60 2024-04-24T22:47:02.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a800ca10 con 0x7f54b80fce60 2024-04-24T22:47:02.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.765+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b80706d0 con 0x7f54b80fce60 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.767+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b804c710 con 0x7f54b80fce60 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.767+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f54a800cb70 con 0x7f54b80fce60 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.767+0000 7f549f7fe700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 0x7f54a003b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.767+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f54a804c7c0 con 0x7f54b80fce60 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.767+0000 7f54b67fc700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 0x7f54a003b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:02.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.768+0000 7f54b67fc700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 0x7f54a003b3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009b30 tx=0x7f54ac01c040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:02.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:02.772+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f54a8051060 con 0x7f54b80fce60 2024-04-24T22:47:03.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.025+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54b81022a0 con 0x7f54a0038f20 2024-04-24T22:47:03.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.027+0000 7f549f7fe700 1 -- 172.21.15.63:0/2868059981 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f54b81022a0 con 0x7f54a0038f20 2024-04-24T22:47:03.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 msgr2=0x7f54a003b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 0x7f54a003b3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009b30 tx=0x7f54ac01c040 comp rx=0 tx=0).stop 2024-04-24T22:47:03.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 msgr2=0x7f54b810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f54a800eb40 tx=0x7f54a800b040 comp rx=0 tx=0).stop 2024-04-24T22:47:03.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 shutdown_connections 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f54a0038f20 0x7f54a003b3d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54b80fce60 0x7f54b810d8c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 --2- 172.21.15.63:0/2868059981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b80fdfd0 0x7f54b810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 >> 172.21.15.63:0/2868059981 conn(0x7f54b80f8540 msgr2=0x7f54b80f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 shutdown_connections 2024-04-24T22:47:03.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.030+0000 7f54bd58b700 1 -- 172.21.15.63:0/2868059981 wait complete. 2024-04-24T22:47:03.044 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 -- 172.21.15.63:0/3494820181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 msgr2=0x7f37100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3494820181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f37100fc0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f370c004940 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 -- 172.21.15.63:0/3494820181 shutdown_connections 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3494820181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 0x7f37100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3494820181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f37100fc0c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 -- 172.21.15.63:0/3494820181 >> 172.21.15.63:0/3494820181 conn(0x7f37100f7430 msgr2=0x7f37100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 -- 172.21.15.63:0/3494820181 shutdown_connections 2024-04-24T22:47:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.151+0000 7f3717ebb700 1 -- 172.21.15.63:0/3494820181 wait complete. 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 Processor -- start 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 -- start start 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 0x7f37101145b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710114bb0 con 0x7f37100fbcb0 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.152+0000 7f3717ebb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710115400 con 0x7f37100fcf60 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36436/0 (socket says 172.21.15.63:36436) 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 -- 172.21.15.63:0/3315992880 learned_addr learned my addr 172.21.15.63:0/3315992880 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:03.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715456700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 0x7f37101145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 -- 172.21.15.63:0/3315992880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 msgr2=0x7f37101145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 0x7f37101145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 -- 172.21.15.63:0/3315992880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c005040 con 0x7f37100fbcb0 2024-04-24T22:47:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.153+0000 7f3715c57700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f370c009150 tx=0x7f370c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.154+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c00e040 con 0x7f37100fbcb0 2024-04-24T22:47:03.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.154+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f370c004290 con 0x7f37100fbcb0 2024-04-24T22:47:03.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.154+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3710115680 con 0x7f37100fbcb0 2024-04-24T22:47:03.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.154+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3710115bc0 con 0x7f37100fbcb0 2024-04-24T22:47:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.155+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c01c630 con 0x7f37100fbcb0 2024-04-24T22:47:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.155+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f40052e0 con 0x7f37100fbcb0 2024-04-24T22:47:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.155+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f370c026440 con 0x7f37100fbcb0 2024-04-24T22:47:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.156+0000 7f37067fc700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 0x7f36fc03b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.156+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f370c04d9f0 con 0x7f37100fbcb0 2024-04-24T22:47:03.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.156+0000 7f3715456700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 0x7f36fc03b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.157+0000 7f3715456700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 0x7f36fc03b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3700004f40 tx=0x7f370000b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.161+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f370c011370 con 0x7f37100fbcb0 2024-04-24T22:47:03.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.478+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36f4000bf0 con 0x7f36fc038f10 2024-04-24T22:47:03.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.479+0000 7f37067fc700 1 -- 172.21.15.63:0/3315992880 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f36f4000bf0 con 0x7f36fc038f10 2024-04-24T22:47:03.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 msgr2=0x7f36fc03b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 0x7f36fc03b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3700004f40 tx=0x7f370000b040 comp rx=0 tx=0).stop 2024-04-24T22:47:03.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 msgr2=0x7f3710114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f370c009150 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 shutdown_connections 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f36fc038f10 0x7f36fc03b3c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37100fbcb0 0x7f3710114070 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 --2- 172.21.15.63:0/3315992880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37100fcf60 0x7f37101145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 >> 172.21.15.63:0/3315992880 conn(0x7f37100f7430 msgr2=0x7f37100f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 shutdown_connections 2024-04-24T22:47:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.482+0000 7f3717ebb700 1 -- 172.21.15.63:0/3315992880 wait complete. 2024-04-24T22:47:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 -- 172.21.15.63:0/1694888427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 msgr2=0x7f9c740fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 --2- 172.21.15.63:0/1694888427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c740fc0c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9c70004940 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 -- 172.21.15.63:0/1694888427 shutdown_connections 2024-04-24T22:47:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 --2- 172.21.15.63:0/1694888427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 0x7f9c740fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 --2- 172.21.15.63:0/1694888427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c740fc0c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.601+0000 7f9c7c160700 1 -- 172.21.15.63:0/1694888427 >> 172.21.15.63:0/1694888427 conn(0x7f9c740f7430 msgr2=0x7f9c740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 -- 172.21.15.63:0/1694888427 shutdown_connections 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 -- 172.21.15.63:0/1694888427 wait complete. 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 Processor -- start 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 -- start start 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 0x7f9c74110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74110710 con 0x7f9c740fbcb0 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c7c160700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74110f10 con 0x7f9c740fcf60 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c79efc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c79efc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36444/0 (socket says 172.21.15.63:36444) 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c79efc700 1 -- 172.21.15.63:0/980879070 learned_addr learned my addr 172.21.15.63:0/980879070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.602+0000 7f9c796fb700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 0x7f9c74110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c79efc700 1 -- 172.21.15.63:0/980879070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 msgr2=0x7f9c74110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c79efc700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 0x7f9c74110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c79efc700 1 -- 172.21.15.63:0/980879070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70005040 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c79efc700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9c70013870 tx=0x7f9c70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c70021070 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c70009430 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.603+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7000e070 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.604+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c74111190 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.604+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c741116d0 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.605+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f9c7001c750 con 0x7f9c740fbcb0 2024-04-24T22:47:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.605+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c741018f0 con 0x7f9c740fbcb0 2024-04-24T22:47:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.605+0000 7f9c6a7fc700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 0x7f9c6003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.605+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9c7004df20 con 0x7f9c740fbcb0 2024-04-24T22:47:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.606+0000 7f9c796fb700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 0x7f9c6003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.607+0000 7f9c796fb700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 0x7f9c6003b360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009b80 tx=0x7f9c6400b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:03.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.611+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f9c70024e10 con 0x7f9c740fbcb0 2024-04-24T22:47:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: pgmap v6: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:03.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:03 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.857+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c74111940 con 0x7f9c60038eb0 2024-04-24T22:47:03.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.870+0000 7f9c6a7fc700 1 -- 172.21.15.63:0/980879070 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9c74111940 con 0x7f9c60038eb0 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (65s) 1s ago 10m 17.0M - 0.25.0 c8568f914cd2 f156d383c8ab 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (10m) 1s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (9m) 2s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 unknown 1s ago 9m - - 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (4m) 1s ago 4m 14.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (4m) 1s ago 4m 12.3M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (4m) 2s ago 4m 210M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (4m) 2s ago 4m 14.1M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (19s) 1s ago 11m 473M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283 running (2m) 2s ago 9m 69.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef a965183fccc3 2024-04-24T22:47:03.873 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 1s ago 11m 53.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (8m) 2s ago 8m 42.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (70s) 1s ago 10m 8795k - 1.5.0 0da6a335fe13 60870a70d2e7 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (32s) 2s ago 9m 8140k - 1.5.0 0da6a335fe13 ded879d61b71 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 1s ago 8m 202M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 1s ago 7m 202M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (7m) 1s ago 7m 171M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (6m) 2s ago 6m 228M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (6m) 2s ago 6m 204M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (6m) 2s ago 6m 183M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:47:03.874 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (42s) 1s ago 9m 44.5M - 2.43.0 a07b618ecd1d 9ac2bd9c2e9d 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 msgr2=0x7f9c6003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 0x7f9c6003b360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009b80 tx=0x7f9c6400b040 comp rx=0 tx=0).stop 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 msgr2=0x7f9c7410fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9c70013870 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 shutdown_connections 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f9c60038eb0 0x7f9c6003b360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c740fbcb0 0x7f9c7410fbd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 --2- 172.21.15.63:0/980879070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c740fcf60 0x7f9c74110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 >> 172.21.15.63:0/980879070 conn(0x7f9c740f7430 msgr2=0x7f9c74065ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 shutdown_connections 2024-04-24T22:47:03.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:03.873+0000 7f9c7c160700 1 -- 172.21.15.63:0/980879070 wait complete. 2024-04-24T22:47:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: pgmap v6: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:03.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:03.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:03 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- 172.21.15.63:0/3202220880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 msgr2=0x7f61d00ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 --2- 172.21.15.63:0/3202220880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d00ff9e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f61c0004940 tx=0x7f61c0009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- 172.21.15.63:0/3202220880 shutdown_connections 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 --2- 172.21.15.63:0/3202220880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d0102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 --2- 172.21.15.63:0/3202220880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d00ff9e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- 172.21.15.63:0/3202220880 >> 172.21.15.63:0/3202220880 conn(0x7f61d00f7430 msgr2=0x7f61d00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- 172.21.15.63:0/3202220880 shutdown_connections 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- 172.21.15.63:0/3202220880 wait complete. 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 Processor -- start 2024-04-24T22:47:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.000+0000 7f61d5040700 1 -- start start 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61d5040700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61d5040700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61d5040700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d010e540 con 0x7f61d0066280 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61d5040700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d010ed40 con 0x7f61d00fff20 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36464/0 (socket says 172.21.15.63:36464) 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 -- 172.21.15.63:0/988710694 learned_addr learned my addr 172.21.15.63:0/988710694 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ce59c700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 -- 172.21.15.63:0/988710694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 msgr2=0x7f61d010df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d010df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 -- 172.21.15.63:0/988710694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0005040 con 0x7f61d0066280 2024-04-24T22:47:04.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ce59c700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.001+0000 7f61ced9d700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000c00 tx=0x7f61c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.002+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c000e030 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.002+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d010efc0 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.002+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d010f500 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.002+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61c0013de0 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.002+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c001ce50 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.003+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d005f5c0 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.003+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f61c000b460 con 0x7f61d0066280 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.003+0000 7f61b77fe700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 0x7f61b803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.003+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f61c004e5f0 con 0x7f61d0066280 2024-04-24T22:47:04.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.004+0000 7f61ce59c700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 0x7f61b803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.004+0000 7f61ce59c700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 0x7f61b803b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009b80 tx=0x7f61c400b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:04.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.009+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f61c0023350 con 0x7f61d0066280 2024-04-24T22:47:04.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.377+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f61d010f830 con 0x7f61d0066280 2024-04-24T22:47:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.381+0000 7f61b77fe700 1 -- 172.21.15.63:0/988710694 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f61c0011c00 con 0x7f61d0066280 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:47:04.384 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:47:04.385 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:47:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 msgr2=0x7f61b803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 0x7f61b803b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009b80 tx=0x7f61c400b040 comp rx=0 tx=0).stop 2024-04-24T22:47:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 msgr2=0x7f61d010da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000c00 tx=0x7f61c0009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 shutdown_connections 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f61b8038f10 0x7f61b803b3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d0066280 0x7f61d010da00 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000c00 tx=0x7f61c0009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 --2- 172.21.15.63:0/988710694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61d00fff20 0x7f61d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 >> 172.21.15.63:0/988710694 conn(0x7f61d00f7430 msgr2=0x7f61d00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 shutdown_connections 2024-04-24T22:47:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.386+0000 7f61d5040700 1 -- 172.21.15.63:0/988710694 wait complete. 2024-04-24T22:47:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 -- 172.21.15.63:0/2040757565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 msgr2=0x7f801c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 --2- 172.21.15.63:0/2040757565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c0fc0c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8018004940 tx=0x7f8018009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 -- 172.21.15.63:0/2040757565 shutdown_connections 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 --2- 172.21.15.63:0/2040757565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 0x7f801c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 --2- 172.21.15.63:0/2040757565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c0fc0c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 -- 172.21.15.63:0/2040757565 >> 172.21.15.63:0/2040757565 conn(0x7f801c0f7430 msgr2=0x7f801c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 -- 172.21.15.63:0/2040757565 shutdown_connections 2024-04-24T22:47:04.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.530+0000 7f80240c3700 1 -- 172.21.15.63:0/2040757565 wait complete. 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 Processor -- start 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 -- start start 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 0x7f801c112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c066330 con 0x7f801c0fbcb0 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f80240c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c066470 con 0x7f801c0fcf60 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.531+0000 7f8021e5f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36482/0 (socket says 172.21.15.63:36482) 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 -- 172.21.15.63:0/2340105697 learned_addr learned my addr 172.21.15.63:0/2340105697 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f802165e700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 0x7f801c112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 -- 172.21.15.63:0/2340105697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 msgr2=0x7f801c112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 0x7f801c112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 -- 172.21.15.63:0/2340105697 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018005040 con 0x7f801c0fbcb0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f8021e5f700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f8018009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801800e070 con 0x7f801c0fbcb0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c113120 con 0x7f801c0fbcb0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8018013d80 con 0x7f801c0fbcb0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8018014670 con 0x7f801c0fbcb0 2024-04-24T22:47:04.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.532+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c113660 con 0x7f801c0fbcb0 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.533+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f801801c400 con 0x7f801c0fbcb0 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.534+0000 7f80127fc700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 0x7f800803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.534+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f801804e1c0 con 0x7f801c0fbcb0 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.534+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c05f5c0 con 0x7f801c0fbcb0 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.535+0000 7f802165e700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 0x7f800803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.535+0000 7f802165e700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 0x7f800803b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f800c009b80 tx=0x7f800c00b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:04.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.540+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8018011400 con 0x7f801c0fbcb0 2024-04-24T22:47:04.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.805+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f801c101200 con 0x7f8008038f10 2024-04-24T22:47:04.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.807+0000 7f80127fc700 1 -- 172.21.15.63:0/2340105697 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f801c101200 con 0x7f8008038f10 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:47:04.809 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T22:47:04.810 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-04-24T22:47:04.810 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:47:04.810 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 msgr2=0x7f800803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 0x7f800803b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f800c009b80 tx=0x7f800c00b040 comp rx=0 tx=0).stop 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 msgr2=0x7f801c110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f8018009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 shutdown_connections 2024-04-24T22:47:04.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8008038f10 0x7f800803b3c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801c0fbcb0 0x7f801c110690 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 --2- 172.21.15.63:0/2340105697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0fcf60 0x7f801c112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:04.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 >> 172.21.15.63:0/2340105697 conn(0x7f801c0f7430 msgr2=0x7f801c0f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:04.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 shutdown_connections 2024-04-24T22:47:04.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:04.809+0000 7f80240c3700 1 -- 172.21.15.63:0/2340105697 wait complete. 2024-04-24T22:47:05.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:04 smithi063 ceph-mon[26912]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:05.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:04 smithi063 ceph-mon[26912]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:05.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:04 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:05.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:04 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:05.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:04 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/988710694' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:05.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:04 smithi192 ceph-mon[33991]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:05.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:04 smithi192 ceph-mon[33991]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:05.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:04 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:05.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:04 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:05.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:04 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/988710694' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:06.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:05 smithi192 ceph-mon[33991]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:06.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:05 smithi192 ceph-mon[33991]: pgmap v7: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-04-24T22:47:06.246 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:05 smithi063 ceph-mon[26912]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:06.246 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:05 smithi063 ceph-mon[26912]: pgmap v7: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-04-24T22:47:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:06 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:47:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:06 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:47:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:06 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:47:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:06 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:47:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:06 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:47:07.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:06 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:47:07.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:06 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:47:07.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:06 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:47:07.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:06 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:47:07.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:06 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/1651825650' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:47:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:07 smithi192 ceph-mon[33991]: pgmap v8: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 821 KiB/s wr, 99 op/s 2024-04-24T22:47:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:07 smithi192 ceph-mon[33991]: mgrmap e32: smithi063.upcnvd(active, since 10s), standbys: smithi192.ctmwwc 2024-04-24T22:47:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:07 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:47:08.251 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:07 smithi063 ceph-mon[26912]: pgmap v8: 65 pgs: 65 active+clean; 294 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 821 KiB/s wr, 99 op/s 2024-04-24T22:47:08.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:07 smithi063 ceph-mon[26912]: mgrmap e32: smithi063.upcnvd(active, since 10s), standbys: smithi192.ctmwwc 2024-04-24T22:47:08.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:07 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:47:10.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: pgmap v9: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-24T22:47:10.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:10.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:10.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:10.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:10.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:47:10.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:10 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:47:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: pgmap v9: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-24T22:47:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:10.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:10.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:47:10.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:10 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:47:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:47:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:47:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:47:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:11 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:47:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:47:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:47:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:11.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:11 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:47:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: pgmap v10: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-24T22:47:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:47:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:47:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T22:47:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:12 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:47:12.719 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: pgmap v10: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-24T22:47:12.719 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-04-24T22:47:12.719 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-04-24T22:47:12.719 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T22:47:12.719 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:12 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:13.994 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-24T22:47:13.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T22:47:14.457 DEBUG:teuthology.parallel:result is None 2024-04-24T22:47:14.636 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:14 smithi063 ceph-mon[26912]: pgmap v11: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-24T22:47:14.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:14 smithi192 ceph-mon[33991]: pgmap v11: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-24T22:47:15.001 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-24T22:47:15.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-24T22:47:15.283 DEBUG:teuthology.parallel:result is None 2024-04-24T22:47:15.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:47:15.318 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:47:15.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T22:47:15.376 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T22:47:15.377 DEBUG:teuthology.parallel:result is None 2024-04-24T22:47:16.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:16 smithi063 ceph-mon[26912]: pgmap v12: 65 pgs: 65 active+clean; 274 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-04-24T22:47:16.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:16 smithi192 ceph-mon[33991]: pgmap v12: 65 pgs: 65 active+clean; 274 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-04-24T22:47:18.522 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: pgmap v13: 65 pgs: 65 active+clean; 274 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 207 op/s 2024-04-24T22:47:18.522 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.522 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.522 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:47:18.522 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:47:18.523 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.523 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: pgmap v13: 65 pgs: 65 active+clean; 274 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 207 op/s 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:18.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:19 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:47:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:19 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:47:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:19 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:47:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:19 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063:9095"}]: dispatch 2024-04-24T22:47:20.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:20 smithi192 ceph-mon[33991]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 281 op/s 2024-04-24T22:47:20.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:20 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:47:20.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:20 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:47:20.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:20 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:20.792 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:20 smithi063 ceph-mon[26912]: pgmap v14: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 281 op/s 2024-04-24T22:47:20.792 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:20 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:47:20.792 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:20 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:47:20.792 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:20 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:21.470 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:21 smithi192 ceph-mon[33991]: Upgrade: Updating mgr.smithi192.ctmwwc 2024-04-24T22:47:21.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:21 smithi192 ceph-mon[33991]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:47:21.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:21 smithi063 ceph-mon[26912]: Upgrade: Updating mgr.smithi192.ctmwwc 2024-04-24T22:47:21.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:21 smithi063 ceph-mon[26912]: Deploying daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:47:22.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:22 smithi063 ceph-mon[26912]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 965 KiB/s wr, 178 op/s 2024-04-24T22:47:22.641 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:22 smithi192 ceph-mon[33991]: pgmap v15: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 965 KiB/s wr, 178 op/s 2024-04-24T22:47:24.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:24 smithi063 ceph-mon[26912]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 962 KiB/s wr, 177 op/s 2024-04-24T22:47:24.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:24 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:24.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:24 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:24.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:24 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:24 smithi192 ceph-mon[33991]: pgmap v16: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 962 KiB/s wr, 177 op/s 2024-04-24T22:47:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:24 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:24 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:24 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:26.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:26 smithi192 ceph-mon[33991]: pgmap v17: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 991 KiB/s wr, 178 op/s 2024-04-24T22:47:26.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:26 smithi063 ceph-mon[26912]: pgmap v17: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 991 KiB/s wr, 178 op/s 2024-04-24T22:47:27.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:27 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:27.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:27 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:27.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:27 smithi192 ceph-mon[33991]: pgmap v18: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 364 KiB/s wr, 74 op/s 2024-04-24T22:47:27.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:27 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:27 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:27 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:27 smithi063 ceph-mon[26912]: pgmap v18: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 364 KiB/s wr, 74 op/s 2024-04-24T22:47:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:27 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:28.731 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.732 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.732 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.732 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:28.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:29.745 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:29 smithi192 ceph-mon[33991]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 364 KiB/s wr, 75 op/s 2024-04-24T22:47:29.883 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:29 smithi063 ceph-mon[26912]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 364 KiB/s wr, 75 op/s 2024-04-24T22:47:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:47:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:47:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:47:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:47:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:47:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:30 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:47:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:47:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:47:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:47:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:47:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:47:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:30 smithi063 ceph-mon[26912]: from='mgr.? 172.21.15.192:0/732094208' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: mgrmap e33: smithi063.upcnvd(active, since 33s), standbys: smithi192.ctmwwc 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 30 KiB/s wr, 1 op/s 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 198 B/s rd, 35 KiB/s wr, 2 op/s 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 246 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all mgr 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.upcnvd"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.upcnvd"}]': finished 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.ctmwwc"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.ctmwwc"}]': finished 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all rgw 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all iscsi 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all nfs 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all nvmeof 2024-04-24T22:47:31.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: mgrmap e33: smithi063.upcnvd(active, since 33s), standbys: smithi192.ctmwwc 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 30 KiB/s wr, 1 op/s 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:31.982 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 198 B/s rd, 35 KiB/s wr, 2 op/s 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 246 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all mgr 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.upcnvd"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.upcnvd"}]': finished 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.ctmwwc"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.ctmwwc"}]': finished 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.983 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all rgw 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all iscsi 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.984 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all nfs 2024-04-24T22:47:31.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:31.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:31.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all nvmeof 2024-04-24T22:47:31.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:34.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:34 smithi063 ceph-mon[26912]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 493 B/s wr, 0 op/s 2024-04-24T22:47:34.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:34 smithi063 ceph-mon[26912]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-04-24T22:47:34.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:34 smithi063 ceph-mon[26912]: Deploying daemon node-exporter.smithi063 on smithi063 2024-04-24T22:47:34.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:34 smithi192 ceph-mon[33991]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 493 B/s wr, 0 op/s 2024-04-24T22:47:34.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:34 smithi192 ceph-mon[33991]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-04-24T22:47:34.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:34 smithi192 ceph-mon[33991]: Deploying daemon node-exporter.smithi063 on smithi063 2024-04-24T22:47:34.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 -- 172.21.15.63:0/4172991280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fcf90 msgr2=0x7feb600fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 --2- 172.21.15.63:0/4172991280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fcf90 0x7feb600fd400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feb54004940 tx=0x7feb54009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 -- 172.21.15.63:0/4172991280 shutdown_connections 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 --2- 172.21.15.63:0/4172991280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fcf90 0x7feb600fd400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 --2- 172.21.15.63:0/4172991280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fbce0 0x7feb600fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 -- 172.21.15.63:0/4172991280 >> 172.21.15.63:0/4172991280 conn(0x7feb600f7440 msgr2=0x7feb600f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 -- 172.21.15.63:0/4172991280 shutdown_connections 2024-04-24T22:47:34.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.918+0000 7feb67b57700 1 -- 172.21.15.63:0/4172991280 wait complete. 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 Processor -- start 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 -- start start 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fcf90 0x7feb60112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60112c60 con 0x7feb600fbce0 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb67b57700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60112dd0 con 0x7feb600fcf90 2024-04-24T22:47:34.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb658f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb658f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47366/0 (socket says 172.21.15.63:47366) 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.919+0000 7feb658f3700 1 -- 172.21.15.63:0/708252078 learned_addr learned my addr 172.21.15.63:0/708252078 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb650f2700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fcf90 0x7feb60112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb658f3700 1 -- 172.21.15.63:0/708252078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fcf90 msgr2=0x7feb60112720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb658f3700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fcf90 0x7feb60112720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:34.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb658f3700 1 -- 172.21.15.63:0/708252078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54005040 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb658f3700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb5000bbb0 tx=0x7feb50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb500092e0 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb5000cb20 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb601130b0 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb50015d70 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.920+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb60113670 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.921+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6004c710 con 0x7feb600fbce0 2024-04-24T22:47:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.922+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7feb50012070 con 0x7feb600fbce0 2024-04-24T22:47:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.922+0000 7feb5e7fc700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 0x7feb48070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.922+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7feb5008d0d0 con 0x7feb600fbce0 2024-04-24T22:47:34.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.922+0000 7feb650f2700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 0x7feb48070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.923+0000 7feb650f2700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 0x7feb48070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b970 tx=0x7feb54009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:34.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:34.927+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7feb5005a6c0 con 0x7feb600fbce0 2024-04-24T22:47:35.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.158+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb60101230 con 0x7feb4806db70 2024-04-24T22:47:35.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.159+0000 7feb5e7fc700 1 -- 172.21.15.63:0/708252078 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7feb60101230 con 0x7feb4806db70 2024-04-24T22:47:35.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 msgr2=0x7feb48070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 0x7feb48070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b970 tx=0x7feb54009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 msgr2=0x7feb60066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb5000bbb0 tx=0x7feb50009840 comp rx=0 tx=0).stop 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 shutdown_connections 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feb4806db70 0x7feb48070020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb600fbce0 0x7feb60066cb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 --2- 172.21.15.63:0/708252078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600fcf90 0x7feb60112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 >> 172.21.15.63:0/708252078 conn(0x7feb600f7440 msgr2=0x7feb600f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 shutdown_connections 2024-04-24T22:47:35.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.161+0000 7feb67b57700 1 -- 172.21.15.63:0/708252078 wait complete. 2024-04-24T22:47:35.173 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:47:35.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 -- 172.21.15.63:0/2522535426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fbcc0 msgr2=0x7ff9900fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 --2- 172.21.15.63:0/2522535426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fbcc0 0x7ff9900fc0d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff984004940 tx=0x7ff984009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 -- 172.21.15.63:0/2522535426 shutdown_connections 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 --2- 172.21.15.63:0/2522535426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fcf70 0x7ff9900fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 --2- 172.21.15.63:0/2522535426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fbcc0 0x7ff9900fc0d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 -- 172.21.15.63:0/2522535426 >> 172.21.15.63:0/2522535426 conn(0x7ff9900f7440 msgr2=0x7ff9900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 -- 172.21.15.63:0/2522535426 shutdown_connections 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.270+0000 7ff996de8700 1 -- 172.21.15.63:0/2522535426 wait complete. 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 Processor -- start 2024-04-24T22:47:35.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 -- start start 2024-04-24T22:47:35.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fbcc0 0x7ff99010fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901106c0 con 0x7ff9900fcf70 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff996de8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff990110830 con 0x7ff9900fbcc0 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47380/0 (socket says 172.21.15.63:47380) 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 -- 172.21.15.63:0/3768967323 learned_addr learned my addr 172.21.15.63:0/3768967323 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff994b84700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fbcc0 0x7ff99010fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 -- 172.21.15.63:0/3768967323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fbcc0 msgr2=0x7ff99010fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fbcc0 0x7ff99010fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.271+0000 7ff98ffff700 1 -- 172.21.15.63:0/3768967323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984005040 con 0x7ff9900fcf70 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff98ffff700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c830 tx=0x7ff97c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97c0106f0 con 0x7ff9900fcf70 2024-04-24T22:47:35.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff990115290 con 0x7ff9900fcf70 2024-04-24T22:47:35.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff97c010d20 con 0x7ff9900fcf70 2024-04-24T22:47:35.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97c0113a0 con 0x7ff9900fcf70 2024-04-24T22:47:35.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.272+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff990115850 con 0x7ff9900fcf70 2024-04-24T22:47:35.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.273+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990101900 con 0x7ff9900fcf70 2024-04-24T22:47:35.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.273+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7ff97c012400 con 0x7ff9900fcf70 2024-04-24T22:47:35.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.274+0000 7ff98d7fa700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 0x7ff980070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.274+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff97c08d320 con 0x7ff9900fcf70 2024-04-24T22:47:35.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.274+0000 7ff994b84700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 0x7ff980070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.274+0000 7ff994b84700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 0x7ff980070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b840 tx=0x7ff984009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.279+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff97c05a8f0 con 0x7ff9900fcf70 2024-04-24T22:47:35.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.505+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff99002b340 con 0x7ff98006db70 2024-04-24T22:47:35.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.506+0000 7ff98d7fa700 1 -- 172.21.15.63:0/3768967323 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ff99002b340 con 0x7ff98006db70 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 msgr2=0x7ff980070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 0x7ff980070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b840 tx=0x7ff984009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 msgr2=0x7ff9901100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c830 tx=0x7ff97c004470 comp rx=0 tx=0).stop 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 shutdown_connections 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff98006db70 0x7ff980070020 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9900fbcc0 0x7ff99010fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 --2- 172.21.15.63:0/3768967323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9900fcf70 0x7ff9901100c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 >> 172.21.15.63:0/3768967323 conn(0x7ff9900f7440 msgr2=0x7ff990066870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 shutdown_connections 2024-04-24T22:47:35.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.509+0000 7ff996de8700 1 -- 172.21.15.63:0/3768967323 wait complete. 2024-04-24T22:47:35.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 -- 172.21.15.63:0/1600627958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 msgr2=0x7f07880702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 --2- 172.21.15.63:0/1600627958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f07880702d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0778004940 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 -- 172.21.15.63:0/1600627958 shutdown_connections 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 --2- 172.21.15.63:0/1600627958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 0x7f0788070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 --2- 172.21.15.63:0/1600627958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f07880702d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 -- 172.21.15.63:0/1600627958 >> 172.21.15.63:0/1600627958 conn(0x7f07880f8560 msgr2=0x7f07880fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 -- 172.21.15.63:0/1600627958 shutdown_connections 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.618+0000 7f078f9dc700 1 -- 172.21.15.63:0/1600627958 wait complete. 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 Processor -- start 2024-04-24T22:47:35.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 -- start start 2024-04-24T22:47:35.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 0x7f078810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788110760 con 0x7f0788071dc0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078f9dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788110fb0 con 0x7f0788070810 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47398/0 (socket says 172.21.15.63:47398) 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 -- 172.21.15.63:0/4253454840 learned_addr learned my addr 172.21.15.63:0/4253454840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078d778700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 0x7f078810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 -- 172.21.15.63:0/4253454840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 msgr2=0x7f078810fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 0x7f078810fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.619+0000 7f078cf77700 1 -- 172.21.15.63:0/4253454840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778005040 con 0x7f0788071dc0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f078cf77700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f078400c8c0 tx=0x7f07840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078400a760 con 0x7f0788071dc0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f078400ad90 con 0x7f0788071dc0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0788111290 con 0x7f0788071dc0 2024-04-24T22:47:35.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07840113a0 con 0x7f0788071dc0 2024-04-24T22:47:35.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.620+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881117d0 con 0x7f0788071dc0 2024-04-24T22:47:35.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.621+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078805f5c0 con 0x7f0788071dc0 2024-04-24T22:47:35.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.621+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f0784012400 con 0x7f0788071dc0 2024-04-24T22:47:35.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.622+0000 7f077dffb700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 0x7f0774070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.622+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f078408d340 con 0x7f0788071dc0 2024-04-24T22:47:35.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.622+0000 7f078d778700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 0x7f0774070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.623+0000 7f078d778700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 0x7f0774070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f077800b970 tx=0x7f0778009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.627+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f078405a990 con 0x7f0788071dc0 2024-04-24T22:47:35.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.859+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0788111b40 con 0x7f077406db60 2024-04-24T22:47:35.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.870+0000 7f077dffb700 1 -- 172.21.15.63:0/4253454840 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0788111b40 con 0x7f077406db60 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (97s) 7s ago 10m 17.1M - 0.25.0 c8568f914cd2 f156d383c8ab 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (10m) 7s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (9m) 9s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 unknown 7s ago 10m - - 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (5m) 7s ago 5m 14.3M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (5m) 7s ago 5m 12.6M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (5m) 9s ago 5m 159M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (4m) 9s ago 4m 14.3M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (51s) 7s ago 12m 492M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (12s) 9s ago 9m 65.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (12m) 7s ago 12m 54.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:47:35.872 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (9m) 9s ago 9m 42.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (102s) 7s ago 10m 8514k - 1.5.0 0da6a335fe13 60870a70d2e7 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (64s) 9s ago 9m 8506k - 1.5.0 0da6a335fe13 ded879d61b71 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 7s ago 8m 212M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (8m) 7s ago 8m 215M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (7m) 7s ago 7m 179M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (7m) 9s ago 7m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (7m) 9s ago 7m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (6m) 9s ago 6m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:47:35.873 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (18s) 7s ago 10m 39.0M - 2.43.0 a07b618ecd1d 7cc101c093f1 2024-04-24T22:47:35.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 msgr2=0x7f0774070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 0x7f0774070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f077800b970 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 msgr2=0x7f0788110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f078400c8c0 tx=0x7f07840044c0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 shutdown_connections 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f077406db60 0x7f0774070010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0788070810 0x7f078810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 --2- 172.21.15.63:0/4253454840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0788071dc0 0x7f0788110160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.872+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 >> 172.21.15.63:0/4253454840 conn(0x7f07880f8560 msgr2=0x7f07880f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.873+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 shutdown_connections 2024-04-24T22:47:35.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.873+0000 7f078f9dc700 1 -- 172.21.15.63:0/4253454840 wait complete. 2024-04-24T22:47:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 -- 172.21.15.63:0/1906607059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 msgr2=0x7f1588102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 --2- 172.21.15.63:0/1906607059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f1588102340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1584004940 tx=0x7f1584009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 -- 172.21.15.63:0/1906607059 shutdown_connections 2024-04-24T22:47:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 --2- 172.21.15.63:0/1906607059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f1588102340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 --2- 172.21.15.63:0/1906607059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 0x7f15880ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 -- 172.21.15.63:0/1906607059 >> 172.21.15.63:0/1906607059 conn(0x7f15880f7430 msgr2=0x7f15880f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 -- 172.21.15.63:0/1906607059 shutdown_connections 2024-04-24T22:47:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.978+0000 7f158fa50700 1 -- 172.21.15.63:0/1906607059 wait complete. 2024-04-24T22:47:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 Processor -- start 2024-04-24T22:47:35.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 -- start start 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 0x7f158810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810e560 con 0x7f15880fff60 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158fa50700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810edb0 con 0x7f1588066240 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158cfeb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158d7ec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 0x7f158810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158cfeb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47410/0 (socket says 172.21.15.63:47410) 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.979+0000 7f158cfeb700 1 -- 172.21.15.63:0/1152222077 learned_addr learned my addr 172.21.15.63:0/1152222077 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158cfeb700 1 -- 172.21.15.63:0/1152222077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 msgr2=0x7f158810da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158cfeb700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 0x7f158810da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158cfeb700 1 -- 172.21.15.63:0/1152222077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584005040 con 0x7f15880fff60 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158cfeb700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f158400b3b0 tx=0x7f1584009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158400e040 con 0x7f15880fff60 2024-04-24T22:47:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f158400bbf0 con 0x7f15880fff60 2024-04-24T22:47:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158810f030 con 0x7f15880fff60 2024-04-24T22:47:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158401c630 con 0x7f15880fff60 2024-04-24T22:47:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.980+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158810f570 con 0x7f15880fff60 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.981+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158805f5c0 con 0x7f15880fff60 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.982+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f158400bd60 con 0x7f15880fff60 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.982+0000 7f157dffb700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 0x7f1574070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.982+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f158408e000 con 0x7f15880fff60 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.982+0000 7f158d7ec700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 0x7f1574070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.983+0000 7f158d7ec700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 0x7f1574070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1588065f80 tx=0x7f157800b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:35.988+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f158405b560 con 0x7f15880fff60 2024-04-24T22:47:36.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.302+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f158810f790 con 0x7f15880fff60 2024-04-24T22:47:36.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.303+0000 7f157dffb700 1 -- 172.21.15.63:0/1152222077 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f158405b0f0 con 0x7f15880fff60 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:36.305 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:47:36.306 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 msgr2=0x7f1574070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 0x7f1574070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1588065f80 tx=0x7f157800b040 comp rx=0 tx=0).stop 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 msgr2=0x7f158810df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f158400b3b0 tx=0x7f1584009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 shutdown_connections 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f157406db60 0x7f1574070010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588066240 0x7f158810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 --2- 172.21.15.63:0/1152222077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880fff60 0x7f158810df60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 >> 172.21.15.63:0/1152222077 conn(0x7f15880f7430 msgr2=0x7f15880f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 shutdown_connections 2024-04-24T22:47:36.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.305+0000 7f158fa50700 1 -- 172.21.15.63:0/1152222077 wait complete. 2024-04-24T22:47:36.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 -- 172.21.15.63:0/474830925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 msgr2=0x7f2558102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 --2- 172.21.15.63:0/474830925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f2558102330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2548004940 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:36.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 -- 172.21.15.63:0/474830925 shutdown_connections 2024-04-24T22:47:36.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 --2- 172.21.15.63:0/474830925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f2558102330 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 --2- 172.21.15.63:0/474830925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 0x7f25580ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.414+0000 7f2560220700 1 -- 172.21.15.63:0/474830925 >> 172.21.15.63:0/474830925 conn(0x7f25580f7440 msgr2=0x7f25580f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 -- 172.21.15.63:0/474830925 shutdown_connections 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 -- 172.21.15.63:0/474830925 wait complete. 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 Processor -- start 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 -- start start 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 0x7f255810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255810e460 con 0x7f25580fff50 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f2560220700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255810e5d0 con 0x7f2558066ed0 2024-04-24T22:47:36.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f255d7bb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f255d7bb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47432/0 (socket says 172.21.15.63:47432) 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.415+0000 7f255d7bb700 1 -- 172.21.15.63:0/1766181915 learned_addr learned my addr 172.21.15.63:0/1766181915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f255dfbc700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 0x7f255810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f255d7bb700 1 -- 172.21.15.63:0/1766181915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 msgr2=0x7f255810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f255d7bb700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 0x7f255810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f255d7bb700 1 -- 172.21.15.63:0/1766181915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548005040 con 0x7f25580fff50 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f255d7bb700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2548009150 tx=0x7f2548009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254800e040 con 0x7f25580fff50 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2548004290 con 0x7f25580fff50 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2558112f80 con 0x7f25580fff50 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254801c630 con 0x7f25580fff50 2024-04-24T22:47:36.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.416+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2558113510 con 0x7f25580fff50 2024-04-24T22:47:36.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.417+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255804c710 con 0x7f25580fff50 2024-04-24T22:47:36.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.418+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f254800bbf0 con 0x7f25580fff50 2024-04-24T22:47:36.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.418+0000 7f254e7fc700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 0x7f2544070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:47:36.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.418+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f254808def0 con 0x7f25580fff50 2024-04-24T22:47:36.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.418+0000 7f255dfbc700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 0x7f2544070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:47:36.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.419+0000 7f255dfbc700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 0x7f2544070020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2554009fd0 tx=0x7f255400b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:47:36.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.423+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f254805b4f0 con 0x7f25580fff50 2024-04-24T22:47:36.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:36 smithi063 ceph-mon[26912]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:36.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:36 smithi063 ceph-mon[26912]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 493 B/s wr, 0 op/s 2024-04-24T22:47:36.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.656+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25580f8f30 con 0x7f254406db70 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.657+0000 7f254e7fc700 1 -- 172.21.15.63:0/1766181915 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f25580f8f30 con 0x7f254406db70 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:47:36.659 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:47:36.660 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:47:36.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 msgr2=0x7f2544070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 0x7f2544070020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2554009fd0 tx=0x7f255400b040 comp rx=0 tx=0).stop 2024-04-24T22:47:36.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 msgr2=0x7f255810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:47:36.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2548009150 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-24T22:47:36.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 shutdown_connections 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f254406db70 0x7f2544070020 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558066ed0 0x7f255810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 --2- 172.21.15.63:0/1766181915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25580fff50 0x7f255810de60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 >> 172.21.15.63:0/1766181915 conn(0x7f25580f7440 msgr2=0x7f25580f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 shutdown_connections 2024-04-24T22:47:36.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:47:36.659+0000 7f2560220700 1 -- 172.21.15.63:0/1766181915 wait complete. 2024-04-24T22:47:36.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:36 smithi192 ceph-mon[33991]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:36.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:36 smithi192 ceph-mon[33991]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 493 B/s wr, 0 op/s 2024-04-24T22:47:37.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:37 smithi192 ceph-mon[33991]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:37.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:37 smithi192 ceph-mon[33991]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:37.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:37 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/1152222077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:37.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:37 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:37.471 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:37 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:37 smithi063 ceph-mon[26912]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:37 smithi063 ceph-mon[26912]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:37 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/1152222077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:37 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:37.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:37 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:38.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:38 smithi063 ceph-mon[26912]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:38.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:38 smithi063 ceph-mon[26912]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:38 smithi192 ceph-mon[33991]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:47:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:38 smithi192 ceph-mon[33991]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:39.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:39 smithi063 ceph-mon[26912]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-04-24T22:47:39.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:39 smithi063 ceph-mon[26912]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-24T22:47:39.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:39 smithi192 ceph-mon[33991]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-04-24T22:47:39.572 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:39 smithi192 ceph-mon[33991]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-24T22:47:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:40 smithi063 ceph-mon[26912]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:40.570 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:40 smithi192 ceph-mon[33991]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:41.891 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:41.891 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:41.891 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:41.891 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:41 smithi192 ceph-mon[33991]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:42.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:42.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:41 smithi063 ceph-mon[26912]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:43.094 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:42 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:43.192 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:42 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:44.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:43 smithi063 ceph-mon[26912]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:44.408 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:43 smithi192 ceph-mon[33991]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:44 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:44 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:44 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:44 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:45 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:45 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:45 smithi063 ceph-mon[26912]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:45 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:45 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:45 smithi192 ceph-mon[33991]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:47.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:47.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:47.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:47.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:48 smithi063 ceph-mon[26912]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:48 smithi192 ceph-mon[33991]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:50 smithi192 ceph-mon[33991]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:50.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:50 smithi063 ceph-mon[26912]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:52.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:52 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:52.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:47:54.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:54 smithi063 ceph-mon[26912]: Upgrade: Updating prometheus.smithi063 2024-04-24T22:47:54.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:54 smithi063 ceph-mon[26912]: Deploying daemon prometheus.smithi063 on smithi063 2024-04-24T22:47:54.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:54 smithi063 ceph-mon[26912]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:54.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:54 smithi192 ceph-mon[33991]: Upgrade: Updating prometheus.smithi063 2024-04-24T22:47:54.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:54 smithi192 ceph-mon[33991]: Deploying daemon prometheus.smithi063 on smithi063 2024-04-24T22:47:54.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:54 smithi192 ceph-mon[33991]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:56.393 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:56 smithi063 ceph-mon[26912]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:56.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:56 smithi192 ceph-mon[33991]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:57.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:47:58.465 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:47:58 smithi063 ceph-mon[26912]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:47:58.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:47:58 smithi192 ceph-mon[33991]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:00.598 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:00 smithi063 ceph-mon[26912]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:00.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:00 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:00.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:00 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:00.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:00 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:00.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:00 smithi192 ceph-mon[33991]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:00.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:00 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:00.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:00 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:00.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:00 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:01.533 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:01 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:01.533 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:01 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:01.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:01 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:01.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:01 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:02.671 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:02 smithi063 ceph-mon[26912]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:02.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:02 smithi192 ceph-mon[33991]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:04.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:04 smithi192 ceph-mon[33991]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:04.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:04 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:04.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:04 smithi063 ceph-mon[26912]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:04.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:04 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:05 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:05 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:06.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:06 smithi063 ceph-mon[26912]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:06.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:06 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:06.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:06 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:06.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:06 smithi192 ceph-mon[33991]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:06.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:06 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:06.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:06 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:06.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 -- 172.21.15.63:0/4102301368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 msgr2=0x7f3b24102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:06.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/4102301368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b24102300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c004940 tx=0x7f3b0c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:06.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 -- 172.21.15.63:0/4102301368 shutdown_connections 2024-04-24T22:48:06.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/4102301368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b24102300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:06.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/4102301368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 0x7f3b240ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 -- 172.21.15.63:0/4102301368 >> 172.21.15.63:0/4102301368 conn(0x7f3b240f7430 msgr2=0x7f3b240f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 -- 172.21.15.63:0/4102301368 shutdown_connections 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.767+0000 7f3b2a69e700 1 -- 172.21.15.63:0/4102301368 wait complete. 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 Processor -- start 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 -- start start 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 0x7f3b2410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2410e580 con 0x7f3b240ffee0 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b2a69e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2410edd0 con 0x7f3b240662c0 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46874/0 (socket says 172.21.15.63:46874) 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b237fe700 1 -- 172.21.15.63:0/2871715596 learned_addr learned my addr 172.21.15.63:0/2871715596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.768+0000 7f3b23fff700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 0x7f3b2410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:06.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b237fe700 1 -- 172.21.15.63:0/2871715596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 msgr2=0x7f3b2410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b237fe700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 0x7f3b2410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b237fe700 1 -- 172.21.15.63:0/2871715596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c005040 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b237fe700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009150 tx=0x7f3b0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c00e040 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b0c004290 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2410f050 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c01c630 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.769+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2410f590 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.770+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2405f5c0 con 0x7f3b240ffee0 2024-04-24T22:48:06.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.771+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f3b0c00bbf0 con 0x7f3b240ffee0 2024-04-24T22:48:06.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.771+0000 7f3b20ff9700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 0x7f3b10070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:06.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.771+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3b0c08dfe0 con 0x7f3b240ffee0 2024-04-24T22:48:06.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.771+0000 7f3b23fff700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 0x7f3b10070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:06.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.772+0000 7f3b23fff700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 0x7f3b10070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007fd0 tx=0x7f3b14009040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:06.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:06.776+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3b0c05b540 con 0x7f3b240ffee0 2024-04-24T22:48:07.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.002+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b2410f870 con 0x7f3b1006db60 2024-04-24T22:48:07.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.003+0000 7f3b20ff9700 1 -- 172.21.15.63:0/2871715596 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f3b2410f870 con 0x7f3b1006db60 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 msgr2=0x7f3b10070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 0x7f3b10070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007fd0 tx=0x7f3b14009040 comp rx=0 tx=0).stop 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 msgr2=0x7f3b2410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009150 tx=0x7f3b0c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 shutdown_connections 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f3b1006db60 0x7f3b10070010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b240662c0 0x7f3b2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 --2- 172.21.15.63:0/2871715596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b240ffee0 0x7f3b2410df80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 >> 172.21.15.63:0/2871715596 conn(0x7f3b240f7430 msgr2=0x7f3b240f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 shutdown_connections 2024-04-24T22:48:07.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.005+0000 7f3b2a69e700 1 -- 172.21.15.63:0/2871715596 wait complete. 2024-04-24T22:48:07.018 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- 172.21.15.63:0/2232267366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fbcb0 msgr2=0x7f197c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 --2- 172.21.15.63:0/2232267366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fbcb0 0x7f197c0fc0c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1974004940 tx=0x7f1974009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- 172.21.15.63:0/2232267366 shutdown_connections 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 --2- 172.21.15.63:0/2232267366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fcf60 0x7f197c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 --2- 172.21.15.63:0/2232267366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fbcb0 0x7f197c0fc0c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- 172.21.15.63:0/2232267366 >> 172.21.15.63:0/2232267366 conn(0x7f197c0f7430 msgr2=0x7f197c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- 172.21.15.63:0/2232267366 shutdown_connections 2024-04-24T22:48:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- 172.21.15.63:0/2232267366 wait complete. 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 Processor -- start 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.115+0000 7f198339d700 1 -- start start 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f198339d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fbcb0 0x7f197c10da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f198339d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f198339d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c10e590 con 0x7f197c0fcf60 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f198339d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c10ed90 con 0x7f197c0fbcb0 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46896/0 (socket says 172.21.15.63:46896) 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 -- 172.21.15.63:0/3770626824 learned_addr learned my addr 172.21.15.63:0/3770626824 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1981139700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fbcb0 0x7f197c10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 -- 172.21.15.63:0/3770626824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fbcb0 msgr2=0x7f197c10da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fbcb0 0x7f197c10da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 -- 172.21.15.63:0/3770626824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974005040 con 0x7f197c0fcf60 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1980938700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f197000c830 tx=0x7f1970004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.116+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19700106f0 con 0x7f197c0fcf60 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.117+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1970010d20 con 0x7f197c0fcf60 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.117+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c10f070 con 0x7f197c0fcf60 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.117+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19700113a0 con 0x7f197c0fcf60 2024-04-24T22:48:07.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.117+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c10f5b0 con 0x7f197c0fcf60 2024-04-24T22:48:07.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.118+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c101af0 con 0x7f197c0fcf60 2024-04-24T22:48:07.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.118+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f1970012400 con 0x7f197c0fcf60 2024-04-24T22:48:07.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.118+0000 7f1979ffb700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 0x7f1964070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.119+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f197008d1c0 con 0x7f197c0fcf60 2024-04-24T22:48:07.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.119+0000 7f1981139700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 0x7f1964070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.119+0000 7f1981139700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 0x7f1964070010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f197400b970 tx=0x7f1974009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.123+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f197005a8d0 con 0x7f197c0fcf60 2024-04-24T22:48:07.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.354+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f197c10f880 con 0x7f196406db60 2024-04-24T22:48:07.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.356+0000 7f1979ffb700 1 -- 172.21.15.63:0/3770626824 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f197c10f880 con 0x7f196406db60 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 msgr2=0x7f1964070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 0x7f1964070010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f197400b970 tx=0x7f1974009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 msgr2=0x7f197c10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f197000c830 tx=0x7f1970004470 comp rx=0 tx=0).stop 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 shutdown_connections 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f196406db60 0x7f1964070010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c0fbcb0 0x7f197c10da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 --2- 172.21.15.63:0/3770626824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f197c0fcf60 0x7f197c10df90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 >> 172.21.15.63:0/3770626824 conn(0x7f197c0f7430 msgr2=0x7f197c065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 shutdown_connections 2024-04-24T22:48:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.358+0000 7f198339d700 1 -- 172.21.15.63:0/3770626824 wait complete. 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/2310754042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fbcb0 msgr2=0x7f7a680fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/2310754042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fbcb0 0x7f7a680fc0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004940 tx=0x7f7a58009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/2310754042 shutdown_connections 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/2310754042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fcf60 0x7f7a680fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/2310754042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fbcb0 0x7f7a680fc0c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.462+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/2310754042 >> 172.21.15.63:0/2310754042 conn(0x7f7a680f7430 msgr2=0x7f7a680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/2310754042 shutdown_connections 2024-04-24T22:48:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/2310754042 wait complete. 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 Processor -- start 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 -- start start 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fbcb0 0x7f7a6810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68110690 con 0x7f7a680fcf60 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6ccc3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68110800 con 0x7f7a680fbcb0 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a65d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a65d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46922/0 (socket says 172.21.15.63:46922) 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a65d9b700 1 -- 172.21.15.63:0/3362901383 learned_addr learned my addr 172.21.15.63:0/3362901383 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.463+0000 7f7a6659c700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fbcb0 0x7f7a6810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a65d9b700 1 -- 172.21.15.63:0/3362901383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fbcb0 msgr2=0x7f7a6810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a65d9b700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fbcb0 0x7f7a6810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a65d9b700 1 -- 172.21.15.63:0/3362901383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a58005040 con 0x7f7a680fcf60 2024-04-24T22:48:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a65d9b700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c002a50 tx=0x7f7a5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5c007cc0 con 0x7f7a680fcf60 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a5c00f040 con 0x7f7a680fcf60 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68115260 con 0x7f7a680fcf60 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5c00da70 con 0x7f7a680fcf60 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.464+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68115820 con 0x7f7a680fcf60 2024-04-24T22:48:07.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.465+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a681018f0 con 0x7f7a680fcf60 2024-04-24T22:48:07.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.466+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f7a5c00dc10 con 0x7f7a680fcf60 2024-04-24T22:48:07.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.466+0000 7f7a56ffd700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 0x7f7a50070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.466+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7a5c08caa0 con 0x7f7a680fcf60 2024-04-24T22:48:07.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.467+0000 7f7a6659c700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 0x7f7a50070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.467+0000 7f7a6659c700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 0x7f7a50070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800b970 tx=0x7f7a58009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.472+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7a5c059f80 con 0x7f7a680fcf60 2024-04-24T22:48:07.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.690+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a6802b390 con 0x7f7a5006db70 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.700+0000 7f7a56ffd700 1 -- 172.21.15.63:0/3362901383 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7a6802b390 con 0x7f7a5006db70 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 3s ago 11m 19.3M - 0.25.0 c8568f914cd2 f156d383c8ab 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (11m) 3s ago 11m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (10m) 23s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 unknown 3s ago 10m - - 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (5m) 3s ago 5m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (5m) 3s ago 5m 12.8M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (5m) 23s ago 5m 151M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (5m) 23s ago 5m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:48:07.703 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (83s) 3s ago 12m 501M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (44s) 23s ago 10m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (12m) 3s ago 12m 50.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (10m) 23s ago 10m 43.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (31s) 3s ago 11m 8190k - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (27s) 23s ago 10m 3586k - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (9m) 3s ago 9m 213M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (8m) 3s ago 8m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (8m) 3s ago 8m 180M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (7m) 23s ago 7m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (7m) 23s ago 7m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (7m) 23s ago 7m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:48:07.704 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (8s) 3s ago 10m 38.6M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 msgr2=0x7f7a50070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 0x7f7a50070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800b970 tx=0x7f7a58009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 msgr2=0x7f7a68110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c002a50 tx=0x7f7a5c0046b0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 shutdown_connections 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7a5006db70 0x7f7a50070020 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a680fbcb0 0x7f7a6810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 --2- 172.21.15.63:0/3362901383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a680fcf60 0x7f7a68110090 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 >> 172.21.15.63:0/3362901383 conn(0x7f7a680f7430 msgr2=0x7f7a680667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 shutdown_connections 2024-04-24T22:48:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.702+0000 7f7a6ccc3700 1 -- 172.21.15.63:0/3362901383 wait complete. 2024-04-24T22:48:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 -- 172.21.15.63:0/1289737411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0066260 msgr2=0x7f7cd0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/1289737411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0066260 0x7f7cd0101e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4004940 tx=0x7f7cc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 -- 172.21.15.63:0/1289737411 shutdown_connections 2024-04-24T22:48:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/1289737411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0102340 0x7f7cd0104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/1289737411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0066260 0x7f7cd0101e00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 -- 172.21.15.63:0/1289737411 >> 172.21.15.63:0/1289737411 conn(0x7f7cd00f7430 msgr2=0x7f7cd00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 -- 172.21.15.63:0/1289737411 shutdown_connections 2024-04-24T22:48:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 -- 172.21.15.63:0/1289737411 wait complete. 2024-04-24T22:48:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.807+0000 7f7cd5d31700 1 Processor -- start 2024-04-24T22:48:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cd5d31700 1 -- start start 2024-04-24T22:48:07.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cd5d31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0066260 0x7f7cd010fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cd5d31700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cd5d31700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd0110780 con 0x7f7cd0102340 2024-04-24T22:48:07.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cd5d31700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd0110fd0 con 0x7f7cd0066260 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46936/0 (socket says 172.21.15.63:46936) 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 -- 172.21.15.63:0/3691152936 learned_addr learned my addr 172.21.15.63:0/3691152936 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 -- 172.21.15.63:0/3691152936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0066260 msgr2=0x7f7cd010fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0066260 0x7f7cd010fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 -- 172.21.15.63:0/3691152936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4005040 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cceffd700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0002a50 tx=0x7f7cc00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.808+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc0007cc0 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.809+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd01112b0 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.809+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc000f040 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.809+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc0009640 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.809+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd01117f0 con 0x7f7cd0102340 2024-04-24T22:48:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.810+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd005f5c0 con 0x7f7cd0102340 2024-04-24T22:48:07.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.811+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f7cc00097c0 con 0x7f7cd0102340 2024-04-24T22:48:07.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.811+0000 7f7cb7fff700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 0x7f7cb8070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:07.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.811+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7cc008d1e0 con 0x7f7cd0102340 2024-04-24T22:48:07.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.811+0000 7f7ccf7fe700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 0x7f7cb8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:07.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.812+0000 7f7ccf7fe700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 0x7f7cb8070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4004f90 tx=0x7f7cc4009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:07.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:07.816+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7cc005a6c0 con 0x7f7cd0102340 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.110+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7cd0107a90 con 0x7f7cd0102340 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.111+0000 7f7cb7fff700 1 -- 172.21.15.63:0/3691152936 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f7cc005a250 con 0x7f7cd0102340 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:48:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:48:08.115 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:48:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 msgr2=0x7f7cb8070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 0x7f7cb8070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4004f90 tx=0x7f7cc4009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 msgr2=0x7f7cd0110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0002a50 tx=0x7f7cc00046b0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 shutdown_connections 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7cb806db60 0x7f7cb8070010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0066260 0x7f7cd010fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 --2- 172.21.15.63:0/3691152936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd0102340 0x7f7cd0110180 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 >> 172.21.15.63:0/3691152936 conn(0x7f7cd00f7430 msgr2=0x7f7cd00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 shutdown_connections 2024-04-24T22:48:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.113+0000 7f7cd5d31700 1 -- 172.21.15.63:0/3691152936 wait complete. 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 -- 172.21.15.63:0/2387454744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 msgr2=0x7f95ac0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 --2- 172.21.15.63:0/2387454744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac0fd3d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f959c004940 tx=0x7f959c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 -- 172.21.15.63:0/2387454744 shutdown_connections 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 --2- 172.21.15.63:0/2387454744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac0fd3d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 --2- 172.21.15.63:0/2387454744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 0x7f95ac0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 -- 172.21.15.63:0/2387454744 >> 172.21.15.63:0/2387454744 conn(0x7f95ac0f7430 msgr2=0x7f95ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:08.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 -- 172.21.15.63:0/2387454744 shutdown_connections 2024-04-24T22:48:08.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.221+0000 7f95b46df700 1 -- 172.21.15.63:0/2387454744 wait complete. 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 Processor -- start 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 -- start start 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 0x7f95ac110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac066330 con 0x7f95ac0fcf60 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b46df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac066470 con 0x7f95ac0fbcb0 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b1c7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:08.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b1c7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46964/0 (socket says 172.21.15.63:46964) 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b1c7a700 1 -- 172.21.15.63:0/3719658017 learned_addr learned my addr 172.21.15.63:0/3719658017 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.222+0000 7f95b247b700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 0x7f95ac110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b1c7a700 1 -- 172.21.15.63:0/3719658017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 msgr2=0x7f95ac110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b1c7a700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 0x7f95ac110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b1c7a700 1 -- 172.21.15.63:0/3719658017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c005040 con 0x7f95ac0fcf60 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b1c7a700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f959c00b970 tx=0x7f959c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959c00e040 con 0x7f95ac0fcf60 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1131b0 con 0x7f95ac0fcf60 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac113770 con 0x7f95ac0fcf60 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f959c00bbf0 con 0x7f95ac0fcf60 2024-04-24T22:48:08.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.223+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959c01c630 con 0x7f95ac0fcf60 2024-04-24T22:48:08.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.224+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95900052e0 con 0x7f95ac0fcf60 2024-04-24T22:48:08.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.225+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f959c013420 con 0x7f95ac0fcf60 2024-04-24T22:48:08.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.225+0000 7f95a2ffd700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 0x7f9598070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:08.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.225+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f959c08e600 con 0x7f95ac0fcf60 2024-04-24T22:48:08.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.225+0000 7f95b247b700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 0x7f9598070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:08.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.226+0000 7f95b247b700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 0x7f9598070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f95ac065fb0 tx=0x7f95a800b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:08.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.230+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f959c0580d0 con 0x7f95ac0fcf60 2024-04-24T22:48:08.445 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:08 smithi063 ceph-mon[26912]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:08.445 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:08 smithi063 ceph-mon[26912]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:08.445 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:08 smithi063 ceph-mon[26912]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:08.445 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:08 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3691152936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:08.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.459+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9590000bf0 con 0x7f959806db60 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.460+0000 7f95a2ffd700 1 -- 172.21.15.63:0/3719658017 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f9590000bf0 con 0x7f959806db60 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:48:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:48:08.464 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:48:08.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.462+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 msgr2=0x7f9598070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.462+0000 7f95b46df700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 0x7f9598070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f95ac065fb0 tx=0x7f95a800b040 comp rx=0 tx=0).stop 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.462+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 msgr2=0x7f95ac112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.462+0000 7f95b46df700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f959c00b970 tx=0x7f959c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.462+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 shutdown_connections 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f959806db60 0x7f9598070010 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95ac0fbcb0 0x7f95ac110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 --2- 172.21.15.63:0/3719658017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95ac0fcf60 0x7f95ac112c70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 >> 172.21.15.63:0/3719658017 conn(0x7f95ac0f7430 msgr2=0x7f95ac0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 shutdown_connections 2024-04-24T22:48:08.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:08.463+0000 7f95b46df700 1 -- 172.21.15.63:0/3719658017 wait complete. 2024-04-24T22:48:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:08 smithi192 ceph-mon[33991]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:08 smithi192 ceph-mon[33991]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:08 smithi192 ceph-mon[33991]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:08 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3691152936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:09 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:09.673 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:09.674 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:09 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:10.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:10 smithi063 ceph-mon[26912]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:10.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:10 smithi063 ceph-mon[26912]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:10.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:10 smithi063 ceph-mon[26912]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:10.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:10 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:48:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:10 smithi192 ceph-mon[33991]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:10 smithi192 ceph-mon[33991]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:10 smithi192 ceph-mon[33991]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:10.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:10 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T22:48:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:11 smithi063 ceph-mon[26912]: Upgrade: Updating alertmanager.smithi063 2024-04-24T22:48:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:11 smithi063 ceph-mon[26912]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:11 smithi063 ceph-mon[26912]: Deploying daemon alertmanager.smithi063 on smithi063 2024-04-24T22:48:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:11 smithi192 ceph-mon[33991]: Upgrade: Updating alertmanager.smithi063 2024-04-24T22:48:11.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:11 smithi192 ceph-mon[33991]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:11.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:11 smithi192 ceph-mon[33991]: Deploying daemon alertmanager.smithi063 on smithi063 2024-04-24T22:48:12.691 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:12 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:12.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:12 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:13.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:13 smithi063 ceph-mon[26912]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:13 smithi192 ceph-mon[33991]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:15.744 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:15 smithi063 ceph-mon[26912]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:15 smithi192 ceph-mon[33991]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:17.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:17 smithi063 ceph-mon[26912]: pgmap v48: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:17.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:17 smithi192 ceph-mon[33991]: pgmap v48: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:18 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:18.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:18 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:19 smithi063 ceph-mon[26912]: pgmap v49: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:19.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:19 smithi192 ceph-mon[33991]: pgmap v49: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:21.740 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:21 smithi063 ceph-mon[26912]: pgmap v50: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:21 smithi192 ceph-mon[33991]: pgmap v50: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:23 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:23 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:23 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:23 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:24 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:24.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:24 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:24.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:24 smithi192 ceph-mon[33991]: pgmap v51: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:24.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:24 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:24.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:24 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:24.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:24 smithi063 ceph-mon[26912]: pgmap v51: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:25.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:25 smithi063 ceph-mon[26912]: pgmap v52: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:25.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:25 smithi192 ceph-mon[33991]: pgmap v52: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:27 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:27 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: pgmap v53: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: pgmap v54: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: pgmap v55: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:48:28.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:28 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: pgmap v53: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: pgmap v54: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: pgmap v55: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:28 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:29 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:48:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:29 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:48:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:29 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T22:48:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:29 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063:9093"}]: dispatch 2024-04-24T22:48:30.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:30 smithi063 ceph-mon[26912]: Upgrade: Updating grafana.smithi063 2024-04-24T22:48:30.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:30 smithi063 ceph-mon[26912]: Deploying daemon grafana.smithi063 on smithi063 2024-04-24T22:48:30.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:30 smithi063 ceph-mon[26912]: mgrmap e34: smithi063.upcnvd(active, since 92s), standbys: smithi192.ctmwwc 2024-04-24T22:48:30.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:30 smithi063 ceph-mon[26912]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:30 smithi192 ceph-mon[33991]: Upgrade: Updating grafana.smithi063 2024-04-24T22:48:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:30 smithi192 ceph-mon[33991]: Deploying daemon grafana.smithi063 on smithi063 2024-04-24T22:48:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:30 smithi192 ceph-mon[33991]: mgrmap e34: smithi063.upcnvd(active, since 92s), standbys: smithi192.ctmwwc 2024-04-24T22:48:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:30 smithi192 ceph-mon[33991]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:32.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:32 smithi063 ceph-mon[26912]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:32.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:32 smithi192 ceph-mon[33991]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:34.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:34 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:34.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:34 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:34.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:34 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:34.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:34 smithi192 ceph-mon[33991]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:34.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:34 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:34.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:34 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:34.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:34 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:34.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:34 smithi063 ceph-mon[26912]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:35 smithi063 ceph-mon[26912]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:35 smithi192 ceph-mon[33991]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 381 B/s wr, 0 op/s 2024-04-24T22:48:37.666 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:37 smithi063 ceph-mon[26912]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-24T22:48:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:37 smithi192 ceph-mon[33991]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-24T22:48:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 -- 172.21.15.63:0/3194338627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fcf60 msgr2=0x7fb2580fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 --2- 172.21.15.63:0/3194338627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fcf60 0x7fb2580fd3d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb24c004940 tx=0x7fb24c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 -- 172.21.15.63:0/3194338627 shutdown_connections 2024-04-24T22:48:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 --2- 172.21.15.63:0/3194338627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fcf60 0x7fb2580fd3d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 --2- 172.21.15.63:0/3194338627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2580fbcb0 0x7fb2580fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 -- 172.21.15.63:0/3194338627 >> 172.21.15.63:0/3194338627 conn(0x7fb2580f7430 msgr2=0x7fb2580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:38.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 -- 172.21.15.63:0/3194338627 shutdown_connections 2024-04-24T22:48:38.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.582+0000 7fb25d07b700 1 -- 172.21.15.63:0/3194338627 wait complete. 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 Processor -- start 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 -- start start 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2580fcf60 0x7fb25810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25810e450 con 0x7fb2580fbcb0 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb25d07b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25810e5c0 con 0x7fb2580fcf60 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:38.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57182/0 (socket says 172.21.15.63:57182) 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 -- 172.21.15.63:0/2405815701 learned_addr learned my addr 172.21.15.63:0/2405815701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 -- 172.21.15.63:0/2405815701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2580fcf60 msgr2=0x7fb25810de50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2580fcf60 0x7fb25810de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.583+0000 7fb256d9d700 1 -- 172.21.15.63:0/2405815701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c005040 con 0x7fb2580fbcb0 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb256d9d700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c830 tx=0x7fb248004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb24800a760 con 0x7fb2580fbcb0 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb24800ad90 con 0x7fb2580fbcb0 2024-04-24T22:48:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb258112fc0 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2480113a0 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.584+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb258113580 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.585+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258101af0 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.586+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fb24800a8c0 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.586+0000 7fb23f7fe700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 0x7fb24006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.586+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb24808d1d0 con 0x7fb2580fbcb0 2024-04-24T22:48:38.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.586+0000 7fb25659c700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 0x7fb24006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:38.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.587+0000 7fb25659c700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 0x7fb24006fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b970 tx=0x7fb24c009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:38.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.592+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb24805a950 con 0x7fb2580fbcb0 2024-04-24T22:48:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.881+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb25802b410 con 0x7fb24006da40 2024-04-24T22:48:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.883+0000 7fb23f7fe700 1 -- 172.21.15.63:0/2405815701 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fb25802b410 con 0x7fb24006da40 2024-04-24T22:48:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 msgr2=0x7fb24006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 0x7fb24006fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b970 tx=0x7fb24c009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 msgr2=0x7fb25810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c830 tx=0x7fb248004470 comp rx=0 tx=0).stop 2024-04-24T22:48:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 shutdown_connections 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb24006da40 0x7fb24006fef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2580fbcb0 0x7fb25810d910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 --2- 172.21.15.63:0/2405815701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2580fcf60 0x7fb25810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 >> 172.21.15.63:0/2405815701 conn(0x7fb2580f7430 msgr2=0x7fb258066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 shutdown_connections 2024-04-24T22:48:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.885+0000 7fb25d07b700 1 -- 172.21.15.63:0/2405815701 wait complete. 2024-04-24T22:48:38.898 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:48:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 -- 172.21.15.63:0/2241709904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 msgr2=0x7f26f4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/2241709904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f4102300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f26e8004940 tx=0x7f26e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 -- 172.21.15.63:0/2241709904 shutdown_connections 2024-04-24T22:48:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/2241709904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f4102300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/2241709904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 0x7f26f40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 -- 172.21.15.63:0/2241709904 >> 172.21.15.63:0/2241709904 conn(0x7f26f40f7430 msgr2=0x7f26f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 -- 172.21.15.63:0/2241709904 shutdown_connections 2024-04-24T22:48:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.993+0000 7f26f9ff0700 1 -- 172.21.15.63:0/2241709904 wait complete. 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 Processor -- start 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 -- start start 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 0x7f26f410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f410e580 con 0x7f26f40ffee0 2024-04-24T22:48:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f9ff0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f410edd0 con 0x7f26f40662c0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.994+0000 7f26f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43814/0 (socket says 172.21.15.63:43814) 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 -- 172.21.15.63:0/4226373372 learned_addr learned my addr 172.21.15.63:0/4226373372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f37fe700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 0x7f26f410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 -- 172.21.15.63:0/4226373372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 msgr2=0x7f26f410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 0x7f26f410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 -- 172.21.15.63:0/4226373372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8005040 con 0x7f26f40ffee0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f2ffd700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009150 tx=0x7f26e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e800e040 con 0x7f26f40ffee0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f410f050 con 0x7f26f40ffee0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26e8004290 con 0x7f26f40ffee0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e801c630 con 0x7f26f40ffee0 2024-04-24T22:48:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.995+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f410f590 con 0x7f26f40ffee0 2024-04-24T22:48:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.996+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f405f5c0 con 0x7f26f40ffee0 2024-04-24T22:48:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.997+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f26e800bbf0 con 0x7f26f40ffee0 2024-04-24T22:48:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.997+0000 7f26dbfff700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 0x7f26dc06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.997+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f26e808dc90 con 0x7f26f40ffee0 2024-04-24T22:48:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.998+0000 7f26f37fe700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 0x7f26dc06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:38.998+0000 7f26f37fe700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 0x7f26dc06fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26f4066000 tx=0x7f26e400b040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:39.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.002+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f26e805b4b0 con 0x7f26f40ffee0 2024-04-24T22:48:39.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.230+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26f410f8c0 con 0x7f26dc06da40 2024-04-24T22:48:39.233 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:39 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:39.233 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:39 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:39.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.232+0000 7f26dbfff700 1 -- 172.21.15.63:0/4226373372 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f26f410f8c0 con 0x7f26dc06da40 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 msgr2=0x7f26dc06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 0x7f26dc06fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26f4066000 tx=0x7f26e400b040 comp rx=0 tx=0).stop 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 msgr2=0x7f26f410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009150 tx=0x7f26e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 shutdown_connections 2024-04-24T22:48:39.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f26dc06da40 0x7f26dc06fef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f40662c0 0x7f26f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 --2- 172.21.15.63:0/4226373372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f40ffee0 0x7f26f410df80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 >> 172.21.15.63:0/4226373372 conn(0x7f26f40f7430 msgr2=0x7f26f40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:39.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 shutdown_connections 2024-04-24T22:48:39.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.235+0000 7f26f9ff0700 1 -- 172.21.15.63:0/4226373372 wait complete. 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.358+0000 7f1f8e662700 1 -- 172.21.15.63:0/2204490132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fbcc0 msgr2=0x7f1f880fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.358+0000 7f1f8e662700 1 --2- 172.21.15.63:0/2204490132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fbcc0 0x7f1f880fc0d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1f78004940 tx=0x7f1f78009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- 172.21.15.63:0/2204490132 shutdown_connections 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 --2- 172.21.15.63:0/2204490132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fcf70 0x7f1f880fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 --2- 172.21.15.63:0/2204490132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fbcc0 0x7f1f880fc0d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- 172.21.15.63:0/2204490132 >> 172.21.15.63:0/2204490132 conn(0x7f1f880f7420 msgr2=0x7f1f880f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- 172.21.15.63:0/2204490132 shutdown_connections 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- 172.21.15.63:0/2204490132 wait complete. 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 Processor -- start 2024-04-24T22:48:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- start start 2024-04-24T22:48:39.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fbcc0 0x7f1f8810da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810e5d0 con 0x7f1f880fcf70 2024-04-24T22:48:39.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.359+0000 7f1f8e662700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810edd0 con 0x7f1f880fbcc0 2024-04-24T22:48:39.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43834/0 (socket says 172.21.15.63:43834) 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 -- 172.21.15.63:0/378803401 learned_addr learned my addr 172.21.15.63:0/378803401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f87fff700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fbcc0 0x7f1f8810da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 -- 172.21.15.63:0/378803401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fbcc0 msgr2=0x7f1f8810da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fbcc0 0x7f1f8810da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 -- 172.21.15.63:0/378803401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78005040 con 0x7f1f880fcf70 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f877fe700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c002a50 tx=0x7f1f7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7c007cc0 con 0x7f1f880fcf70 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.360+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f7c00f040 con 0x7f1f880fcf70 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.361+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7c00db70 con 0x7f1f880fcf70 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.361+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8810f0b0 con 0x7f1f880fcf70 2024-04-24T22:48:39.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.361+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8810f5f0 con 0x7f1f880fcf70 2024-04-24T22:48:39.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.362+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f88101b00 con 0x7f1f880fcf70 2024-04-24T22:48:39.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.362+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f1f7c00dd10 con 0x7f1f880fcf70 2024-04-24T22:48:39.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.363+0000 7f1f84ff9700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 0x7f1f7006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.363+0000 7f1f87fff700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 0x7f1f7006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.363+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1f7c08c800 con 0x7f1f880fcf70 2024-04-24T22:48:39.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.364+0000 7f1f87fff700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 0x7f1f7006fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:39.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.368+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f1f7c059f80 con 0x7f1f880fcf70 2024-04-24T22:48:39.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:39 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:39.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:39 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.586+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f8810f860 con 0x7f1f7006da40 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.596+0000 7f1f84ff9700 1 -- 172.21.15.63:0/378803401 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1f8810f860 con 0x7f1f7006da40 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (22s) 1s ago 11m 14.9M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (11m) 1s ago 11m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (10m) 55s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:48:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6s) 1s ago 11m 51.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (6m) 1s ago 6m 14.6M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (6m) 1s ago 6m 12.9M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (6m) 55s ago 6m 151M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (6m) 55s ago 6m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (115s) 1s ago 13m 508M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (76s) 55s ago 10m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (13m) 1s ago 13m 52.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (10m) 55s ago 10m 43.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (63s) 1s ago 11m 8108k - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (59s) 55s ago 10m 3586k - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (9m) 1s ago 9m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (9m) 1s ago 9m 215M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (8m) 1s ago 8m 180M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:48:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (8m) 55s ago 8m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (8m) 55s ago 8m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (7m) 55s ago 7m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (40s) 1s ago 11m 44.6M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 msgr2=0x7f1f7006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 0x7f1f7006fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 msgr2=0x7f1f8810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c002a50 tx=0x7f1f7c0046b0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 shutdown_connections 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f1f7006da40 0x7f1f7006fef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f880fbcc0 0x7f1f8810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.598+0000 7f1f8e662700 1 --2- 172.21.15.63:0/378803401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f880fcf70 0x7f1f8810dfd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.599+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 >> 172.21.15.63:0/378803401 conn(0x7f1f880f7420 msgr2=0x7f1f88066460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.599+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 shutdown_connections 2024-04-24T22:48:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.599+0000 7f1f8e662700 1 -- 172.21.15.63:0/378803401 wait complete. 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 -- 172.21.15.63:0/176166297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fce60 msgr2=0x7fe8400fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 --2- 172.21.15.63:0/176166297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fce60 0x7fe8400fd270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe830004940 tx=0x7fe830009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 -- 172.21.15.63:0/176166297 shutdown_connections 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 --2- 172.21.15.63:0/176166297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fdfd0 0x7fe8400fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 --2- 172.21.15.63:0/176166297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fce60 0x7fe8400fd270 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 -- 172.21.15.63:0/176166297 >> 172.21.15.63:0/176166297 conn(0x7fe8400f8560 msgr2=0x7fe8400fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 -- 172.21.15.63:0/176166297 shutdown_connections 2024-04-24T22:48:39.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.705+0000 7fe84597d700 1 -- 172.21.15.63:0/176166297 wait complete. 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 Processor -- start 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 -- start start 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fce60 0x7fe84010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840110740 con 0x7fe8400fdfd0 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe84597d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840110f90 con 0x7fe8400fce60 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43862/0 (socket says 172.21.15.63:43862) 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 -- 172.21.15.63:0/56084554 learned_addr learned my addr 172.21.15.63:0/56084554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83effd700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fce60 0x7fe84010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 -- 172.21.15.63:0/56084554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fce60 msgr2=0x7fe84010fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fce60 0x7fe84010fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.706+0000 7fe83e7fc700 1 -- 172.21.15.63:0/56084554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830005040 con 0x7fe8400fdfd0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe83e7fc700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe828002a50 tx=0x7fe8280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe828007cc0 con 0x7fe8400fdfd0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe840111270 con 0x7fe8400fdfd0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82800f040 con 0x7fe8400fdfd0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82800da30 con 0x7fe8400fdfd0 2024-04-24T22:48:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.707+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401117b0 con 0x7fe8400fdfd0 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.708+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840101960 con 0x7fe8400fdfd0 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.708+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fe82800db90 con 0x7fe8400fdfd0 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.709+0000 7fe8377fe700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 0x7fe82c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.709+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe82808cf60 con 0x7fe8400fdfd0 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.709+0000 7fe83effd700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 0x7fe82c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.710+0000 7fe83effd700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 0x7fe82c06fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe83000b970 tx=0x7fe830009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:39.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:39.714+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe8280c1100 con 0x7fe8400fdfd0 2024-04-24T22:48:40.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.027+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe840105a10 con 0x7fe8400fdfd0 2024-04-24T22:48:40.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.028+0000 7fe8377fe700 1 -- 172.21.15.63:0/56084554 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe82805a3d0 con 0x7fe8400fdfd0 2024-04-24T22:48:40.030 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:48:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:48:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:48:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:48:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:48:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:48:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T22:48:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:48:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:48:40.032 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 msgr2=0x7fe82c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 0x7fe82c06fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe83000b970 tx=0x7fe830009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 msgr2=0x7fe840110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe828002a50 tx=0x7fe8280046b0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 shutdown_connections 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe82c06da40 0x7fe82c06fef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8400fce60 0x7fe84010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 --2- 172.21.15.63:0/56084554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8400fdfd0 0x7fe840110140 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 >> 172.21.15.63:0/56084554 conn(0x7fe8400f8560 msgr2=0x7fe8400f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 shutdown_connections 2024-04-24T22:48:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.030+0000 7fe84597d700 1 -- 172.21.15.63:0/56084554 wait complete. 2024-04-24T22:48:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 -- 172.21.15.63:0/1544487193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fdfd0 msgr2=0x7f65300fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 --2- 172.21.15.63:0/1544487193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fdfd0 0x7f65300fe440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6524004940 tx=0x7f6524009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 -- 172.21.15.63:0/1544487193 shutdown_connections 2024-04-24T22:48:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 --2- 172.21.15.63:0/1544487193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fdfd0 0x7f65300fe440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 --2- 172.21.15.63:0/1544487193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65300fce60 0x7f65300fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 -- 172.21.15.63:0/1544487193 >> 172.21.15.63:0/1544487193 conn(0x7f65300f8540 msgr2=0x7f65300fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 -- 172.21.15.63:0/1544487193 shutdown_connections 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.137+0000 7f653519f700 1 -- 172.21.15.63:0/1544487193 wait complete. 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 Processor -- start 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 -- start start 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65300fdfd0 0x7f65300702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530072240 con 0x7f65300fce60 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f653519f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530070820 con 0x7f65300fdfd0 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f652ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f652ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43874/0 (socket says 172.21.15.63:43874) 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.138+0000 7f652ed9d700 1 -- 172.21.15.63:0/2823599249 learned_addr learned my addr 172.21.15.63:0/2823599249 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f652ed9d700 1 -- 172.21.15.63:0/2823599249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65300fdfd0 msgr2=0x7f65300702b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f652ed9d700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65300fdfd0 0x7f65300702b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f652ed9d700 1 -- 172.21.15.63:0/2823599249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524005040 con 0x7f65300fce60 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f652ed9d700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f652000b790 tx=0x7f6520009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652000f770 con 0x7f65300fce60 2024-04-24T22:48:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f652000fda0 con 0x7f65300fce60 2024-04-24T22:48:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6530070b00 con 0x7f65300fce60 2024-04-24T22:48:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652000d310 con 0x7f65300fce60 2024-04-24T22:48:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.139+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65300710c0 con 0x7f65300fce60 2024-04-24T22:48:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.140+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653004c710 con 0x7f65300fce60 2024-04-24T22:48:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.141+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f6520010420 con 0x7f65300fce60 2024-04-24T22:48:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.142+0000 7f65177fe700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 0x7f651806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:48:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.142+0000 7f652e59c700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 0x7f651806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:48:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.142+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f652008d1e0 con 0x7f65300fce60 2024-04-24T22:48:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.142+0000 7f652e59c700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 0x7f651806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f652400b970 tx=0x7f6524009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:48:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.146+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f652005a960 con 0x7f65300fce60 2024-04-24T22:48:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.377+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65301022c0 con 0x7f651806da40 2024-04-24T22:48:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.378+0000 7f65177fe700 1 -- 172.21.15.63:0/2823599249 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f65301022c0 con 0x7f651806da40 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading grafana daemons", 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:48:40.380 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 msgr2=0x7f651806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 0x7f651806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f652400b970 tx=0x7f6524009e30 comp rx=0 tx=0).stop 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 msgr2=0x7f6530071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f652000b790 tx=0x7f6520009250 comp rx=0 tx=0).stop 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 shutdown_connections 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f651806da40 0x7f651806fef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65300fce60 0x7f6530071bb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 --2- 172.21.15.63:0/2823599249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65300fdfd0 0x7f65300702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 >> 172.21.15.63:0/2823599249 conn(0x7f65300f8540 msgr2=0x7f65300f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:48:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 shutdown_connections 2024-04-24T22:48:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:48:40.380+0000 7f653519f700 1 -- 172.21.15.63:0/2823599249 wait complete. 2024-04-24T22:48:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T22:48:40.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:40 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/56084554' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:40.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T22:48:40.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:40 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/56084554' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:41.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:41 smithi063 ceph-mon[26912]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:41.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:41 smithi063 ceph-mon[26912]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:41.610 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:41 smithi192 ceph-mon[33991]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:41.610 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:41 smithi192 ceph-mon[33991]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:48:42.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:42 smithi063 ceph-mon[26912]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:42.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:42 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:42.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:42 smithi192 ceph-mon[33991]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:42.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:42 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:44.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: Upgrade: Finalizing container_image settings 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T22:48:44.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: Upgrade: Complete! 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.183 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:43 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063:3000"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: Upgrade: Finalizing container_image settings 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.306 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T22:48:44.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:48:44.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: Upgrade: Complete! 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:48:44.309 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:43 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:44 smithi192 ceph-mon[33991]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T22:48:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:44 smithi063 ceph-mon[26912]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T22:48:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:45 smithi192 ceph-mon[33991]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:46.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:45 smithi063 ceph-mon[26912]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:48.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:47 smithi063 ceph-mon[26912]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:48.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:47 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:48.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:47 smithi192 ceph-mon[33991]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:48.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:47 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:50.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:49 smithi063 ceph-mon[26912]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:50.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:49 smithi192 ceph-mon[33991]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:52.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:52 smithi063 ceph-mon[26912]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:52.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:52 smithi192 ceph-mon[33991]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:54.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:54 smithi063 ceph-mon[26912]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:54.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:54 smithi192 ceph-mon[33991]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:56.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:56 smithi063 ceph-mon[26912]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:56.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:56 smithi192 ceph-mon[33991]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:57 smithi063 ceph-mon[26912]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:57 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:48:58.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:57 smithi192 ceph-mon[33991]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:48:58.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:48:58.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:57 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:48:59 smithi063 ceph-mon[26912]: pgmap v71: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:48:59 smithi192 ceph-mon[33991]: pgmap v71: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:01 smithi063 ceph-mon[26912]: pgmap v72: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:01 smithi192 ceph-mon[33991]: pgmap v72: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:04 smithi063 ceph-mon[26912]: pgmap v73: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:04.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:04 smithi192 ceph-mon[33991]: pgmap v73: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:06.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:06 smithi063 ceph-mon[26912]: pgmap v74: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:06.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:06 smithi192 ceph-mon[33991]: pgmap v74: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:08.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:08 smithi063 ceph-mon[26912]: pgmap v75: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:08 smithi192 ceph-mon[33991]: pgmap v75: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:10.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:10 smithi063 ceph-mon[26912]: pgmap v76: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:10.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:10 smithi192 ceph-mon[33991]: pgmap v76: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 -- 172.21.15.63:0/3816818627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe80fff20 msgr2=0x7fcbe8102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/3816818627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe80fff20 0x7fcbe8102340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8004940 tx=0x7fcbd8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 -- 172.21.15.63:0/3816818627 shutdown_connections 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/3816818627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe80fff20 0x7fcbe8102340 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/3816818627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbe8066f30 0x7fcbe80ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 -- 172.21.15.63:0/3816818627 >> 172.21.15.63:0/3816818627 conn(0x7fcbe80f7450 msgr2=0x7fcbe80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 -- 172.21.15.63:0/3816818627 shutdown_connections 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.501+0000 7fcbf02c1700 1 -- 172.21.15.63:0/3816818627 wait complete. 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 Processor -- start 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 -- start start 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbe80fff20 0x7fcbe810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe810e4c0 con 0x7fcbe8066f30 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbf02c1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe810e630 con 0x7fcbe80fff20 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44966/0 (socket says 172.21.15.63:44966) 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 -- 172.21.15.63:0/2367989010 learned_addr learned my addr 172.21.15.63:0/2367989010 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 -- 172.21.15.63:0/2367989010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbe80fff20 msgr2=0x7fcbe810dec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbe80fff20 0x7fcbe810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.502+0000 7fcbee05d700 1 -- 172.21.15.63:0/2367989010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8005040 con 0x7fcbe8066f30 2024-04-24T22:49:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbee05d700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4002a50 tx=0x7fcbe4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbe4007cc0 con 0x7fcbe8066f30 2024-04-24T22:49:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbe400f040 con 0x7fcbe8066f30 2024-04-24T22:49:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe8112fe0 con 0x7fcbe8066f30 2024-04-24T22:49:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbe400da40 con 0x7fcbe8066f30 2024-04-24T22:49:10.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.503+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe81135a0 con 0x7fcbe8066f30 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.504+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe804c710 con 0x7fcbe8066f30 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.504+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fcbe40097c0 con 0x7fcbe8066f30 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.505+0000 7fcbdeffd700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 0x7fcbd406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.505+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcbe408cf20 con 0x7fcbe8066f30 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.505+0000 7fcbed85c700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 0x7fcbd406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:10.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.505+0000 7fcbed85c700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 0x7fcbd406fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8004f90 tx=0x7fcbd8009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.510+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fcbe40c1100 con 0x7fcbe8066f30 2024-04-24T22:49:10.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.733+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe80f8f90 con 0x7fcbd406da40 2024-04-24T22:49:10.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.735+0000 7fcbdeffd700 1 -- 172.21.15.63:0/2367989010 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fcbe80f8f90 con 0x7fcbd406da40 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 msgr2=0x7fcbd406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 0x7fcbd406fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8004f90 tx=0x7fcbd8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 msgr2=0x7fcbe810d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4002a50 tx=0x7fcbe4004600 comp rx=0 tx=0).stop 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 shutdown_connections 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fcbd406da40 0x7fcbd406fef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe8066f30 0x7fcbe810d980 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 --2- 172.21.15.63:0/2367989010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbe80fff20 0x7fcbe810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 >> 172.21.15.63:0/2367989010 conn(0x7fcbe80f7450 msgr2=0x7fcbe80f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 shutdown_connections 2024-04-24T22:49:10.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:10.737+0000 7fcbf02c1700 1 -- 172.21.15.63:0/2367989010 wait complete. 2024-04-24T22:49:11.519 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-24T22:49:11.831 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:12.255 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:12 smithi063 ceph-mon[26912]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:12.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:12 smithi063 ceph-mon[26912]: pgmap v77: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:12.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:12 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:12.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:12 smithi192 ceph-mon[33991]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:12.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:12 smithi192 ceph-mon[33991]: pgmap v77: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:12.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:12 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 -- 172.21.15.63:0/832984397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 msgr2=0x7f39341031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 --2- 172.21.15.63:0/832984397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39341031f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3924004940 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 -- 172.21.15.63:0/832984397 shutdown_connections 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 --2- 172.21.15.63:0/832984397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 0x7f39340fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 --2- 172.21.15.63:0/832984397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39341031f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 -- 172.21.15.63:0/832984397 >> 172.21.15.63:0/832984397 conn(0x7f3934072380 msgr2=0x7f3934072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 -- 172.21.15.63:0/832984397 shutdown_connections 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.549+0000 7f393933e700 1 -- 172.21.15.63:0/832984397 wait complete. 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 Processor -- start 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 -- start start 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 0x7f39340fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340fe580 con 0x7f3934102e20 2024-04-24T22:49:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.550+0000 7f393933e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340fd020 con 0x7f39341037c0 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44986/0 (socket says 172.21.15.63:44986) 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 -- 172.21.15.63:0/2887379039 learned_addr learned my addr 172.21.15.63:0/2887379039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f39327fc700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 0x7f39340fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 -- 172.21.15.63:0/2887379039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 msgr2=0x7f39340fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 0x7f39340fcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 -- 172.21.15.63:0/2887379039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924005040 con 0x7f3934102e20 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f3932ffd700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3924009100 tx=0x7f3924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392400e040 con 0x7f3934102e20 2024-04-24T22:49:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3924009af0 con 0x7f3934102e20 2024-04-24T22:49:13.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39340fd2a0 con 0x7f3934102e20 2024-04-24T22:49:13.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392401c3f0 con 0x7f3934102e20 2024-04-24T22:49:13.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.551+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393411a140 con 0x7f3934102e20 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.552+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39340fe6e0 con 0x7f3934102e20 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.553+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f392400baf0 con 0x7f3934102e20 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.553+0000 7f391b7fe700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 0x7f391c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.553+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f392408e3e0 con 0x7f3934102e20 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.553+0000 7f39327fc700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 0x7f391c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.554+0000 7f39327fc700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 0x7f391c06fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f39340fdcd0 tx=0x7f39280091a0 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:13.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.558+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3924058030 con 0x7f3934102e20 2024-04-24T22:49:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.854+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f39340fda80 con 0x7f3934102e20 2024-04-24T22:49:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.854+0000 7f391b7fe700 1 -- 172.21.15.63:0/2887379039 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f392405b6f0 con 0x7f3934102e20 2024-04-24T22:49:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 msgr2=0x7f391c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 0x7f391c06fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f39340fdcd0 tx=0x7f39280091a0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 msgr2=0x7f39340fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3924009100 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 shutdown_connections 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f391c06da40 0x7f391c06fef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3934102e20 0x7f39340fc5a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 --2- 172.21.15.63:0/2887379039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39341037c0 0x7f39340fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 >> 172.21.15.63:0/2887379039 conn(0x7f3934072380 msgr2=0x7f39340f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 shutdown_connections 2024-04-24T22:49:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:13.856+0000 7f393933e700 1 -- 172.21.15.63:0/2887379039 wait complete. 2024-04-24T22:49:13.869 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:49:14.218 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:14 smithi063 ceph-mon[26912]: pgmap v78: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:14.218 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:14 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2887379039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:14 smithi192 ceph-mon[33991]: pgmap v78: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:14 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2887379039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:14.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-24T22:49:14.852 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:16 smithi063 ceph-mon[26912]: pgmap v79: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:16.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:16 smithi192 ceph-mon[33991]: pgmap v79: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:16.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- 172.21.15.63:0/1202607153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28103780 msgr2=0x7f8b280fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:16.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 --2- 172.21.15.63:0/1202607153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28103780 0x7f8b280fcdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8b14004940 tx=0x7f8b14009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:16.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- 172.21.15.63:0/1202607153 shutdown_connections 2024-04-24T22:49:16.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 --2- 172.21.15.63:0/1202607153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28103780 0x7f8b280fcdf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 --2- 172.21.15.63:0/1202607153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28102de0 0x7f8b281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- 172.21.15.63:0/1202607153 >> 172.21.15.63:0/1202607153 conn(0x7f8b280f8990 msgr2=0x7f8b280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- 172.21.15.63:0/1202607153 shutdown_connections 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- 172.21.15.63:0/1202607153 wait complete. 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 Processor -- start 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.578+0000 7f8b2f197700 1 -- start start 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2f197700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2f197700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 0x7f8b28113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2f197700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281145a0 con 0x7f8b28102de0 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2f197700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2810db60 con 0x7f8b28103780 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45004/0 (socket says 172.21.15.63:45004) 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 -- 172.21.15.63:0/2254429751 learned_addr learned my addr 172.21.15.63:0/2254429751 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:16.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b27fff700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 0x7f8b28113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 -- 172.21.15.63:0/2254429751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 msgr2=0x7f8b28113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 0x7f8b28113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 -- 172.21.15.63:0/2254429751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14005040 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b27fff700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 0x7f8b28113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b2cf33700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00b790 tx=0x7f8b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.579+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c00f770 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.580+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2810de40 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.580+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1c00fda0 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.580+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c00d310 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.580+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2810e400 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.580+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b281000f0 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.581+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8b1c010420 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.581+0000 7f8b257fa700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 0x7f8b1806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.582+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8b1c08d1e0 con 0x7f8b28102de0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.582+0000 7f8b27fff700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 0x7f8b1806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:16.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.582+0000 7f8b27fff700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 0x7f8b1806fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b2810ecd0 tx=0x7f8b14009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:16.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.586+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8b1c05a960 con 0x7f8b28102de0 2024-04-24T22:49:16.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.879+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b2810efd0 con 0x7f8b28102de0 2024-04-24T22:49:16.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.880+0000 7f8b257fa700 1 -- 172.21.15.63:0/2254429751 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f8b1c05a4f0 con 0x7f8b28102de0 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 msgr2=0x7f8b1806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 0x7f8b1806fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b2810ecd0 tx=0x7f8b14009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 msgr2=0x7f8b28113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00b790 tx=0x7f8b1c009250 comp rx=0 tx=0).stop 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 shutdown_connections 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f8b1806da40 0x7f8b1806fef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b28102de0 0x7f8b28113980 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 --2- 172.21.15.63:0/2254429751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b28103780 0x7f8b28113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 >> 172.21.15.63:0/2254429751 conn(0x7f8b280f8990 msgr2=0x7f8b28107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 shutdown_connections 2024-04-24T22:49:16.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:16.882+0000 7f8b2f197700 1 -- 172.21.15.63:0/2254429751 wait complete. 2024-04-24T22:49:16.894 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-24T22:49:17.215 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:17 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2254429751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:17.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:17 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2254429751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:17.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-24T22:49:17.807 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:18.243 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:18 smithi063 ceph-mon[26912]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:18.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:18 smithi192 ceph-mon[33991]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 -- 172.21.15.63:0/1600302797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740102df0 msgr2=0x7f17401031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 --2- 172.21.15.63:0/1600302797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740102df0 0x7f17401031c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1730004940 tx=0x7f1730009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 -- 172.21.15.63:0/1600302797 shutdown_connections 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 --2- 172.21.15.63:0/1600302797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740103790 0x7f17400fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 --2- 172.21.15.63:0/1600302797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740102df0 0x7f17401031c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 -- 172.21.15.63:0/1600302797 >> 172.21.15.63:0/1600302797 conn(0x7f17400f8980 msgr2=0x7f17400fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 -- 172.21.15.63:0/1600302797 shutdown_connections 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.593+0000 7f1747b35700 1 -- 172.21.15.63:0/1600302797 wait complete. 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 Processor -- start 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 -- start start 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740102df0 0x7f1740113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401147b0 con 0x7f1740103790 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f1747b35700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174010db10 con 0x7f1740102df0 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f17450d0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f17450d0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57376/0 (socket says 172.21.15.63:57376) 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f17450d0700 1 -- 172.21.15.63:0/3697786720 learned_addr learned my addr 172.21.15.63:0/3697786720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.594+0000 7f17458d1700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740102df0 0x7f1740113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f17450d0700 1 -- 172.21.15.63:0/3697786720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740102df0 msgr2=0x7f1740113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f17450d0700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740102df0 0x7f1740113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f17450d0700 1 -- 172.21.15.63:0/3697786720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1730005040 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f17450d0700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f173400b790 tx=0x7f1734009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173400f6f0 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173400fd20 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174010ddf0 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17340103a0 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.595+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174010e3b0 con 0x7f1740103790 2024-04-24T22:49:19.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.596+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17400fe540 con 0x7f1740103790 2024-04-24T22:49:19.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.596+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f1734011400 con 0x7f1740103790 2024-04-24T22:49:19.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.597+0000 7f173e7fc700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 0x7f172806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:19.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.597+0000 7f17458d1700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 0x7f172806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:19.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.597+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f173408ce80 con 0x7f1740103790 2024-04-24T22:49:19.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.598+0000 7f17458d1700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 0x7f172806ff00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f173000b3b0 tx=0x7f1730009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:19.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.602+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f173405a6a0 con 0x7f1740103790 2024-04-24T22:49:19.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.895+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f174005f5c0 con 0x7f1740103790 2024-04-24T22:49:19.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.895+0000 7f173e7fc700 1 -- 172.21.15.63:0/3697786720 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f173405a230 con 0x7f1740103790 2024-04-24T22:49:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.897+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 msgr2=0x7f172806ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.897+0000 7f1747b35700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 0x7f172806ff00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f173000b3b0 tx=0x7f1730009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.897+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 msgr2=0x7f17401140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.897+0000 7f1747b35700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f173400b790 tx=0x7f1734009220 comp rx=0 tx=0).stop 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 shutdown_connections 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f172806da50 0x7f172806ff00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740102df0 0x7f1740113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 --2- 172.21.15.63:0/3697786720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103790 0x7f17401140d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 >> 172.21.15.63:0/3697786720 conn(0x7f17400f8980 msgr2=0x7f17400f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 shutdown_connections 2024-04-24T22:49:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:19.898+0000 7f1747b35700 1 -- 172.21.15.63:0/3697786720 wait complete. 2024-04-24T22:49:19.910 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:49:20.204 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:20 smithi063 ceph-mon[26912]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:20.204 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:20 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/3697786720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:20.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:20 smithi192 ceph-mon[33991]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:20.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:20 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/3697786720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:20.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-24T22:49:20.801 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:22.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:22 smithi063 ceph-mon[26912]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- 172.21.15.63:0/683361572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c04102e00 msgr2=0x7f7c041031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 --2- 172.21.15.63:0/683361572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c04102e00 0x7f7c041031d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4004940 tx=0x7f7bf4009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- 172.21.15.63:0/683361572 shutdown_connections 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 --2- 172.21.15.63:0/683361572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c041037a0 0x7f7c040fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 --2- 172.21.15.63:0/683361572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c04102e00 0x7f7c041031d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- 172.21.15.63:0/683361572 >> 172.21.15.63:0/683361572 conn(0x7f7c040f89d0 msgr2=0x7f7c040fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- 172.21.15.63:0/683361572 shutdown_connections 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- 172.21.15.63:0/683361572 wait complete. 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 Processor -- start 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.401+0000 7f7c09b02700 1 -- start start 2024-04-24T22:49:22.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c09b02700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 0x7f7c04113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c09b02700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c09b02700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04114570 con 0x7f7c041037a0 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c09b02700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0410dae0 con 0x7f7c04102e00 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57390/0 (socket says 172.21.15.63:57390) 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 -- 172.21.15.63:0/3167765706 learned_addr learned my addr 172.21.15.63:0/3167765706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c037fe700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 0x7f7c04113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 -- 172.21.15.63:0/3167765706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 msgr2=0x7f7c04113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 0x7f7c04113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 -- 172.21.15.63:0/3167765706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4005040 con 0x7f7c041037a0 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c037fe700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 0x7f7c04113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:49:22.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.402+0000 7f7c02ffd700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800b790 tx=0x7f7bf8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:22.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.403+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf800f6f0 con 0x7f7c041037a0 2024-04-24T22:49:22.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.403+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0410ddc0 con 0x7f7c041037a0 2024-04-24T22:49:22.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.403+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf800fd20 con 0x7f7c041037a0 2024-04-24T22:49:22.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.403+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf80103a0 con 0x7f7c041037a0 2024-04-24T22:49:22.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.403+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0410e380 con 0x7f7c041037a0 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.404+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040fe550 con 0x7f7c041037a0 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.404+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f7bf8011400 con 0x7f7c041037a0 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.405+0000 7f7bebfff700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 0x7f7bec06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.405+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7bf808d070 con 0x7f7c041037a0 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.405+0000 7f7c037fe700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 0x7f7bec06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:22.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.405+0000 7f7c037fe700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 0x7f7bec06fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4004560 tx=0x7f7bf4009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:22.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.409+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7bf805a770 con 0x7f7c041037a0 2024-04-24T22:49:22.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:22 smithi192 ceph-mon[33991]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:22.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:22.637+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c04109f70 con 0x7f7bec06da40 2024-04-24T22:49:24.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:24 smithi063 ceph-mon[26912]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:24.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:24 smithi063 ceph-mon[26912]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:24.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:24 smithi192 ceph-mon[33991]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:24.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:24 smithi192 ceph-mon[33991]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:24.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.857+0000 7f7bebfff700 1 -- 172.21.15.63:0/3167765706 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f7c04109f70 con 0x7f7bec06da40 2024-04-24T22:49:24.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.859+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 msgr2=0x7f7bec06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:24.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.859+0000 7f7c09b02700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 0x7f7bec06fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4004560 tx=0x7f7bf4009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.859+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 msgr2=0x7f7c04113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.859+0000 7f7c09b02700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800b790 tx=0x7f7bf8009220 comp rx=0 tx=0).stop 2024-04-24T22:49:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 shutdown_connections 2024-04-24T22:49:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f7bec06da40 0x7f7bec06fef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c04102e00 0x7f7c04113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 --2- 172.21.15.63:0/3167765706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c041037a0 0x7f7c04113e90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 >> 172.21.15.63:0/3167765706 conn(0x7f7c040f89d0 msgr2=0x7f7c040f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 shutdown_connections 2024-04-24T22:49:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:24.860+0000 7f7c09b02700 1 -- 172.21.15.63:0/3167765706 wait complete. 2024-04-24T22:49:24.873 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:49:25.462 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T22:49:25.727 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:26.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:26 smithi063 ceph-mon[26912]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:26.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:26 smithi192 ceph-mon[33991]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:27.154 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:27 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 -- 172.21.15.63:0/1562355791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070280 msgr2=0x7ff238070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/1562355791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070280 0x7ff238070650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff230009ce0 tx=0x7ff230004d10 comp rx=0 tx=0).stop 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 -- 172.21.15.63:0/1562355791 shutdown_connections 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/1562355791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070b90 0x7ff238109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/1562355791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070280 0x7ff238070650 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 -- 172.21.15.63:0/1562355791 >> 172.21.15.63:0/1562355791 conn(0x7ff2380f8950 msgr2=0x7ff2380fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 -- 172.21.15.63:0/1562355791 shutdown_connections 2024-04-24T22:49:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.379+0000 7ff23d2e7700 1 -- 172.21.15.63:0/1562355791 wait complete. 2024-04-24T22:49:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 Processor -- start 2024-04-24T22:49:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 -- start start 2024-04-24T22:49:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070280 0x7ff238115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238116af0 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff23d2e7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23810fe30 con 0x7ff238070280 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57412/0 (socket says 172.21.15.63:57412) 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 -- 172.21.15.63:0/3286020329 learned_addr learned my addr 172.21.15.63:0/3286020329 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff236ffd700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070280 0x7ff238115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 -- 172.21.15.63:0/3286020329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070280 msgr2=0x7ff238115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070280 0x7ff238115ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.380+0000 7ff2367fc700 1 -- 172.21.15.63:0/3286020329 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009940 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff2367fc700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b790 tx=0x7ff220009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22000f6f0 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2381100b0 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff22000fd20 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2200103a0 con 0x7ff238070b90 2024-04-24T22:49:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.381+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2381105f0 con 0x7ff238070b90 2024-04-24T22:49:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.382+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2380719c0 con 0x7ff238070b90 2024-04-24T22:49:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.382+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff220011400 con 0x7ff238070b90 2024-04-24T22:49:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.383+0000 7ff22f7fe700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 0x7ff22406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.383+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff22008cfb0 con 0x7ff238070b90 2024-04-24T22:49:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.383+0000 7ff236ffd700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 0x7ff22406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.383+0000 7ff236ffd700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 0x7ff22406fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b740 tx=0x7ff230004d10 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:27.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.387+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff22005a730 con 0x7ff238070b90 2024-04-24T22:49:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:27 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.602+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff23810c1b0 con 0x7ff22406da40 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.613+0000 7ff22f7fe700 1 -- 172.21.15.63:0/3286020329 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff23810c1b0 con 0x7ff22406da40 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (70s) 49s ago 12m 14.9M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (12m) 49s ago 12m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (11m) 103s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (55s) 49s ago 12m 51.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (6m) 49s ago 6m 14.6M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:49:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (7m) 49s ago 7m 12.9M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (6m) 103s ago 6m 151M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (6m) 103s ago 6m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (2m) 49s ago 13m 508M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (2m) 103s ago 11m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (14m) 49s ago 14m 52.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (11m) 103s ago 11m 43.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (111s) 49s ago 12m 8108k - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (107s) 103s ago 11m 3586k - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 49s ago 10m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (10m) 49s ago 10m 215M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 49s ago 9m 180M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (9m) 103s ago 9m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (8m) 103s ago 8m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (8m) 103s ago 8m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:49:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (88s) 49s ago 11m 44.6M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 msgr2=0x7ff22406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 0x7ff22406fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b740 tx=0x7ff230004d10 comp rx=0 tx=0).stop 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 msgr2=0x7ff238116410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b790 tx=0x7ff220009250 comp rx=0 tx=0).stop 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 shutdown_connections 2024-04-24T22:49:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7ff22406da40 0x7ff22406fef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff238070280 0x7ff238115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 --2- 172.21.15.63:0/3286020329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff238070b90 0x7ff238116410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 >> 172.21.15.63:0/3286020329 conn(0x7ff2380f8950 msgr2=0x7ff2380f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 shutdown_connections 2024-04-24T22:49:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:27.615+0000 7ff23d2e7700 1 -- 172.21.15.63:0/3286020329 wait complete. 2024-04-24T22:49:28.216 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:28 smithi063 ceph-mon[26912]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:28.243 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T22:49:28.244 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:49:28.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-24T22:49:28.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:28 smithi192 ceph-mon[33991]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:28.513 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:29.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:29 smithi063 ceph-mon[26912]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:29.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:29 smithi192 ceph-mon[33991]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:30.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.255+0000 7feafe6dc700 1 -- 172.21.15.63:0/2783416310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf8102f60 msgr2=0x7feaf8103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.255+0000 7feafe6dc700 1 --2- 172.21.15.63:0/2783416310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf8102f60 0x7feaf8103330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feae8004940 tx=0x7feae8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.255+0000 7feafe6dc700 1 -- 172.21.15.63:0/2783416310 shutdown_connections 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.255+0000 7feafe6dc700 1 --2- 172.21.15.63:0/2783416310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf80fb520 0x7feaf80fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.255+0000 7feafe6dc700 1 --2- 172.21.15.63:0/2783416310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf8102f60 0x7feaf8103330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- 172.21.15.63:0/2783416310 >> 172.21.15.63:0/2783416310 conn(0x7feaf80714d0 msgr2=0x7feaf80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- 172.21.15.63:0/2783416310 shutdown_connections 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- 172.21.15.63:0/2783416310 wait complete. 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 Processor -- start 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- start start 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 0x7feaf8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8114530 con 0x7feaf80fb520 2024-04-24T22:49:30.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.256+0000 7feafe6dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf810db30 con 0x7feaf8102f60 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43002/0 (socket says 172.21.15.63:43002) 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 -- 172.21.15.63:0/877895489 learned_addr learned my addr 172.21.15.63:0/877895489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf77fe700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 0x7feaf8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 -- 172.21.15.63:0/877895489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 msgr2=0x7feaf8113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 0x7feaf8113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 -- 172.21.15.63:0/877895489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8005040 con 0x7feaf80fb520 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf77fe700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 0x7feaf8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf7fff700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feae8009100 tx=0x7feae8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae800e040 con 0x7feaf80fb520 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae8013420 con 0x7feaf80fb520 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf810ddb0 con 0x7feaf80fb520 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae801c610 con 0x7feaf80fb520 2024-04-24T22:49:30.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.257+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf810e2f0 con 0x7feaf80fb520 2024-04-24T22:49:30.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.258+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8100250 con 0x7feaf80fb520 2024-04-24T22:49:30.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.259+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7feae8013590 con 0x7feaf80fb520 2024-04-24T22:49:30.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.259+0000 7feaf4ff9700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 0x7feae006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:30.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.259+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7feae808e450 con 0x7feaf80fb520 2024-04-24T22:49:30.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.259+0000 7feaf77fe700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 0x7feae006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:30.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.260+0000 7feaf77fe700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 0x7feae006fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feaec009fd0 tx=0x7feaec009e50 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:30.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.264+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7feae805bbd0 con 0x7feaf80fb520 2024-04-24T22:49:30.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:30 smithi063 ceph-mon[26912]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:30.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:30 smithi192 ceph-mon[33991]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.470+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7feaf810eff0 con 0x7feaf80fb520 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.474+0000 7feaf4ff9700 1 -- 172.21.15.63:0/877895489 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v52)=0 v52) v1 ==== 125+0+0 (secure 0 0 0) 0x7feae805b760 con 0x7feaf80fb520 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 msgr2=0x7feae006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 0x7feae006fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feaec009fd0 tx=0x7feaec009e50 comp rx=0 tx=0).stop 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 msgr2=0x7feaf81139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feae8009100 tx=0x7feae8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 shutdown_connections 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7feae006da40 0x7feae006fef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaf80fb520 0x7feaf81139a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 --2- 172.21.15.63:0/877895489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaf8102f60 0x7feaf8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 >> 172.21.15.63:0/877895489 conn(0x7feaf80714d0 msgr2=0x7feaf80fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 shutdown_connections 2024-04-24T22:49:30.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:30.477+0000 7feafe6dc700 1 -- 172.21.15.63:0/877895489 wait complete. 2024-04-24T22:49:31.157 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T22:49:31.158 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:49:31.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T22:49:31.425 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/877895489' entity='client.admin' 2024-04-24T22:49:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:31.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:31 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/877895489' entity='client.admin' 2024-04-24T22:49:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:31.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:31 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 -- 172.21.15.63:0/610088810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3900fb520 msgr2=0x7fb3900fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 --2- 172.21.15.63:0/610088810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3900fb520 0x7fb3900fb990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb380009ce0 tx=0x7fb380004d10 comp rx=0 tx=0).stop 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 -- 172.21.15.63:0/610088810 shutdown_connections 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 --2- 172.21.15.63:0/610088810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3900fb520 0x7fb3900fb990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 --2- 172.21.15.63:0/610088810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb390102f60 0x7fb390103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 -- 172.21.15.63:0/610088810 >> 172.21.15.63:0/610088810 conn(0x7fb3900714d0 msgr2=0x7fb3900718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 -- 172.21.15.63:0/610088810 shutdown_connections 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.060+0000 7fb396f1f700 1 -- 172.21.15.63:0/610088810 wait complete. 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 Processor -- start 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 -- start start 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3900fb520 0x7fb390113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901147a0 con 0x7fb390102f60 2024-04-24T22:49:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.061+0000 7fb396f1f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39010dcf0 con 0x7fb3900fb520 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43020/0 (socket says 172.21.15.63:43020) 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 -- 172.21.15.63:0/3051652600 learned_addr learned my addr 172.21.15.63:0/3051652600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 -- 172.21.15.63:0/3051652600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3900fb520 msgr2=0x7fb390113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3900fb520 0x7fb390113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 -- 172.21.15.63:0/3051652600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380009940 con 0x7fb390102f60 2024-04-24T22:49:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38ffff700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb38000be90 tx=0x7fb380004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38000e040 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb380008ea0 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39010df10 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38001c810 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.062+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39010e450 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.063+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390100250 con 0x7fb390102f60 2024-04-24T22:49:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.064+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fb380012c00 con 0x7fb390102f60 2024-04-24T22:49:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.064+0000 7fb38d7fa700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 0x7fb37c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.065+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb38008ed10 con 0x7fb390102f60 2024-04-24T22:49:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.065+0000 7fb394cbb700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 0x7fb37c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.065+0000 7fb394cbb700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 0x7fb37c06fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb378006fd0 tx=0x7fb378006db0 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:33.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.069+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb38005c490 con 0x7fb390102f60 2024-04-24T22:49:33.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.272+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fb3900fe840 con 0x7fb390102f60 2024-04-24T22:49:33.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.273+0000 7fb38d7fa700 1 -- 172.21.15.63:0/3051652600 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v52)=0 v52) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb380004e10 con 0x7fb390102f60 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 msgr2=0x7fb37c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 0x7fb37c06fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb378006fd0 tx=0x7fb378006db0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 msgr2=0x7fb3901140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb38000be90 tx=0x7fb380004d10 comp rx=0 tx=0).stop 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 shutdown_connections 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fb37c06da40 0x7fb37c06fef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3900fb520 0x7fb390113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 --2- 172.21.15.63:0/3051652600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb390102f60 0x7fb3901140c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:33.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 >> 172.21.15.63:0/3051652600 conn(0x7fb3900714d0 msgr2=0x7fb3900fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:33.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 shutdown_connections 2024-04-24T22:49:33.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:33.275+0000 7fb396f1f700 1 -- 172.21.15.63:0/3051652600 wait complete. 2024-04-24T22:49:33.959 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T22:49:34.232 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:34.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:33 smithi063 ceph-mon[26912]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:34.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:33 smithi192 ceph-mon[33991]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 -- 172.21.15.63:0/2778274918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 msgr2=0x7fa6bc0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 --2- 172.21.15.63:0/2778274918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc0fcdd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004940 tx=0x7fa6ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 -- 172.21.15.63:0/2778274918 shutdown_connections 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 --2- 172.21.15.63:0/2778274918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc0fcdd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 --2- 172.21.15.63:0/2778274918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 0x7fa6bc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.842+0000 7fa6c372e700 1 -- 172.21.15.63:0/2778274918 >> 172.21.15.63:0/2778274918 conn(0x7fa6bc0f8930 msgr2=0x7fa6bc0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 -- 172.21.15.63:0/2778274918 shutdown_connections 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 -- 172.21.15.63:0/2778274918 wait complete. 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 Processor -- start 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 -- start start 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 0x7fa6bc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc114590 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.843+0000 7fa6c372e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc10db20 con 0x7fa6bc102dc0 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43046/0 (socket says 172.21.15.63:43046) 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 -- 172.21.15.63:0/3957538373 learned_addr learned my addr 172.21.15.63:0/3957538373 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c14ca700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 0x7fa6bc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 -- 172.21.15.63:0/3957538373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 msgr2=0x7fa6bc113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 0x7fa6bc113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 -- 172.21.15.63:0/3957538373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac005040 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c0cc9700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009100 tx=0x7fa6ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac00e040 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6ac009af0 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc10dda0 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.844+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac01c3f0 con 0x7fa6bc103760 2024-04-24T22:49:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.845+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc10e360 con 0x7fa6bc103760 2024-04-24T22:49:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.845+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc1000d0 con 0x7fa6bc103760 2024-04-24T22:49:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.846+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fa6ac00baf0 con 0x7fa6bc103760 2024-04-24T22:49:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.847+0000 7fa6b1ffb700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 0x7fa6a806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.847+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa6ac08e330 con 0x7fa6bc103760 2024-04-24T22:49:35.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.847+0000 7fa6c14ca700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 0x7fa6a806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:35.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.847+0000 7fa6c14ca700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 0x7fa6a806fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009b60 tx=0x7fa6b80091a0 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:35.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:35.851+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa6ac058020 con 0x7fa6bc103760 2024-04-24T22:49:36.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:35 smithi063 ceph-mon[26912]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:36.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.056+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fa6bc0fe510 con 0x7fa6bc103760 2024-04-24T22:49:36.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.057+0000 7fa6b1ffb700 1 -- 172.21.15.63:0/3957538373 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v52)=0 v52) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa6ac05b6e0 con 0x7fa6bc103760 2024-04-24T22:49:36.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 msgr2=0x7fa6a806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 0x7fa6a806fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009b60 tx=0x7fa6b80091a0 comp rx=0 tx=0).stop 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 msgr2=0x7fa6bc113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009100 tx=0x7fa6ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 shutdown_connections 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fa6a806da40 0x7fa6a806fef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6bc102dc0 0x7fa6bc113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 --2- 172.21.15.63:0/3957538373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6bc103760 0x7fa6bc113eb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 >> 172.21.15.63:0/3957538373 conn(0x7fa6bc0f8930 msgr2=0x7fa6bc107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 shutdown_connections 2024-04-24T22:49:36.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:36.059+0000 7fa6c372e700 1 -- 172.21.15.63:0/3957538373 wait complete. 2024-04-24T22:49:36.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:35 smithi192 ceph-mon[33991]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:36.701 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T22:49:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:38.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:37 smithi063 ceph-mon[26912]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:38.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:37 smithi192 ceph-mon[33991]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 -- 172.21.15.63:0/1359887514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 msgr2=0x7f59a01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 --2- 172.21.15.63:0/1359887514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a01031b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5990004940 tx=0x7f5990009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 -- 172.21.15.63:0/1359887514 shutdown_connections 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 --2- 172.21.15.63:0/1359887514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 --2- 172.21.15.63:0/1359887514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a01031b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 -- 172.21.15.63:0/1359887514 >> 172.21.15.63:0/1359887514 conn(0x7f59a00f8990 msgr2=0x7f59a00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 -- 172.21.15.63:0/1359887514 shutdown_connections 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.571+0000 7f59a6275700 1 -- 172.21.15.63:0/1359887514 wait complete. 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 Processor -- start 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 -- start start 2024-04-24T22:49:38.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a0113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a0114550 con 0x7f59a0102de0 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f59a6275700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a010dac0 con 0x7f59a0103780 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f599ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f599ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43072/0 (socket says 172.21.15.63:43072) 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.572+0000 7f599ffff700 1 -- 172.21.15.63:0/3926629120 learned_addr learned my addr 172.21.15.63:0/3926629120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599f7fe700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a0113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599ffff700 1 -- 172.21.15.63:0/3926629120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 msgr2=0x7f59a0113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599ffff700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a0113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599ffff700 1 -- 172.21.15.63:0/3926629120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5990005040 con 0x7f59a0102de0 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599f7fe700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599ffff700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f599000b3b0 tx=0x7f5990009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:38.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f599000e040 con 0x7f59a0102de0 2024-04-24T22:49:38.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a010dd40 con 0x7f59a0102de0 2024-04-24T22:49:38.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f599000bbc0 con 0x7f59a0102de0 2024-04-24T22:49:38.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f599001c3f0 con 0x7f59a0102de0 2024-04-24T22:49:38.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.573+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a010e300 con 0x7f59a0102de0 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.574+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a01000f0 con 0x7f59a0102de0 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.575+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f599000bd30 con 0x7f59a0102de0 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.575+0000 7f599cff9700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 0x7f598806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.575+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f599008e460 con 0x7f59a0102de0 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.575+0000 7f599f7fe700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 0x7f598806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:38.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.576+0000 7f599f7fe700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 0x7f598806fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59a010ec80 tx=0x7f5994008040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:38.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.580+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f599005bb60 con 0x7f59a0102de0 2024-04-24T22:49:38.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.786+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f59a010ded0 con 0x7f59a0102de0 2024-04-24T22:49:38.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.787+0000 7f599cff9700 1 -- 172.21.15.63:0/3926629120 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v52)=0 v52) v1 ==== 135+0+0 (secure 0 0 0) 0x7f59a010ded0 con 0x7f59a0102de0 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 msgr2=0x7f598806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 0x7f598806fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59a010ec80 tx=0x7f5994008040 comp rx=0 tx=0).stop 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 msgr2=0x7f59a0113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f599000b3b0 tx=0x7f5990009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 shutdown_connections 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f598806da40 0x7f598806fef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a0102de0 0x7f59a0113930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 --2- 172.21.15.63:0/3926629120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59a0103780 0x7f59a0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 >> 172.21.15.63:0/3926629120 conn(0x7f59a00f8990 msgr2=0x7f59a00fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 shutdown_connections 2024-04-24T22:49:38.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:38.789+0000 7f59a6275700 1 -- 172.21.15.63:0/3926629120 wait complete. 2024-04-24T22:49:39.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-24T22:49:39.715 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:40.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:39 smithi063 ceph-mon[26912]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:40.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:39 smithi192 ceph-mon[33991]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:41.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 -- 172.21.15.63:0/2403950583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 msgr2=0x7f6db0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/2403950583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0106e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004940 tx=0x7f6da4009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:41.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 -- 172.21.15.63:0/2403950583 shutdown_connections 2024-04-24T22:49:41.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/2403950583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0106e20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/2403950583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 0x7f6db00fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 -- 172.21.15.63:0/2403950583 >> 172.21.15.63:0/2403950583 conn(0x7f6db00f7850 msgr2=0x7f6db00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 -- 172.21.15.63:0/2403950583 shutdown_connections 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.278+0000 7f6db7ba9700 1 -- 172.21.15.63:0/2403950583 wait complete. 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 Processor -- start 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 -- start start 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 0x7f6db0113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01146b0 con 0x7f6db00fe7e0 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db7ba9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010da10 con 0x7f6db00fde40 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db5144700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db5144700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44846/0 (socket says 172.21.15.63:44846) 2024-04-24T22:49:41.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db5144700 1 -- 172.21.15.63:0/1120003535 learned_addr learned my addr 172.21.15.63:0/1120003535 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.279+0000 7f6db5945700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 0x7f6db0113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db5144700 1 -- 172.21.15.63:0/1120003535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 msgr2=0x7f6db0113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db5144700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 0x7f6db0113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db5144700 1 -- 172.21.15.63:0/1120003535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4005040 con 0x7f6db00fe7e0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db5144700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b3b0 tx=0x7f6da4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da400e040 con 0x7f6db00fe7e0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6da4013420 con 0x7f6db00fe7e0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db010dcf0 con 0x7f6db00fe7e0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da401c630 con 0x7f6db00fe7e0 2024-04-24T22:49:41.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.280+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db010e230 con 0x7f6db00fe7e0 2024-04-24T22:49:41.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.281+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db00666f0 con 0x7f6db00fe7e0 2024-04-24T22:49:41.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.281+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f6da4013590 con 0x7f6db00fe7e0 2024-04-24T22:49:41.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.282+0000 7f6dae7fc700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 0x7f6d9806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:41.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.282+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6da408e350 con 0x7f6db00fe7e0 2024-04-24T22:49:41.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.282+0000 7f6db5945700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 0x7f6d9806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:41.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.282+0000 7f6db5945700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 0x7f6d9806ff00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007580 tx=0x7f6da8009780 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:41.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.286+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6da405bb70 con 0x7f6db00fe7e0 2024-04-24T22:49:41.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.515+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7f6db0109f20 con 0x7f6d9806da50 2024-04-24T22:49:41.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.520+0000 7f6dae7fc700 1 -- 172.21.15.63:0/1120003535 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f6da800f040 con 0x7f6d9806da50 2024-04-24T22:49:41.523 INFO:teuthology.orchestra.run.smithi063.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:49:41.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 msgr2=0x7f6d9806ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 0x7f6d9806ff00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007580 tx=0x7f6da8009780 comp rx=0 tx=0).stop 2024-04-24T22:49:41.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 msgr2=0x7f6db0113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b3b0 tx=0x7f6da4009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 shutdown_connections 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f6d9806da50 0x7f6d9806ff00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db00fde40 0x7f6db0113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 --2- 172.21.15.63:0/1120003535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db00fe7e0 0x7f6db0113fd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:41.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.522+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 >> 172.21.15.63:0/1120003535 conn(0x7f6db00f7850 msgr2=0x7f6db00f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:41.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.523+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 shutdown_connections 2024-04-24T22:49:41.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:41.523+0000 7f6db7ba9700 1 -- 172.21.15.63:0/1120003535 wait complete. 2024-04-24T22:49:42.160 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:41 smithi063 ceph-mon[26912]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:42.160 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:42.160 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:42.160 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:41 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:42.187 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T22:49:42.187 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:49:42.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-24T22:49:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:41 smithi192 ceph-mon[33991]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:42.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:41 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:49:42.460 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:49:43.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:43.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:49:43.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:43.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:43.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:43.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:42 smithi063 ceph-mon[26912]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:49:43.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:43.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:49:43.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:43.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:49:43.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:43.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:42 smithi192 ceph-mon[33991]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:49:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2232563616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 msgr2=0x7fd1c8070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2232563616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8070670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8004970 tx=0x7fd1b8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2232563616 shutdown_connections 2024-04-24T22:49:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2232563616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1c8070bb0 0x7fd1c8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2232563616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8070670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2232563616 >> 172.21.15.63:0/2232563616 conn(0x7fd1c80f8980 msgr2=0x7fd1c80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2232563616 shutdown_connections 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2232563616 wait complete. 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.131+0000 7fd1ce3c7700 1 Processor -- start 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1ce3c7700 1 -- start start 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1ce3c7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1ce3c7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1c8070bb0 0x7fd1c8116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1ce3c7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8116890 con 0x7fd1c80702a0 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1ce3c7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c810fca0 con 0x7fd1c8070bb0 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44858/0 (socket says 172.21.15.63:44858) 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 -- 172.21.15.63:0/2408988054 learned_addr learned my addr 172.21.15.63:0/2408988054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 -- 172.21.15.63:0/2408988054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1c8070bb0 msgr2=0x7fd1c8116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1c8070bb0 0x7fd1c8116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 -- 172.21.15.63:0/2408988054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8005040 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c7fff700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000c00 tx=0x7fd1b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.132+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1b800e040 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.133+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1b800bc10 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.133+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c810ff20 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.133+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1b801c3f0 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.133+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8110460 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.134+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80719e0 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.134+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fd1b800bd80 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.135+0000 7fd1c4ff9700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 0x7fd1b006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.135+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd1b808e500 con 0x7fd1c80702a0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.135+0000 7fd1c77fe700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 0x7fd1b006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.135+0000 7fd1c77fe700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 0x7fd1b006fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81115e0 tx=0x7fd1bc007cc0 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.140+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd1b8058150 con 0x7fd1c80702a0 2024-04-24T22:49:44.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.373+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1c810c130 con 0x7fd1b006da40 2024-04-24T22:49:44.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.374+0000 7fd1c4ff9700 1 -- 172.21.15.63:0/2408988054 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd1c810c130 con 0x7fd1b006da40 2024-04-24T22:49:44.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.376+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 msgr2=0x7fd1b006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.376+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 0x7fd1b006fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81115e0 tx=0x7fd1bc007cc0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.376+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 msgr2=0x7fd1c8115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.376+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000c00 tx=0x7fd1b8009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 shutdown_connections 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fd1b006da40 0x7fd1b006fef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1c80702a0 0x7fd1c8115d00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 --2- 172.21.15.63:0/2408988054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1c8070bb0 0x7fd1c8116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 >> 172.21.15.63:0/2408988054 conn(0x7fd1c80f8980 msgr2=0x7fd1c80f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 shutdown_connections 2024-04-24T22:49:44.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.377+0000 7fd1ce3c7700 1 -- 172.21.15.63:0/2408988054 wait complete. 2024-04-24T22:49:44.393 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:49:44.393 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:44 smithi063 ceph-mon[26912]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:44.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 -- 172.21.15.63:0/3140188392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fc790 msgr2=0x7f24840fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3140188392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fc790 0x7f24840fcb60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2474004940 tx=0x7f2474009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 -- 172.21.15.63:0/3140188392 shutdown_connections 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3140188392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fd0a0 0x7f2484109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3140188392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fc790 0x7f24840fcb60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 -- 172.21.15.63:0/3140188392 >> 172.21.15.63:0/3140188392 conn(0x7f2484071d00 msgr2=0x7f2484072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 -- 172.21.15.63:0/3140188392 shutdown_connections 2024-04-24T22:49:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.504+0000 7f2488cb1700 1 -- 172.21.15.63:0/3140188392 wait complete. 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 Processor -- start 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 -- start start 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fc790 0x7f2484115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484116b10 con 0x7f24840fd0a0 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2488cb1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248410fe90 con 0x7f24840fc790 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2481d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2481d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44872/0 (socket says 172.21.15.63:44872) 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.505+0000 7f2481d9b700 1 -- 172.21.15.63:0/3008680523 learned_addr learned my addr 172.21.15.63:0/3008680523 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f248259c700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fc790 0x7f2484115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2481d9b700 1 -- 172.21.15.63:0/3008680523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fc790 msgr2=0x7f2484115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2481d9b700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fc790 0x7f2484115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2481d9b700 1 -- 172.21.15.63:0/3008680523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474005040 con 0x7f24840fd0a0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2481d9b700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f247800b580 tx=0x7f24780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247800f6f0 con 0x7f24840fd0a0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f247800fd20 con 0x7f24840fd0a0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484110170 con 0x7f24840fd0a0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.506+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2478011ea0 con 0x7f24840fd0a0 2024-04-24T22:49:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.507+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484110660 con 0x7f24840fd0a0 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.508+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f247800f850 con 0x7f24840fd0a0 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.508+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24840fc330 con 0x7f24840fd0a0 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.508+0000 7f2472ffd700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 0x7f246c06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.508+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f247808ce80 con 0x7f24840fd0a0 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.508+0000 7f248259c700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 0x7f246c06ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.509+0000 7f248259c700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 0x7f246c06ff00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247400b3b0 tx=0x7f2474009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.513+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2478056ad0 con 0x7f24840fd0a0 2024-04-24T22:49:44.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:44 smithi192 ceph-mon[33991]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:44.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.747+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f248410c1a0 con 0x7f246c06da50 2024-04-24T22:49:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.749+0000 7f2472ffd700 1 -- 172.21.15.63:0/3008680523 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f248410c1a0 con 0x7f246c06da50 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 msgr2=0x7f246c06ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 0x7f246c06ff00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247400b3b0 tx=0x7f2474009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 msgr2=0x7f2484116430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f247800b580 tx=0x7f24780091a0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 shutdown_connections 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f246c06da50 0x7f246c06ff00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840fc790 0x7f2484115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 --2- 172.21.15.63:0/3008680523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24840fd0a0 0x7f2484116430 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 >> 172.21.15.63:0/3008680523 conn(0x7f2484071d00 msgr2=0x7f24840f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:44.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 shutdown_connections 2024-04-24T22:49:44.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.751+0000 7f2488cb1700 1 -- 172.21.15.63:0/3008680523 wait complete. 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 -- 172.21.15.63:0/4228351545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48065640 msgr2=0x7fab48065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 --2- 172.21.15.63:0/4228351545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48065640 0x7fab48065ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fab38004940 tx=0x7fab38009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 -- 172.21.15.63:0/4228351545 shutdown_connections 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 --2- 172.21.15.63:0/4228351545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48065640 0x7fab48065ab0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 --2- 172.21.15.63:0/4228351545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48102e50 0x7fab48103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 -- 172.21.15.63:0/4228351545 >> 172.21.15.63:0/4228351545 conn(0x7fab48071d00 msgr2=0x7fab48072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 -- 172.21.15.63:0/4228351545 shutdown_connections 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.862+0000 7fab4cebd700 1 -- 172.21.15.63:0/4228351545 wait complete. 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 Processor -- start 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 -- start start 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48065640 0x7fab48113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab481147a0 con 0x7fab48102e50 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4cebd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4810db00 con 0x7fab48065640 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab45d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab45d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44878/0 (socket says 172.21.15.63:44878) 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab45d9b700 1 -- 172.21.15.63:0/481797153 learned_addr learned my addr 172.21.15.63:0/481797153 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.863+0000 7fab4659c700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48065640 0x7fab48113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab45d9b700 1 -- 172.21.15.63:0/481797153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48065640 msgr2=0x7fab48113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab45d9b700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48065640 0x7fab48113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:44.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab45d9b700 1 -- 172.21.15.63:0/481797153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38005040 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab45d9b700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b3b0 tx=0x7fab38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3800e040 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab3800bbc0 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4810dd80 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4810e340 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.864+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3801c3f0 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.865+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab480fe540 con 0x7fab48102e50 2024-04-24T22:49:44.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.866+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fab3800bd30 con 0x7fab48102e50 2024-04-24T22:49:44.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.866+0000 7fab36ffd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 0x7fab3006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:44.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.866+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fab3808e340 con 0x7fab48102e50 2024-04-24T22:49:44.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.866+0000 7fab4659c700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 0x7fab3006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:44.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.867+0000 7fab4659c700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 0x7fab3006ffc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab3c006fd0 tx=0x7fab3c008040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:44.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:44.871+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fab3805baa0 con 0x7fab48102e50 2024-04-24T22:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.091+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab48109f70 con 0x7fab3006db10 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.109+0000 7fab36ffd700 1 -- 172.21.15.63:0/481797153 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fab48109f70 con 0x7fab3006db10 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (88s) 66s ago 12m 14.9M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (13m) 66s ago 13m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (11m) 2m ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (72s) 66s ago 12m 51.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (7m) 66s ago 7m 14.6M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (7m) 66s ago 7m 12.9M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (7m) 2m ago 7m 151M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (7m) 2m ago 7m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:49:45.111 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (3m) 66s ago 14m 508M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (2m) 2m ago 11m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (14m) 66s ago 14m 52.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2d7c607af9e7 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (11m) 2m ago 11m 43.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 61cd8ff0148c 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (2m) 66s ago 12m 8108k - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 2m ago 11m 3586k - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 66s ago 10m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (10m) 66s ago 10m 215M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 66s ago 9m 180M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (9m) 2m ago 9m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (9m) 2m ago 9m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (8m) 2m ago 8m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:49:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (105s) 66s ago 12m 44.6M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:49:45.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 msgr2=0x7fab3006ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 0x7fab3006ffc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab3c006fd0 tx=0x7fab3c008040 comp rx=0 tx=0).stop 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 msgr2=0x7fab481140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b3b0 tx=0x7fab38009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 shutdown_connections 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fab3006db10 0x7fab3006ffc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab48065640 0x7fab48113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 --2- 172.21.15.63:0/481797153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab48102e50 0x7fab481140c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 >> 172.21.15.63:0/481797153 conn(0x7fab48071d00 msgr2=0x7fab480fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 shutdown_connections 2024-04-24T22:49:45.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.112+0000 7fab4cebd700 1 -- 172.21.15.63:0/481797153 wait complete. 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 -- 172.21.15.63:0/4027314606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978102680 msgr2=0x7fe9780fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 --2- 172.21.15.63:0/4027314606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978102680 0x7fe9780fbcf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe968004940 tx=0x7fe968009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 -- 172.21.15.63:0/4027314606 shutdown_connections 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 --2- 172.21.15.63:0/4027314606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978102680 0x7fe9780fbcf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 --2- 172.21.15.63:0/4027314606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978101ce0 0x7fe9781020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 -- 172.21.15.63:0/4027314606 >> 172.21.15.63:0/4027314606 conn(0x7fe9780f7850 msgr2=0x7fe9780f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 -- 172.21.15.63:0/4027314606 shutdown_connections 2024-04-24T22:49:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 -- 172.21.15.63:0/4027314606 wait complete. 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.217+0000 7fe97fab8700 1 Processor -- start 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97fab8700 1 -- start start 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97fab8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97fab8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978102680 0x7fe97810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97fab8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97810f6d0 con 0x7fe978101ce0 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97fab8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97810f810 con 0x7fe978102680 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44898/0 (socket says 172.21.15.63:44898) 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 -- 172.21.15.63:0/2390822472 learned_addr learned my addr 172.21.15.63:0/2390822472 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d053700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978102680 0x7fe97810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 -- 172.21.15.63:0/2390822472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978102680 msgr2=0x7fe97810f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978102680 0x7fe97810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 -- 172.21.15.63:0/2390822472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968005040 con 0x7fe978101ce0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.218+0000 7fe97d854700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe974002a50 tx=0x7fe9740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.219+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe974007cc0 con 0x7fe978101ce0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.219+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe978117bb0 con 0x7fe978101ce0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.219+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe97400f040 con 0x7fe978101ce0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.219+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe974009640 con 0x7fe978101ce0 2024-04-24T22:49:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.219+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978118110 con 0x7fe978101ce0 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.220+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9780fd430 con 0x7fe978101ce0 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.220+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fe9740097a0 con 0x7fe978101ce0 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.221+0000 7fe96e7fc700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 0x7fe96406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.221+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe97408cfc0 con 0x7fe978101ce0 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.221+0000 7fe97d053700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 0x7fe96406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.221+0000 7fe97d053700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 0x7fe96406fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe96800b3b0 tx=0x7fe968009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:45.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.225+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe974056b90 con 0x7fe978101ce0 2024-04-24T22:49:45.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:45 smithi063 ceph-mon[26912]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.538+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe978066620 con 0x7fe978101ce0 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.539+0000 7fe96e7fc700 1 -- 172.21.15.63:0/2390822472 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe97405a250 con 0x7fe978101ce0 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:49:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:49:45.542 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 msgr2=0x7fe96406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 0x7fe96406fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe96800b3b0 tx=0x7fe968009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 msgr2=0x7fe97810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe974002a50 tx=0x7fe9740046b0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 shutdown_connections 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7fe96406da40 0x7fe96406fef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe978101ce0 0x7fe97810eb40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 --2- 172.21.15.63:0/2390822472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe978102680 0x7fe97810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 >> 172.21.15.63:0/2390822472 conn(0x7fe9780f7850 msgr2=0x7fe9780f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 shutdown_connections 2024-04-24T22:49:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.541+0000 7fe97fab8700 1 -- 172.21.15.63:0/2390822472 wait complete. 2024-04-24T22:49:45.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.647+0000 7f64f29f5700 1 -- 172.21.15.63:0/2656513214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 msgr2=0x7f64ec0fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.647+0000 7f64f29f5700 1 --2- 172.21.15.63:0/2656513214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec0fb970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f64dc004940 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 -- 172.21.15.63:0/2656513214 shutdown_connections 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 --2- 172.21.15.63:0/2656513214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 --2- 172.21.15.63:0/2656513214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec0fb970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 -- 172.21.15.63:0/2656513214 >> 172.21.15.63:0/2656513214 conn(0x7f64ec073350 msgr2=0x7f64ec073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 -- 172.21.15.63:0/2656513214 shutdown_connections 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 -- 172.21.15.63:0/2656513214 wait complete. 2024-04-24T22:49:45.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 Processor -- start 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.648+0000 7f64f29f5700 1 -- start start 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64f29f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64f29f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64f29f5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec116800 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64f29f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec10fd50 con 0x7f64ec0fbf40 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44924/0 (socket says 172.21.15.63:44924) 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 -- 172.21.15.63:0/4155360836 learned_addr learned my addr 172.21.15.63:0/4155360836 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64eb7fe700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 -- 172.21.15.63:0/4155360836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 msgr2=0x7f64ec116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec116120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 -- 172.21.15.63:0/4155360836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc005040 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64eb7fe700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.649+0000 7f64ebfff700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b3b0 tx=0x7f64dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.650+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc00e030 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.650+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64dc00bde0 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.650+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc013e50 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.650+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec10ff70 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.650+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec1104b0 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.651+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec0fe670 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.651+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f64dc025400 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.652+0000 7f64e8ff9700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 0x7f64d406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.652+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f64dc08eb30 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.652+0000 7f64eb7fe700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 0x7f64d406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:45.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.652+0000 7f64eb7fe700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 0x7f64d406fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64ec111480 tx=0x7f64e0008040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:45.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.657+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f64dc05c3c0 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:45 smithi192 ceph-mon[33991]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.922+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64ec111230 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.923+0000 7f64e8ff9700 1 -- 172.21.15.63:0/4155360836 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1809 (secure 0 0 0) 0x7f64dc020070 con 0x7f64ec0fb5a0 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:49:45.925 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:49:45.926 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:45.927 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 msgr2=0x7f64d406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 0x7f64d406fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64ec111480 tx=0x7f64e0008040 comp rx=0 tx=0).stop 2024-04-24T22:49:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 msgr2=0x7f64ec115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b3b0 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 shutdown_connections 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f64d406da40 0x7f64d406fef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64ec0fb5a0 0x7f64ec115be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 --2- 172.21.15.63:0/4155360836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64ec0fbf40 0x7f64ec116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 >> 172.21.15.63:0/4155360836 conn(0x7f64ec073350 msgr2=0x7f64ec0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 shutdown_connections 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:45.925+0000 7f64f29f5700 1 -- 172.21.15.63:0/4155360836 wait complete. 2024-04-24T22:49:45.929 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 -- 172.21.15.63:0/660774657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 msgr2=0x7f6588102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 --2- 172.21.15.63:0/660774657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f6588102160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6584004940 tx=0x7f6584009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 -- 172.21.15.63:0/660774657 shutdown_connections 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 --2- 172.21.15.63:0/660774657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 0x7f65880fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 --2- 172.21.15.63:0/660774657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f6588102160 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 -- 172.21.15.63:0/660774657 >> 172.21.15.63:0/660774657 conn(0x7f6588071d00 msgr2=0x7f6588072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 -- 172.21.15.63:0/660774657 shutdown_connections 2024-04-24T22:49:46.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 -- 172.21.15.63:0/660774657 wait complete. 2024-04-24T22:49:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.034+0000 7f658f5aa700 1 Processor -- start 2024-04-24T22:49:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658f5aa700 1 -- start start 2024-04-24T22:49:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658f5aa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 0x7f6588115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658f5aa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658f5aa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881168f0 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658f5aa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658810fcc0 con 0x7f65880fbd40 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44946/0 (socket says 172.21.15.63:44946) 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 -- 172.21.15.63:0/1815036537 learned_addr learned my addr 172.21.15.63:0/1815036537 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658d346700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 0x7f6588115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 -- 172.21.15.63:0/1815036537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 msgr2=0x7f6588115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 0x7f6588115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 -- 172.21.15.63:0/1815036537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584005040 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.035+0000 7f658cb45700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6578002a50 tx=0x7f65780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.036+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6578007cc0 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.036+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657800f040 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.036+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658810ffa0 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.036+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657800da30 con 0x7f6588101d90 2024-04-24T22:49:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.036+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6588110560 con 0x7f6588101d90 2024-04-24T22:49:46.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.037+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65880fd430 con 0x7f6588101d90 2024-04-24T22:49:46.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.037+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f657800db90 con 0x7f6588101d90 2024-04-24T22:49:46.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.038+0000 7f657dffb700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 0x7f657406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.038+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f657808cf60 con 0x7f6588101d90 2024-04-24T22:49:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.038+0000 7f658d346700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 0x7f657406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.039+0000 7f658d346700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 0x7f657406fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6584004560 tx=0x7f6584009e30 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.043+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f65780c1100 con 0x7f6588101d90 2024-04-24T22:49:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.275+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 --> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6588108e60 con 0x7f657406da40 2024-04-24T22:49:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.276+0000 7f657dffb700 1 -- 172.21.15.63:0/1815036537 <== mgr.14690 v2:172.21.15.63:6800/3227237455 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f6588108e60 con 0x7f657406da40 2024-04-24T22:49:46.277 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/21 daemons upgraded", 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:49:46.278 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:49:46.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 msgr2=0x7f657406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 0x7f657406fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6584004560 tx=0x7f6584009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:46.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 msgr2=0x7f65881162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6578002a50 tx=0x7f65780046b0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 shutdown_connections 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f657406da40 0x7f657406fef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65880fbd40 0x7f6588115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 --2- 172.21.15.63:0/1815036537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6588101d90 0x7f65881162a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 >> 172.21.15.63:0/1815036537 conn(0x7f6588071d00 msgr2=0x7f65880655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 shutdown_connections 2024-04-24T22:49:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.278+0000 7f658f5aa700 1 -- 172.21.15.63:0/1815036537 wait complete. 2024-04-24T22:49:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- 172.21.15.63:0/1411148861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 msgr2=0x7f20841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 --2- 172.21.15.63:0/1411148861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f20841031d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2078004940 tx=0x7f2078009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- 172.21.15.63:0/1411148861 shutdown_connections 2024-04-24T22:49:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 --2- 172.21.15.63:0/1411148861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 --2- 172.21.15.63:0/1411148861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f20841031d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- 172.21.15.63:0/1411148861 >> 172.21.15.63:0/1411148861 conn(0x7f20840f89d0 msgr2=0x7f20840fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- 172.21.15.63:0/1411148861 shutdown_connections 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- 172.21.15.63:0/1411148861 wait complete. 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 Processor -- start 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.387+0000 7f208a6dc700 1 -- start start 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f208a6dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f208a6dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f208a6dc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2084072240 con 0x7f2084102e00 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f208a6dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2084070ce0 con 0x7f20841037a0 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44960/0 (socket says 172.21.15.63:44960) 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 -- 172.21.15.63:0/119605736 learned_addr learned my addr 172.21.15.63:0/119605736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f20837fe700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 -- 172.21.15.63:0/119605736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 msgr2=0x7f20840707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 -- 172.21.15.63:0/119605736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078005040 con 0x7f2084102e00 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f20837fe700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2083fff700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f207800b3b0 tx=0x7f2078009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f207800e040 con 0x7f2084102e00 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f207800bbc0 con 0x7f2084102e00 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2084070f60 con 0x7f2084102e00 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.388+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f207801c3f0 con 0x7f2084102e00 2024-04-24T22:49:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.389+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208411a150 con 0x7f2084102e00 2024-04-24T22:49:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.390+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20840fe550 con 0x7f2084102e00 2024-04-24T22:49:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.390+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f207800bd30 con 0x7f2084102e00 2024-04-24T22:49:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.391+0000 7f2080ff9700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 0x7f206c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:49:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.391+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f207808e410 con 0x7f2084102e00 2024-04-24T22:49:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.391+0000 7f20837fe700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 0x7f206c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:49:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.391+0000 7f20837fe700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 0x7f206c06fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2084071990 tx=0x7f2074008040 comp rx=0 tx=0).ready entity=mgr.14690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:49:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.395+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2078057fe0 con 0x7f2084102e00 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/2390822472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:46.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:46.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:46.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:46.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T22:49:46.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:46 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/4155360836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:49:46.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:46.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/2390822472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T22:49:46.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:46 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/4155360836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:49:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.698+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f208411a340 con 0x7f2084102e00 2024-04-24T22:49:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.698+0000 7f2080ff9700 1 -- 172.21.15.63:0/119605736 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f207805b6a0 con 0x7f2084102e00 2024-04-24T22:49:46.700 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:46.700 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:46.701 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:49:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 msgr2=0x7f206c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 0x7f206c06fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2084071990 tx=0x7f2074008040 comp rx=0 tx=0).stop 2024-04-24T22:49:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 msgr2=0x7f2084070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:49:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f207800b3b0 tx=0x7f2078009e30 comp rx=0 tx=0).stop 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 shutdown_connections 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:6800/3227237455,v1:172.21.15.63:6801/3227237455] conn(0x7f206c06da40 0x7f206c06fef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2084102e00 0x7f2084070260 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 --2- 172.21.15.63:0/119605736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20841037a0 0x7f20840707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 >> 172.21.15.63:0/119605736 conn(0x7f20840f89d0 msgr2=0x7f20840f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 shutdown_connections 2024-04-24T22:49:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:49:46.701+0000 7f208a6dc700 1 -- 172.21.15.63:0/119605736 wait complete. 2024-04-24T22:49:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:47 smithi063 ceph-mon[26912]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-24T22:49:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:47 smithi063 ceph-mon[26912]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T22:49:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:47 smithi063 ceph-mon[26912]: Upgrade: Setting container_image for all mgr 2024-04-24T22:49:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:47 smithi063 ceph-mon[26912]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:47 smithi063 ceph-mon[26912]: from='client.? 172.21.15.63:0/119605736' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:49:47.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:47 smithi192 ceph-mon[33991]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-24T22:49:47.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:47 smithi192 ceph-mon[33991]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T22:49:47.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:47 smithi192 ceph-mon[33991]: Upgrade: Setting container_image for all mgr 2024-04-24T22:49:47.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:47 smithi192 ceph-mon[33991]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:49:47.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:47 smithi192 ceph-mon[33991]: from='client.? 172.21.15.63:0/119605736' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:49:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: Upgrade: Updating mon.smithi063 2024-04-24T22:49:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:49:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:49:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:48.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:48 smithi063 ceph-mon[26912]: Deploying daemon mon.smithi063 on smithi063 2024-04-24T22:49:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: Upgrade: Updating mon.smithi063 2024-04-24T22:49:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' 2024-04-24T22:49:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:49:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:49:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:49:48.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:48 smithi192 ceph-mon[33991]: Deploying daemon mon.smithi063 on smithi063 2024-04-24T22:49:49.541 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:49 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:49:49.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:49 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[26908]: 2024-04-24T22:49:49.538+0000 7fd0324df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:49:49.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:49 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[26908]: 2024-04-24T22:49:49.538+0000 7fd0324df700 -1 mon.smithi063@0(leader) e2 *** Got Signal Terminated *** 2024-04-24T22:49:49.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:49 smithi063 podman[111036]: 2024-04-24 22:49:49.773965925 +0000 UTC m=+0.354780330 container died 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T22:49:50.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 podman[111036]: 2024-04-24 22:49:50.089115446 +0000 UTC m=+0.669929857 container cleanup 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T22:49:50.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 bash[111036]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063 2024-04-24T22:49:50.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 podman[111061]: 2024-04-24 22:49:50.431851781 +0000 UTC m=+0.659074533 container remove 2d7c607af9e763f5f7e66ffcb753f9a359aecced7073f6e3869f54ccd2d68b1d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:49:50.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service: Deactivated successfully. 2024-04-24T22:49:50.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:49:50.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service: Consumed 14.208s CPU time. 2024-04-24T22:49:51.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:50 smithi063 systemd[1]: Starting Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 podman[111194]: 2024-04-24 22:49:51.006194553 +0000 UTC m=+0.018378289 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 podman[111194]: 2024-04-24 22:49:51.107958216 +0000 UTC m=+0.120141994 container create 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 podman[111194]: 2024-04-24 22:49:51.318546419 +0000 UTC m=+0.330730155 container init 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 podman[111194]: 2024-04-24 22:49:51.321663833 +0000 UTC m=+0.333847569 container start 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: pidfile_write: ignore empty --pid-file 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: load: jerasure load: lrc 2024-04-24T22:49:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: RocksDB version: 7.9.2 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Git sha 0 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: DB SUMMARY 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: DB Session ID: SN8YS0C7KOTV787JJ03S 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: CURRENT file: CURRENT 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1159 Bytes 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000033.sst 2024-04-24T22:49:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000031.log size: 2140686 ; 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.error_if_exists: 0 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.create_if_missing: 0 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.paranoid_checks: 1 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T22:49:51.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.env: 0x561b928c2c60 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.info_log: 0x561b935df960 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.statistics: (nil) 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.use_fsync: 0 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_log_file_size: 0 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_fallocate: 1 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T22:49:51.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.db_log_dir: 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.wal_dir: 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:49:51.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.write_buffer_manager: 0x561b935b21e0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.unordered_write: 0 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.row_cache: None 2024-04-24T22:49:51.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.wal_filter: None 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.wal_compression: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:49:51.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:49:51.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_open_files: -1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Compression algorithms supported: 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kZSTD supported: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:49:51.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.merge_operator: 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_filter: None 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561b935dfa60) 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:49:51.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 4 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x561b935e3090 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-04-24T22:49:51.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: bloomfilter 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 5 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-04-24T22:49:51.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_auto_readahead_size: 262144 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: prepopulate_block_cache: 0 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: initial_auto_readahead_size: 8192 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression: NoCompression 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.num_levels: 7 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:49:51.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:49:51.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:49:51.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:49:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:49:51.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.ttl: 2592000 2024-04-24T22:49:51.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T22:49:51.807 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12001, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2b59f023-ef53-4379-9a33-e15e7967a456 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998991351867, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998991361658, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1892673, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11995, "largest_seqno": 13709, "table_properties": {"data_size": 1884819, "index_size": 4918, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 18285, "raw_average_key_size": 23, "raw_value_size": 1868429, "raw_average_value_size": 2439, "num_data_blocks": 224, "num_entries": 766, "num_filter_entries": 766, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713998991, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2b59f023-ef53-4379-9a33-e15e7967a456", "db_session_id": "SN8YS0C7KOTV787JJ03S", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713998991361775, "job": 1, "event": "recovery_finished"} 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561b936d4000 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: DB pointer 0x561b936be000 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T22:49:51.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 1/0 1.80 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L6 1/0 6.88 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 8.69 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-24T22:49:51.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 506.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:49:51.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 144.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 144.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache BinnedLRUCache@0x561b935e3090#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???) e2 preinit fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).mds e14 new map 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).mds e14 print_map 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e14 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: 1 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Filesystem 'cephfs' (1) 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: fs_name cephfs 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: epoch 14 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: tableserver 0 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: root 0 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_timeout 60 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_autoclose 300 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_file_size 1099511627776 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_xattr_size 65536 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: required_client_features {} 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure 0 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure_osd_epoch 46 2024-04-24T22:49:51.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_mds 1 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: in 0 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: up {0=24253} 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: failed 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: damaged 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: stopped 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_pools [3] 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_pool 2 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: inline_data enabled 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: balancer 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: bal_rank_mask -1 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: standby_count_wanted 1 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: qdb_cluster leader: 0 members: 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:51.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Standby daemons: 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 ceph-mon[111213]: mon.smithi063@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 bash[111194]: 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 2024-04-24T22:49:51.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:51 smithi063 systemd[1]: Started Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:49:53.159 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: mon.smithi063 calling monitor election 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: monmap epoch 2 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: last_changed 2024-04-24T22:38:07.030569+0000 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: created 2024-04-24T22:35:18.856966+0000 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: min_mon_release 17 (quincy) 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: election_strategy: 1 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-04-24T22:49:53.160 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.smithi192 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: mgrmap e34: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: from='mgr.14690 ' entity='' 2024-04-24T22:49:53.161 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:52 smithi192 ceph-mon[33991]: mgrmap e35: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: from='mgr.14690 172.21.15.63:0/3153198258' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: mon.smithi063 calling monitor election 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: monmap epoch 2 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: last_changed 2024-04-24T22:38:07.030569+0000 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: created 2024-04-24T22:35:18.856966+0000 2024-04-24T22:49:53.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: min_mon_release 17 (quincy) 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: election_strategy: 1 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.smithi192 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: mgrmap e34: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: from='mgr.14690 ' entity='' 2024-04-24T22:49:53.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:52 smithi063 ceph-mon[111213]: mgrmap e35: smithi063.upcnvd(active, since 2m), standbys: smithi192.ctmwwc 2024-04-24T22:49:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:49:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:49:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:49:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:49:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:49:59.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:49:59 smithi192 ceph-mon[33991]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:50:00.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: Standby manager daemon smithi192.ctmwwc restarted 2024-04-24T22:50:00.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: Standby manager daemon smithi192.ctmwwc started 2024-04-24T22:50:00.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/crt"}]: dispatch 2024-04-24T22:50:00.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T22:50:00.020 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.ctmwwc/key"}]: dispatch 2024-04-24T22:50:00.020 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:49:59 smithi063 ceph-mon[111213]: from='mgr.? 172.21.15.192:0/820120990' entity='mgr.smithi192.ctmwwc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: mgrmap e36: smithi063.upcnvd(active, since 3m), standbys: smithi192.ctmwwc 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: mgrmap e37: smithi063.upcnvd(active, starting, since 0.010416s), standbys: smithi192.ctmwwc 2024-04-24T22:50:01.038 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:50:01.039 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:50:01.040 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: mgrmap e36: smithi063.upcnvd(active, since 3m), standbys: smithi192.ctmwwc 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: Active manager daemon smithi063.upcnvd restarted 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: Activating manager daemon smithi063.upcnvd 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: mgrmap e37: smithi063.upcnvd(active, starting, since 0.010416s), standbys: smithi192.ctmwwc 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:50:01.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi063.upcnvd", "id": "smithi063.upcnvd"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr metadata", "who": "smithi192.ctmwwc", "id": "smithi192.ctmwwc"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: Manager daemon smithi063.upcnvd is now available 2024-04-24T22:50:01.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:01.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:01.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/mirror_snapshot_schedule"}]: dispatch 2024-04-24T22:50:01.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:00 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.upcnvd/trash_purge_schedule"}]: dispatch 2024-04-24T22:50:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:02 smithi192 ceph-mon[33991]: mgrmap e38: smithi063.upcnvd(active, since 1.01167s), standbys: smithi192.ctmwwc 2024-04-24T22:50:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:02 smithi192 ceph-mon[33991]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:02 smithi063 ceph-mon[111213]: mgrmap e38: smithi063.upcnvd(active, since 1.01167s), standbys: smithi192.ctmwwc 2024-04-24T22:50:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:02 smithi063 ceph-mon[111213]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:03 smithi063 ceph-mon[111213]: mgrmap e39: smithi063.upcnvd(active, since 2s), standbys: smithi192.ctmwwc 2024-04-24T22:50:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:03 smithi063 ceph-mon[111213]: [24/Apr/2024:22:50:02] ENGINE Bus STARTING 2024-04-24T22:50:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:03 smithi063 ceph-mon[111213]: [24/Apr/2024:22:50:02] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:50:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:03 smithi063 ceph-mon[111213]: [24/Apr/2024:22:50:02] ENGINE Serving on http://172.21.15.63:8765 2024-04-24T22:50:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:03 smithi063 ceph-mon[111213]: [24/Apr/2024:22:50:02] ENGINE Bus STARTED 2024-04-24T22:50:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:03 smithi192 ceph-mon[33991]: mgrmap e39: smithi063.upcnvd(active, since 2s), standbys: smithi192.ctmwwc 2024-04-24T22:50:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:03 smithi192 ceph-mon[33991]: [24/Apr/2024:22:50:02] ENGINE Bus STARTING 2024-04-24T22:50:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:03 smithi192 ceph-mon[33991]: [24/Apr/2024:22:50:02] ENGINE Serving on https://172.21.15.63:7150 2024-04-24T22:50:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:03 smithi192 ceph-mon[33991]: [24/Apr/2024:22:50:02] ENGINE Serving on http://172.21.15.63:8765 2024-04-24T22:50:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:03 smithi192 ceph-mon[33991]: [24/Apr/2024:22:50:02] ENGINE Bus STARTED 2024-04-24T22:50:04.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:04 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:04.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:04 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:04.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:04 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:04.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:04 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:05 smithi063 ceph-mon[111213]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:05 smithi063 ceph-mon[111213]: mgrmap e40: smithi063.upcnvd(active, since 4s), standbys: smithi192.ctmwwc 2024-04-24T22:50:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:05 smithi192 ceph-mon[33991]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:05 smithi192 ceph-mon[33991]: mgrmap e40: smithi063.upcnvd(active, since 4s), standbys: smithi192.ctmwwc 2024-04-24T22:50:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:05 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:05 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:06 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:06.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:06 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:06 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:07.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:06 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:07 smithi063 ceph-mon[111213]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:07 smithi192 ceph-mon[33991]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:07 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:07 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:09.735 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:09 smithi192 ceph-mon[33991]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-24T22:50:10.037 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:09 smithi063 ceph-mon[111213]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-24T22:50:11.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T22:50:11.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Detected new or changed devices on smithi063 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:11 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T22:50:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Detected new or changed devices on smithi063 2024-04-24T22:50:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi063:/etc/ceph/ceph.conf 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: Updating smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:11.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:12.409 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:12.410 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:12 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: Updating smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T22:50:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:13 smithi192 ceph-mon[33991]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T22:50:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:13 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:13 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:13 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:13.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:13 smithi192 ceph-mon[33991]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:13 smithi063 ceph-mon[111213]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T22:50:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:13.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:14 smithi192 ceph-mon[33991]: Upgrade: Updating mon.smithi192 2024-04-24T22:50:14.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:14 smithi192 ceph-mon[33991]: Deploying daemon mon.smithi192 on smithi192 2024-04-24T22:50:14.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:14 smithi063 ceph-mon[111213]: Upgrade: Updating mon.smithi192 2024-04-24T22:50:14.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:14 smithi063 ceph-mon[111213]: Deploying daemon mon.smithi192 on smithi192 2024-04-24T22:50:15.134 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:14 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:50:15.134 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192[33931]: 2024-04-24T22:50:15.010+0000 7f882cd60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:50:15.134 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192[33931]: 2024-04-24T22:50:15.010+0000 7f882cd60700 -1 mon.smithi192@1(peon) e2 *** Got Signal Terminated *** 2024-04-24T22:50:15.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 podman[80295]: 2024-04-24 22:50:15.132976371 +0000 UTC m=+0.217366390 container died 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c) 2024-04-24T22:50:15.817 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 podman[80295]: 2024-04-24 22:50:15.527005807 +0000 UTC m=+0.611395804 container cleanup 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:50:15.817 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 bash[80295]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192 2024-04-24T22:50:16.142 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 podman[80309]: 2024-04-24 22:50:15.816255482 +0000 UTC m=+0.676445375 container remove 61cd8ff0148c1df6f0d217170faecd86b9ef3aa79e900db54515e7bb0e0a140c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, org.label-schema.license=GPLv2, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T22:50:16.143 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi192.service: Deactivated successfully. 2024-04-24T22:50:16.143 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 systemd[1]: Stopped Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:50:16.143 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:15 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi192.service: Consumed 6.814s CPU time. 2024-04-24T22:50:16.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 systemd[1]: Starting Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:50:16.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 podman[80443]: 2024-04-24 22:50:16.259779677 +0000 UTC m=+0.017953620 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:50:16.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 podman[80443]: 2024-04-24 22:50:16.388658513 +0000 UTC m=+0.146832471 container create 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 -- 172.21.15.63:0/3867503907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4141026a0 msgr2=0x7fc4140fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 --2- 172.21.15.63:0/3867503907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4141026a0 0x7fc4140fbd10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc404004940 tx=0x7fc404009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 -- 172.21.15.63:0/3867503907 shutdown_connections 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 --2- 172.21.15.63:0/3867503907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4141026a0 0x7fc4140fbd10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 --2- 172.21.15.63:0/3867503907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc414101d00 0x7fc4141020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 -- 172.21.15.63:0/3867503907 >> 172.21.15.63:0/3867503907 conn(0x7fc4140f7870 msgr2=0x7fc4140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:16.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 -- 172.21.15.63:0/3867503907 shutdown_connections 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.821+0000 7fc419041700 1 -- 172.21.15.63:0/3867503907 wait complete. 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 Processor -- start 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 -- start start 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414116790 con 0x7fc414101d00 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc419041700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41410fd20 con 0x7fc4141026a0 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36056/0 (socket says 172.21.15.63:36056) 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.822+0000 7fc412d9d700 1 -- 172.21.15.63:0/1014619155 learned_addr learned my addr 172.21.15.63:0/1014619155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc41259c700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc41259c700 1 -- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 msgr2=0x7fc4141160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T22:50:16.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc41259c700 1 -- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 msgr2=0x7fc4141160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc41259c700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc41259c700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc412d9d700 1 -- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 msgr2=0x7fc4141160b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc412d9d700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc412d9d700 1 -- 172.21.15.63:0/1014619155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc404005040 con 0x7fc414101d00 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc412d9d700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc40800b790 tx=0x7fc408009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40800f770 con 0x7fc414101d00 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc414110000 con 0x7fc414101d00 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc40800fda0 con 0x7fc414101d00 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40800d310 con 0x7fc414101d00 2024-04-24T22:50:16.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.823+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141105c0 con 0x7fc414101d00 2024-04-24T22:50:16.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.824+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140fd450 con 0x7fc414101d00 2024-04-24T22:50:16.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.825+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc408010420 con 0x7fc414101d00 2024-04-24T22:50:16.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.825+0000 7fc3fb7fe700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 0x7fc3fc070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:16.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.825+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc40808dc70 con 0x7fc414101d00 2024-04-24T22:50:16.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.825+0000 7fc41259c700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 0x7fc3fc070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:16.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.826+0000 7fc41259c700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 0x7fc3fc070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b3b0 tx=0x7fc404009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:16.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:16.831+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc408056fb0 con 0x7fc414101d00 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 podman[80443]: 2024-04-24 22:50:16.599693208 +0000 UTC m=+0.357867124 container init 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 podman[80443]: 2024-04-24 22:50:16.602738315 +0000 UTC m=+0.360912248 container start 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: pidfile_write: ignore empty --pid-file 2024-04-24T22:50:16.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: load: jerasure load: lrc 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: RocksDB version: 7.9.2 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Git sha 0 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: DB SUMMARY 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: DB Session ID: MRK9CAD53GQWHYZQSBIN 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: CURRENT file: CURRENT 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi192/store.db dir, Total Num: 1, files: 000027.sst 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi192/store.db: 000025.log size: 6088809 ; 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.error_if_exists: 0 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.create_if_missing: 0 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.paranoid_checks: 1 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T22:50:16.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.env: 0x559f4ff61c60 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.info_log: 0x559f50fcb960 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.statistics: (nil) 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.use_fsync: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_log_file_size: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_fallocate: 1 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T22:50:16.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.use_direct_reads: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.db_log_dir: 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.wal_dir: 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T22:50:16.931 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.write_buffer_manager: 0x559f50f9e1e0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.unordered_write: 0 2024-04-24T22:50:16.932 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T22:50:16.933 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T22:50:16.933 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T22:50:16.933 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T22:50:16.933 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.row_cache: None 2024-04-24T22:50:16.933 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.wal_filter: None 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.two_write_queues: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.wal_compression: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.atomic_flush: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.log_readahead_size: 0 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T22:50:16.935 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_background_jobs: 2 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_background_compactions: -1 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_subcompactions: 1 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T22:50:16.936 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_open_files: -1 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_background_flushes: -1 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Compression algorithms supported: 2024-04-24T22:50:16.937 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kZSTD supported: 0 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kXpressCompression supported: 0 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kZlibCompression supported: 1 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kSnappyCompression supported: 1 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kLZ4Compression supported: 1 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: kBZip2Compression supported: 0 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000009 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T22:50:16.938 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.merge_operator: 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_filter: None 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559f50fcba60) 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_type: 0 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_index_type: 0 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_shortening: 1 2024-04-24T22:50:16.939 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: checksum: 4 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: no_block_cache: 0 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache: 0x559f50fcf090 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_options: 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: capacity : 536870912 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_shard_bits : 4 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: strict_capacity_limit : 0 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_compressed: (nil) 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: persistent_cache: (nil) 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size: 4096 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size_deviation: 10 2024-04-24T22:50:16.940 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_restart_interval: 16 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_block_restart_interval: 1 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_block_size: 4096 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: partition_filters: 0 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: use_delta_encoding: 1 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: filter_policy: bloomfilter 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: whole_key_filtering: 1 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: verify_compression: 0 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: format_version: 5 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_index_compression: 1 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_align: 0 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_auto_readahead_size: 262144 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: prepopulate_block_cache: 0 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: initial_auto_readahead_size: 8192 2024-04-24T22:50:16.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression: NoCompression 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.num_levels: 7 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T22:50:16.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T22:50:16.943 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T22:50:16.944 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T22:50:16.945 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.table_properties_collectors: 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.inplace_update_support: 0 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T22:50:16.946 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.bloom_locality: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.max_successive_merges: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.ttl: 2592000 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enable_blob_files: false 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.min_blob_size: 0 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T22:50:16.947 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12220, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 994a5535-2f1d-4d51-8120-148e737b1d3b 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999016642463, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999016658120, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3711644, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12217, "largest_seqno": 14876, "table_properties": {"data_size": 3701003, "index_size": 6484, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3141, "raw_key_size": 27654, "raw_average_key_size": 22, "raw_value_size": 3676657, "raw_average_value_size": 2979, "num_data_blocks": 293, "num_entries": 1234, "num_filter_entries": 1234, "num_deletions": 258, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999016, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "994a5535-2f1d-4d51-8120-148e737b1d3b", "db_session_id": "MRK9CAD53GQWHYZQSBIN", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-24T22:50:16.948 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999016658228, "job": 1, "event": "recovery_finished"} 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi192/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559f510c0000 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: DB pointer 0x559f510aa000 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** DB Stats ** 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T22:50:16.949 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L0 1/0 3.54 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 501.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L6 1/0 6.88 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Sum 2/0 10.42 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 501.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 501.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 501.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-24T22:50:16.950 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative compaction: 0.00 GB write, 121.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval compaction: 0.00 GB write, 121.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T22:50:16.951 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache BinnedLRUCache@0x559f50fcf090#2 capacity: 512.00 MB usage: 9.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,3.28 KB,0.000625849%) IndexBlock(1,6.58 KB,0.00125468%) Misc(1,0.00 KB,0%) 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: starting mon.smithi192 rank 1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi192 fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???) e2 preinit fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).mds e14 new map 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).mds e14 print_map 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: e14 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:16.952 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: legacy client fscid: 1 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Filesystem 'cephfs' (1) 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: fs_name cephfs 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: epoch 14 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: tableserver 0 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: root 0 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_timeout 60 2024-04-24T22:50:16.953 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_autoclose 300 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_file_size 1099511627776 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_xattr_size 65536 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: required_client_features {} 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure 0 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure_osd_epoch 46 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:16.954 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_mds 1 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: in 0 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: up {0=24253} 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: failed 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: damaged 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: stopped 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_pools [3] 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_pool 2 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: inline_data enabled 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: balancer 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: bal_rank_mask -1 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: standby_count_wanted 1 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: qdb_cluster leader: 0 members: 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:16.955 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Standby daemons: 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 ceph-mon[80463]: mon.smithi192@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 bash[80443]: 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 2024-04-24T22:50:16.956 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:16 smithi192 systemd[1]: Started Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:50:17.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.059+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc414108ec0 con 0x7fc3fc06e270 2024-04-24T22:50:17.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.061+0000 7fc3fb7fe700 1 -- 172.21.15.63:0/1014619155 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc414108ec0 con 0x7fc3fc06e270 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 msgr2=0x7fc3fc070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 0x7fc3fc070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b3b0 tx=0x7fc404009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 msgr2=0x7fc414115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc40800b790 tx=0x7fc408009250 comp rx=0 tx=0).stop 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 shutdown_connections 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc3fc06e270 0x7fc3fc070720 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc414101d00 0x7fc414115b70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 --2- 172.21.15.63:0/1014619155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4141026a0 0x7fc4141160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 >> 172.21.15.63:0/1014619155 conn(0x7fc4140f7870 msgr2=0x7fc414066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 shutdown_connections 2024-04-24T22:50:17.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.063+0000 7fc419041700 1 -- 172.21.15.63:0/1014619155 wait complete. 2024-04-24T22:50:17.075 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- 172.21.15.63:0/1940517156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044102de0 msgr2=0x7f40441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 --2- 172.21.15.63:0/1940517156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044102de0 0x7f40441031b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4034004940 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- 172.21.15.63:0/1940517156 shutdown_connections 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 --2- 172.21.15.63:0/1940517156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044103780 0x7f40440fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 --2- 172.21.15.63:0/1940517156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044102de0 0x7f40441031b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- 172.21.15.63:0/1940517156 >> 172.21.15.63:0/1940517156 conn(0x7f40440f8990 msgr2=0x7f40440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- 172.21.15.63:0/1940517156 shutdown_connections 2024-04-24T22:50:17.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- 172.21.15.63:0/1940517156 wait complete. 2024-04-24T22:50:17.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 Processor -- start 2024-04-24T22:50:17.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.168+0000 7f404a45d700 1 -- start start 2024-04-24T22:50:17.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f404a45d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044102de0 0x7f4044113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f404a45d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f404a45d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044114500 con 0x7f4044103780 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f404a45d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404410db50 con 0x7f4044102de0 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36076/0 (socket says 172.21.15.63:36076) 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 -- 172.21.15.63:0/2483427391 learned_addr learned my addr 172.21.15.63:0/2483427391 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f4043fff700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044102de0 0x7f4044113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 -- 172.21.15.63:0/2483427391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044102de0 msgr2=0x7f4044113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044102de0 0x7f4044113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 -- 172.21.15.63:0/2483427391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034005040 con 0x7f4044103780 2024-04-24T22:50:17.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f40437fe700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4038002a50 tx=0x7f4038004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4038007cc0 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.169+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f403800f040 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.170+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404410de30 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.170+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403800da40 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.170+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404410e3f0 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.171+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40441000f0 con 0x7f4044103780 2024-04-24T22:50:17.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.171+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f403800dbc0 con 0x7f4044103780 2024-04-24T22:50:17.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.171+0000 7f4040ff9700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 0x7f402c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.171+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f403808db10 con 0x7f4044103780 2024-04-24T22:50:17.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.171+0000 7f4043fff700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 0x7f402c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.172+0000 7f4043fff700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 0x7f402c070720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4034004f90 tx=0x7f4034009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.176+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f403805a870 con 0x7f4044103780 2024-04-24T22:50:17.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.402+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4044113640 con 0x7f402c06e270 2024-04-24T22:50:17.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.403+0000 7f4040ff9700 1 -- 172.21.15.63:0/2483427391 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f4044113640 con 0x7f402c06e270 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 msgr2=0x7f402c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 0x7f402c070720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4034004f90 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 msgr2=0x7f4044113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4038002a50 tx=0x7f4038004600 comp rx=0 tx=0).stop 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 shutdown_connections 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4044102de0 0x7f4044113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f402c06e270 0x7f402c070720 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 --2- 172.21.15.63:0/2483427391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4044103780 0x7f4044113eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 >> 172.21.15.63:0/2483427391 conn(0x7f40440f8990 msgr2=0x7f4044107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 shutdown_connections 2024-04-24T22:50:17.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.405+0000 7f404a45d700 1 -- 172.21.15.63:0/2483427391 wait complete. 2024-04-24T22:50:17.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 -- 172.21.15.63:0/1820407869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fb5a0 msgr2=0x7f49f00fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 --2- 172.21.15.63:0/1820407869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fb5a0 0x7f49f00fb970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004940 tx=0x7f49e0009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 -- 172.21.15.63:0/1820407869 shutdown_connections 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 --2- 172.21.15.63:0/1820407869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fbf40 0x7f49f0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 --2- 172.21.15.63:0/1820407869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fb5a0 0x7f49f00fb970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.508+0000 7f49f50c7700 1 -- 172.21.15.63:0/1820407869 >> 172.21.15.63:0/1820407869 conn(0x7f49f0073350 msgr2=0x7f49f0073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 -- 172.21.15.63:0/1820407869 shutdown_connections 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 -- 172.21.15.63:0/1820407869 wait complete. 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 Processor -- start 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 -- start start 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 0x7f49f0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.509+0000 7f49f50c7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f0114510 con 0x7f49f00fbf40 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49f50c7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f010db30 con 0x7f49f00fb5a0 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36100/0 (socket says 172.21.15.63:36100) 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 -- 172.21.15.63:0/2395094696 learned_addr learned my addr 172.21.15.63:0/2395094696 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49eed9d700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 0x7f49f0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 -- 172.21.15.63:0/2395094696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 msgr2=0x7f49f0113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 0x7f49f0113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 -- 172.21.15.63:0/2395094696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0005040 con 0x7f49f00fbf40 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49eed9d700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 0x7f49f0113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49ee59c700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009b80 tx=0x7f49e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e4007cc0 con 0x7f49f00fbf40 2024-04-24T22:50:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f010de10 con 0x7f49f00fbf40 2024-04-24T22:50:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e400f040 con 0x7f49f00fbf40 2024-04-24T22:50:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e400dab0 con 0x7f49f00fbf40 2024-04-24T22:50:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.510+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f010e350 con 0x7f49f00fbf40 2024-04-24T22:50:17.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.511+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f00fe670 con 0x7f49f00fbf40 2024-04-24T22:50:17.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.512+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f49e400dd30 con 0x7f49f00fbf40 2024-04-24T22:50:17.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.512+0000 7f49d77fe700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 0x7f49d8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.512+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f49e408dba0 con 0x7f49f00fbf40 2024-04-24T22:50:17.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.512+0000 7f49eed9d700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 0x7f49d8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.513+0000 7f49eed9d700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 0x7f49d8070720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004560 tx=0x7f49e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.517+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f49e4056d50 con 0x7f49f00fbf40 2024-04-24T22:50:17.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.731+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f0109f40 con 0x7f49d806e270 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.743+0000 7f49d77fe700 1 -- 172.21.15.63:0/2395094696 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f49f0109f40 con 0x7f49d806e270 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 12s ago 13m 17.5M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (13m) 12s ago 13m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (12m) 14s ago 12m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (105s) 12s ago 13m 75.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:50:17.745 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (7m) 12s ago 7m 15.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (7m) 12s ago 7m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (7m) 14s ago 7m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (7m) 14s ago 7m 15.1M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (3m) 12s ago 14m 472M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (2m) 14s ago 12m 447M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (26s) 12s ago 14m 28.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 starting - - - 2048M 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (2m) 12s ago 13m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 14s ago 12m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:50:17.746 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (11m) 12s ago 11m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (11m) 12s ago 11m 216M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (10m) 12s ago 10m 181M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (10m) 14s ago 10m 235M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (9m) 14s ago 9m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (9m) 14s ago 9m 192M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:50:17.747 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 12s ago 12m 56.8M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 msgr2=0x7f49d8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 0x7f49d8070720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004560 tx=0x7f49e0009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 msgr2=0x7f49f0113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009b80 tx=0x7f49e4004600 comp rx=0 tx=0).stop 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 shutdown_connections 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49f00fb5a0 0x7f49f0113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f49d806e270 0x7f49d8070720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 --2- 172.21.15.63:0/2395094696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49f00fbf40 0x7f49f0113ec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 >> 172.21.15.63:0/2395094696 conn(0x7f49f0073350 msgr2=0x7f49f0100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 shutdown_connections 2024-04-24T22:50:17.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.745+0000 7f49f50c7700 1 -- 172.21.15.63:0/2395094696 wait complete. 2024-04-24T22:50:17.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 -- 172.21.15.63:0/1766886783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 msgr2=0x7f5a48102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 --2- 172.21.15.63:0/1766886783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a48102160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ce0 tx=0x7f5a38004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:17.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 -- 172.21.15.63:0/1766886783 shutdown_connections 2024-04-24T22:50:17.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 --2- 172.21.15.63:0/1766886783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a480fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 --2- 172.21.15.63:0/1766886783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a48102160 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 -- 172.21.15.63:0/1766886783 >> 172.21.15.63:0/1766886783 conn(0x7f5a48071d00 msgr2=0x7f5a48072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 -- 172.21.15.63:0/1766886783 shutdown_connections 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.848+0000 7f5a4f948700 1 -- 172.21.15.63:0/1766886783 wait complete. 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 Processor -- start 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 -- start start 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4810d510 con 0x7f5a48101d90 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4f948700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4810d680 con 0x7f5a480fbd40 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4cee3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4cee3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36110/0 (socket says 172.21.15.63:36110) 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4cee3700 1 -- 172.21.15.63:0/3091625087 learned_addr learned my addr 172.21.15.63:0/3091625087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:17.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4d6e4700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4d6e4700 1 -- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 msgr2=0x7f5a4810ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4d6e4700 1 -- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 msgr2=0x7f5a4810ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.849+0000 7f5a4d6e4700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4cee3700 1 -- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 msgr2=0x7f5a4810ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4d6e4700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4cee3700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4cee3700 1 -- 172.21.15.63:0/3091625087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009940 con 0x7f5a48101d90 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4cee3700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c002a50 tx=0x7f5a3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3c007cc0 con 0x7f5a48101d90 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3c00f040 con 0x7f5a48101d90 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4811a100 con 0x7f5a48101d90 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3c009640 con 0x7f5a48101d90 2024-04-24T22:50:17.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.850+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4811a4d0 con 0x7f5a48101d90 2024-04-24T22:50:17.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.851+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a480fd430 con 0x7f5a48101d90 2024-04-24T22:50:17.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.851+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5a3c0097a0 con 0x7f5a48101d90 2024-04-24T22:50:17.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.852+0000 7f5a45ffb700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 0x7f5a30070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:17.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.852+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5a3c08db00 con 0x7f5a48101d90 2024-04-24T22:50:17.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.852+0000 7f5a4d6e4700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 0x7f5a30070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:17.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.852+0000 7f5a4d6e4700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 0x7f5a30070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b740 tx=0x7f5a38004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:17.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:17.857+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5a3c05a7e0 con 0x7f5a48101d90 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.155+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a48108e20 con 0x7f5a48101d90 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.156+0000 7f5a45ffb700 1 -- 172.21.15.63:0/3091625087 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f5a3c05a370 con 0x7f5a48101d90 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:50:18.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 11, 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:50:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:50:18.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 msgr2=0x7f5a30070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 0x7f5a30070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b740 tx=0x7f5a38004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:18.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 msgr2=0x7f5a4810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c002a50 tx=0x7f5a3c0046b0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 shutdown_connections 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a480fbd40 0x7f5a4810ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f5a3006e260 0x7f5a30070710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 --2- 172.21.15.63:0/3091625087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a48101d90 0x7f5a4810cfd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 >> 172.21.15.63:0/3091625087 conn(0x7f5a48071d00 msgr2=0x7f5a480fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 shutdown_connections 2024-04-24T22:50:18.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.158+0000 7f5a4f948700 1 -- 172.21.15.63:0/3091625087 wait complete. 2024-04-24T22:50:18.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3349247677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 msgr2=0x7fc9440fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3349247677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9440fcdf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc934004940 tx=0x7fc934009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3349247677 shutdown_connections 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3349247677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9440fcdf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3349247677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc9441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3349247677 >> 172.21.15.63:0/3349247677 conn(0x7fc9440f8990 msgr2=0x7fc9440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3349247677 shutdown_connections 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.269+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3349247677 wait complete. 2024-04-24T22:50:18.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 Processor -- start 2024-04-24T22:50:18.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 -- start start 2024-04-24T22:50:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc944115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944116720 con 0x7fc944103780 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc94b8b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94410fd00 con 0x7fc944102de0 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc948e54700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc948e54700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36128/0 (socket says 172.21.15.63:36128) 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.270+0000 7fc948e54700 1 -- 172.21.15.63:0/3247160787 learned_addr learned my addr 172.21.15.63:0/3247160787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc949655700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc944115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc948e54700 1 -- 172.21.15.63:0/3247160787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 msgr2=0x7fc944115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc948e54700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc944115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc948e54700 1 -- 172.21.15.63:0/3247160787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc934005040 con 0x7fc944103780 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc949655700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc944115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc948e54700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc934009100 tx=0x7fc934009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93400e040 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc934013420 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94410ff80 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93401c630 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.271+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944110540 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.272+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440fe530 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.273+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc934013590 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.273+0000 7fc939ffb700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 0x7fc930070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.274+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc93408efa0 con 0x7fc944103780 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.274+0000 7fc949655700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 0x7fc930070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.274+0000 7fc949655700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 0x7fc930070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc940009fd0 tx=0x7fc94000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.278+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc93405bd00 con 0x7fc944103780 2024-04-24T22:50:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.527+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc944111210 con 0x7fc944103780 2024-04-24T22:50:18.530 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:18 smithi063 ceph-mon[111213]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T22:50:18.530 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:18 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:18.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.528+0000 7fc939ffb700 1 -- 172.21.15.63:0/3247160787 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fc93405b890 con 0x7fc944103780 2024-04-24T22:50:18.530 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:50:18.531 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:50:18.532 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:50:18.533 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:18.534 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 msgr2=0x7fc930070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 0x7fc930070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc940009fd0 tx=0x7fc94000b040 comp rx=0 tx=0).stop 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 msgr2=0x7fc9441160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc934009100 tx=0x7fc934009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 shutdown_connections 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc944102de0 0x7fc944115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc93006e270 0x7fc930070720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 --2- 172.21.15.63:0/3247160787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc944103780 0x7fc9441160d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 >> 172.21.15.63:0/3247160787 conn(0x7fc9440f8990 msgr2=0x7fc944071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 shutdown_connections 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.530+0000 7fc94b8b9700 1 -- 172.21.15.63:0/3247160787 wait complete. 2024-04-24T22:50:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.637+0000 7f0fa4772700 1 -- 172.21.15.63:0/832357487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c102de0 msgr2=0x7f0f9c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.637+0000 7f0fa4772700 1 --2- 172.21.15.63:0/832357487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c102de0 0x7f0f9c1031b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004940 tx=0x7f0f98009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- 172.21.15.63:0/832357487 shutdown_connections 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 --2- 172.21.15.63:0/832357487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c103780 0x7f0f9c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 --2- 172.21.15.63:0/832357487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c102de0 0x7f0f9c1031b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- 172.21.15.63:0/832357487 >> 172.21.15.63:0/832357487 conn(0x7f0f9c0f8990 msgr2=0x7f0f9c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- 172.21.15.63:0/832357487 shutdown_connections 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- 172.21.15.63:0/832357487 wait complete. 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 Processor -- start 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- start start 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c1166f0 con 0x7f0f9c103780 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.638+0000 7f0fa4772700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c10fd20 con 0x7f0f9c102de0 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35302/0 (socket says 172.21.15.63:35302) 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 -- 172.21.15.63:0/1267079690 learned_addr learned my addr 172.21.15.63:0/1267079690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa250e700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa250e700 1 -- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 msgr2=0x7f0f9c115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T22:50:18.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa250e700 1 -- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 msgr2=0x7f0f9c115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa250e700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa250e700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 -- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 msgr2=0x7f0f9c115b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 -- 172.21.15.63:0/1267079690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98005040 con 0x7f0f9c103780 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa1d0d700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c002a80 tx=0x7f0f8c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c007cc0 con 0x7f0f9c103780 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c110000 con 0x7f0f9c103780 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f8c00f040 con 0x7f0f9c103780 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c00daf0 con 0x7f0f9c103780 2024-04-24T22:50:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.639+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c1105c0 con 0x7f0f9c103780 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.640+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c106a80 con 0x7f0f9c103780 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.641+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0f8c00dc90 con 0x7f0f9c103780 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.641+0000 7f0f92ffd700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 0x7f0f88070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.642+0000 7f0fa250e700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 0x7f0f88070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.642+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0f8c08d410 con 0x7f0f9c103780 2024-04-24T22:50:18.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.642+0000 7f0fa250e700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 0x7f0f88070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009fd0 tx=0x7f0f98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.646+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f8c05a0f0 con 0x7f0f9c103780 2024-04-24T22:50:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:18 smithi192 ceph-mon[80463]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T22:50:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:18 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.874+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f9c1111b0 con 0x7f0f8806e270 2024-04-24T22:50:18.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.875+0000 7f0f92ffd700 1 -- 172.21.15.63:0/1267079690 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0f9c1111b0 con 0x7f0f8806e270 2024-04-24T22:50:18.876 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "3/21 daemons upgraded", 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:50:18.877 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 msgr2=0x7f0f88070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 0x7f0f88070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009fd0 tx=0x7f0f98009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 msgr2=0x7f0f9c1160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c002a80 tx=0x7f0f8c004630 comp rx=0 tx=0).stop 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 shutdown_connections 2024-04-24T22:50:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f9c102de0 0x7f0f9c115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0f8806e270 0x7f0f88070720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 --2- 172.21.15.63:0/1267079690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f9c103780 0x7f0f9c1160a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 >> 172.21.15.63:0/1267079690 conn(0x7f0f9c0f8990 msgr2=0x7f0f9c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 shutdown_connections 2024-04-24T22:50:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.877+0000 7f0fa4772700 1 -- 172.21.15.63:0/1267079690 wait complete. 2024-04-24T22:50:18.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- 172.21.15.63:0/3731698002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 msgr2=0x7fbee80fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 --2- 172.21.15.63:0/3731698002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee80fbd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbedc004940 tx=0x7fbedc009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- 172.21.15.63:0/3731698002 shutdown_connections 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 --2- 172.21.15.63:0/3731698002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee80fbd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 --2- 172.21.15.63:0/3731698002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- 172.21.15.63:0/3731698002 >> 172.21.15.63:0/3731698002 conn(0x7fbee80f7860 msgr2=0x7fbee80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- 172.21.15.63:0/3731698002 shutdown_connections 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- 172.21.15.63:0/3731698002 wait complete. 2024-04-24T22:50:18.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 Processor -- start 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.981+0000 7fbeee390700 1 -- start start 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbeee390700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbeee390700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbeee390700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8116780 con 0x7fbee8102690 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbeee390700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee810fd60 con 0x7fbee8101cf0 2024-04-24T22:50:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35318/0 (socket says 172.21.15.63:35318) 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 -- 172.21.15.63:0/3869103940 learned_addr learned my addr 172.21.15.63:0/3869103940 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee7fff700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee7fff700 1 -- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 msgr2=0x7fbee8115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee7fff700 1 -- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 msgr2=0x7fbee8115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee7fff700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee7fff700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 -- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 msgr2=0x7fbee8115bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 -- 172.21.15.63:0/3869103940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc005040 con 0x7fbee8102690 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.982+0000 7fbee77fe700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009100 tx=0x7fbedc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.983+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbedc00e040 con 0x7fbee8102690 2024-04-24T22:50:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.983+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbedc013420 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.983+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee810ffe0 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.983+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbedc01c610 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.983+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee81105a0 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.984+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee80fd440 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.984+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fbedc013590 con 0x7fbee8102690 2024-04-24T22:50:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.985+0000 7fbee4ff9700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 0x7fbed0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.985+0000 7fbee7fff700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 0x7fbed0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.985+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbedc08f000 con 0x7fbee8102690 2024-04-24T22:50:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.985+0000 7fbee7fff700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 0x7fbed0070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007fd0 tx=0x7fbed8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:18.989+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbedc05bd60 con 0x7fbee8102690 2024-04-24T22:50:19.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.284+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbee8108df0 con 0x7fbee8102690 2024-04-24T22:50:19.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.284+0000 7fbee4ff9700 1 -- 172.21.15.63:0/3869103940 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fbedc05b8f0 con 0x7fbee8102690 2024-04-24T22:50:19.286 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.286 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.286 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 msgr2=0x7fbed0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 0x7fbed0070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007fd0 tx=0x7fbed8009ee0 comp rx=0 tx=0).stop 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 msgr2=0x7fbee8116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009100 tx=0x7fbedc009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 shutdown_connections 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee8101cf0 0x7fbee8115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:19.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbed006e270 0x7fbed0070720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:19.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 --2- 172.21.15.63:0/3869103940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee8102690 0x7fbee8116130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:19.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 >> 172.21.15.63:0/3869103940 conn(0x7fbee80f7860 msgr2=0x7fbee80f8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:19.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 shutdown_connections 2024-04-24T22:50:19.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:19.286+0000 7fbeee390700 1 -- 172.21.15.63:0/3869103940 wait complete. 2024-04-24T22:50:19.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T22:50:19.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: mgrmap e40: smithi063.upcnvd(active, since 18s), standbys: smithi192.ctmwwc 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: mon.smithi063 calling monitor election 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: mon.smithi192 calling monitor election 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: monmap epoch 3 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:50:19.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: last_changed 2024-04-24T22:50:18.179667+0000 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: created 2024-04-24T22:35:18.856966+0000 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: min_mon_release 19 (squid) 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: election_strategy: 1 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.smithi192 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: mgrmap e40: smithi063.upcnvd(active, since 18s), standbys: smithi192.ctmwwc 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:19.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:19 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3247160787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:50:19.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: mgrmap e40: smithi063.upcnvd(active, since 18s), standbys: smithi192.ctmwwc 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: mon.smithi063 calling monitor election 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: mon.smithi192 calling monitor election 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: mon.smithi063 is new leader, mons smithi063,smithi192 in quorum (ranks 0,1) 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: monmap epoch 3 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: fsid c8a2676e-028a-11ef-bc93-c7b262605968 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: last_changed 2024-04-24T22:50:18.179667+0000 2024-04-24T22:50:19.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: created 2024-04-24T22:35:18.856966+0000 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: min_mon_release 19 (squid) 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: election_strategy: 1 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.smithi192 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: mgrmap e40: smithi063.upcnvd(active, since 18s), standbys: smithi192.ctmwwc 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:19.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:19 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3247160787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:50:20.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:20 smithi192 ceph-mon[80463]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:20.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:20 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3869103940' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:50:20.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:20 smithi063 ceph-mon[111213]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:20.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:20 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3869103940' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:50:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:21 smithi063 ceph-mon[111213]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:21.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:21 smithi192 ceph-mon[80463]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:21.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:21.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:23 smithi192 ceph-mon[80463]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:23 smithi063 ceph-mon[111213]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:25 smithi063 ceph-mon[111213]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:25.578 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:25 smithi192 ceph-mon[80463]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:26.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: Detected new or changed devices on smithi192 2024-04-24T22:50:26.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:26.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: Reconfiguring mon.smithi063 (monmap changed)... 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:26.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:26 smithi192 ceph-mon[80463]: Reconfiguring daemon mon.smithi063 on smithi063 2024-04-24T22:50:27.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: Detected new or changed devices on smithi192 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: Reconfiguring mon.smithi063 (monmap changed)... 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:27.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:26 smithi063 ceph-mon[111213]: Reconfiguring daemon mon.smithi063 on smithi063 2024-04-24T22:50:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:50:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:50:27.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:28.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:28.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.upcnvd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:50:28.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:50:28.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:28 smithi192 ceph-mon[80463]: Reconfiguring mgr.smithi063.upcnvd (monmap changed)... 2024-04-24T22:50:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:28 smithi192 ceph-mon[80463]: Reconfiguring daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:50:29.002 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:28 smithi063 ceph-mon[111213]: Reconfiguring mgr.smithi063.upcnvd (monmap changed)... 2024-04-24T22:50:29.002 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:28 smithi063 ceph-mon[111213]: Reconfiguring daemon mgr.smithi063.upcnvd on smithi063 2024-04-24T22:50:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:29 smithi192 ceph-mon[80463]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:29.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:29.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:29.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:29 smithi063 ceph-mon[111213]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:29.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:29.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:29.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:29.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:30 smithi192 ceph-mon[80463]: Reconfiguring crash.smithi063 (monmap changed)... 2024-04-24T22:50:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:30 smithi192 ceph-mon[80463]: Reconfiguring daemon crash.smithi063 on smithi063 2024-04-24T22:50:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:30 smithi063 ceph-mon[111213]: Reconfiguring crash.smithi063 (monmap changed)... 2024-04-24T22:50:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:30 smithi063 ceph-mon[111213]: Reconfiguring daemon crash.smithi063 on smithi063 2024-04-24T22:50:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:32.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:32.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:32.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T22:50:32.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:50:32.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:32.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:31 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.0 on smithi063 2024-04-24T22:50:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T22:50:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:50:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:31 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.0 on smithi063 2024-04-24T22:50:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T22:50:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:50:33.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:33.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:33 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.1 on smithi063 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:33.709 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:33 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.1 on smithi063 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:50:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:35.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:35 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.2 on smithi063 2024-04-24T22:50:35.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:35.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:35.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T22:50:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:50:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:35.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:35 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.2 on smithi063 2024-04-24T22:50:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:37.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: Reconfiguring mds.cephfs.smithi063.twfoxu (monmap changed)... 2024-04-24T22:50:37.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:37.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:37.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:36 smithi063 ceph-mon[111213]: Reconfiguring daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: Reconfiguring mds.cephfs.smithi063.twfoxu (monmap changed)... 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:37.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:36 smithi192 ceph-mon[80463]: Reconfiguring daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:50:38.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:37 smithi063 ceph-mon[111213]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:38.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:37 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:38.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:37 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:38.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:37 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:38.032 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:37 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:37 smithi192 ceph-mon[80463]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:37 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:37 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:37 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:37 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:38.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:38 smithi063 ceph-mon[111213]: Reconfiguring mds.cephfs.smithi063.dnliiv (monmap changed)... 2024-04-24T22:50:38.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:38 smithi063 ceph-mon[111213]: Reconfiguring daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:50:39.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:38 smithi192 ceph-mon[80463]: Reconfiguring mds.cephfs.smithi063.dnliiv (monmap changed)... 2024-04-24T22:50:39.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:38 smithi192 ceph-mon[80463]: Reconfiguring daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:50:39.873 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:39 smithi192 ceph-mon[80463]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:39.873 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:39 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:39.873 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:39 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:39.873 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:39 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:39.873 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:39 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:39 smithi063 ceph-mon[111213]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:39 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:39 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:39 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:40.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:39 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-24T22:50:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-24T22:50:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:40.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:50:40.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:50:40.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-24T22:50:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-24T22:50:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:41.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.ctmwwc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T22:50:41.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T22:50:41.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:41 smithi192 ceph-mon[80463]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:41 smithi192 ceph-mon[80463]: Reconfiguring mgr.smithi192.ctmwwc (monmap changed)... 2024-04-24T22:50:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:41 smithi192 ceph-mon[80463]: Reconfiguring daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:50:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:41 smithi063 ceph-mon[111213]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:41 smithi063 ceph-mon[111213]: Reconfiguring mgr.smithi192.ctmwwc (monmap changed)... 2024-04-24T22:50:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:41 smithi063 ceph-mon[111213]: Reconfiguring daemon mgr.smithi192.ctmwwc on smithi192 2024-04-24T22:50:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:43.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-24T22:50:43.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:43.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:43.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:43.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:43 smithi192 ceph-mon[80463]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T22:50:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:43 smithi063 ceph-mon[111213]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-24T22:50:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T22:50:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:50:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.3 on smithi192 2024-04-24T22:50:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:44 smithi192 ceph-mon[80463]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T22:50:45.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:50:45.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:45.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.3 on smithi192 2024-04-24T22:50:45.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:44 smithi063 ceph-mon[111213]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:45 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.4 on smithi192 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:46.336 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:45 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.4 on smithi192 2024-04-24T22:50:47.301 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:46 smithi192 ceph-mon[80463]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:47.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:46 smithi063 ceph-mon[111213]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:48.657 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:48.657 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:48.657 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T22:50:48.657 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:50:48.657 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:48.658 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:48 smithi192 ceph-mon[80463]: Reconfiguring daemon osd.5 on smithi192 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:48.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:48 smithi063 ceph-mon[111213]: Reconfiguring daemon osd.5 on smithi192 2024-04-24T22:50:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 -- 172.21.15.63:0/3226546316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 msgr2=0x7ff83c0fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 --2- 172.21.15.63:0/3226546316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c0fe230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff838004940 tx=0x7ff838009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 -- 172.21.15.63:0/3226546316 shutdown_connections 2024-04-24T22:50:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 --2- 172.21.15.63:0/3226546316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 --2- 172.21.15.63:0/3226546316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c0fe230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 -- 172.21.15.63:0/3226546316 >> 172.21.15.63:0/3226546316 conn(0x7ff83c0f7870 msgr2=0x7ff83c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 -- 172.21.15.63:0/3226546316 shutdown_connections 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.409+0000 7ff843eab700 1 -- 172.21.15.63:0/3226546316 wait complete. 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 Processor -- start 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 -- start start 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c114400 con 0x7ff83c0fde60 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff843eab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c10da00 con 0x7ff83c0fe800 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff841c47700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff841c47700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49200/0 (socket says 172.21.15.63:49200) 2024-04-24T22:50:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff841c47700 1 -- 172.21.15.63:0/739373605 learned_addr learned my addr 172.21.15.63:0/739373605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.410+0000 7ff841446700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff841c47700 1 -- 172.21.15.63:0/739373605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 msgr2=0x7ff83c113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff841c47700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c113db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff841c47700 1 -- 172.21.15.63:0/739373605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838005040 con 0x7ff83c0fde60 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff841446700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff841c47700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c00 tx=0x7ff838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff838021070 con 0x7ff83c0fde60 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff838013c00 con 0x7ff83c0fde60 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c10dc80 con 0x7ff83c0fde60 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff83800e070 con 0x7ff83c0fde60 2024-04-24T22:50:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.411+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c10e1c0 con 0x7ff83c0fde60 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.412+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c066ad0 con 0x7ff83c0fde60 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.413+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff83801c400 con 0x7ff83c0fde60 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.413+0000 7ff8327fc700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 0x7ff828070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.413+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff83808f110 con 0x7ff83c0fde60 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.413+0000 7ff841446700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 0x7ff828070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.414+0000 7ff841446700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 0x7ff828070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff83c10ebc0 tx=0x7ff82c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.419+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff83805bf10 con 0x7ff83c0fde60 2024-04-24T22:50:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.642+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff83c109e20 con 0x7ff82806e270 2024-04-24T22:50:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.643+0000 7ff8327fc700 1 -- 172.21.15.63:0/739373605 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff83c109e20 con 0x7ff82806e270 2024-04-24T22:50:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.645+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 msgr2=0x7ff828070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.645+0000 7ff843eab700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 0x7ff828070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff83c10ebc0 tx=0x7ff82c00b040 comp rx=0 tx=0).stop 2024-04-24T22:50:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.645+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 msgr2=0x7ff83c113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.645+0000 7ff843eab700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c00 tx=0x7ff838009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 shutdown_connections 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff82806e270 0x7ff828070720 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff83c0fde60 0x7ff83c113870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 --2- 172.21.15.63:0/739373605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff83c0fe800 0x7ff83c113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 >> 172.21.15.63:0/739373605 conn(0x7ff83c0f7870 msgr2=0x7ff83c0f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 shutdown_connections 2024-04-24T22:50:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.646+0000 7ff843eab700 1 -- 172.21.15.63:0/739373605 wait complete. 2024-04-24T22:50:49.657 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:50:49.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:49 smithi063 ceph-mon[111213]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:49.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:49.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:49.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:49.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:49.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:49 smithi192 ceph-mon[80463]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:49.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:49.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:49.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:49.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:49.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- 172.21.15.63:0/665500441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9600101d00 msgr2=0x7f96001020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 --2- 172.21.15.63:0/665500441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9600101d00 0x7f96001020d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004940 tx=0x7f95f4009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- 172.21.15.63:0/665500441 shutdown_connections 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 --2- 172.21.15.63:0/665500441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96001026a0 0x7f96000fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 --2- 172.21.15.63:0/665500441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9600101d00 0x7f96001020d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- 172.21.15.63:0/665500441 >> 172.21.15.63:0/665500441 conn(0x7f96000f7870 msgr2=0x7f96000f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- 172.21.15.63:0/665500441 shutdown_connections 2024-04-24T22:50:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- 172.21.15.63:0/665500441 wait complete. 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 Processor -- start 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.751+0000 7f9606bca700 1 -- start start 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f9606bca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9600101d00 0x7f9600115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f9606bca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f9606bca700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600116780 con 0x7f96001026a0 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f9606bca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960010fda0 con 0x7f9600101d00 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49214/0 (socket says 172.21.15.63:49214) 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 -- 172.21.15.63:0/2834404644 learned_addr learned my addr 172.21.15.63:0/2834404644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f9604966700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9600101d00 0x7f9600115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 -- 172.21.15.63:0/2834404644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9600101d00 msgr2=0x7f9600115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9600101d00 0x7f9600115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 -- 172.21.15.63:0/2834404644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4005040 con 0x7f96001026a0 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.752+0000 7f95fffff700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c890 tx=0x7f95ec0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.753+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95ec00a700 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.753+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95ec00ad30 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.753+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9600110080 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.753+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95ec0113a0 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.753+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96001105c0 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.754+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96000fd450 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.754+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f95ec012400 con 0x7f96001026a0 2024-04-24T22:50:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.755+0000 7f95fd7fa700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 0x7f95f0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.755+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f95ec08dce0 con 0x7f96001026a0 2024-04-24T22:50:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.755+0000 7f9604966700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 0x7f95f0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.755+0000 7f9604966700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 0x7f95f0070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95f400b3b0 tx=0x7f95f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.760+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f95ec057020 con 0x7f96001026a0 2024-04-24T22:50:49.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.991+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9600108e00 con 0x7f95f006e270 2024-04-24T22:50:49.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.992+0000 7f95fd7fa700 1 -- 172.21.15.63:0/2834404644 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9600108e00 con 0x7f95f006e270 2024-04-24T22:50:49.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 msgr2=0x7f95f0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 0x7f95f0070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95f400b3b0 tx=0x7f95f4009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:49.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 msgr2=0x7f9600116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:49.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c890 tx=0x7f95ec0044c0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 shutdown_connections 2024-04-24T22:50:49.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9600101d00 0x7f9600115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f95f006e270 0x7f95f0070720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 --2- 172.21.15.63:0/2834404644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96001026a0 0x7f9600116130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:49.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 >> 172.21.15.63:0/2834404644 conn(0x7f96000f7870 msgr2=0x7f9600066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:49.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 shutdown_connections 2024-04-24T22:50:49.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:49.995+0000 7f9606bca700 1 -- 172.21.15.63:0/2834404644 wait complete. 2024-04-24T22:50:50.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 -- 172.21.15.63:0/3192000002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 msgr2=0x7f1a00070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 --2- 172.21.15.63:0/3192000002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00070670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009d10 tx=0x7f19f0004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 -- 172.21.15.63:0/3192000002 shutdown_connections 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 --2- 172.21.15.63:0/3192000002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 0x7f1a00109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 --2- 172.21.15.63:0/3192000002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00070670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 -- 172.21.15.63:0/3192000002 >> 172.21.15.63:0/3192000002 conn(0x7f1a000f8980 msgr2=0x7f1a000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 -- 172.21.15.63:0/3192000002 shutdown_connections 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 -- 172.21.15.63:0/3192000002 wait complete. 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.100+0000 7f1a06251700 1 Processor -- start 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f1a06251700 1 -- start start 2024-04-24T22:50:50.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f1a06251700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f1a06251700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 0x7f1a00116510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f1a06251700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00116b60 con 0x7f1a000702a0 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f1a06251700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0010ff30 con 0x7f1a00070bb0 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49244/0 (socket says 172.21.15.63:49244) 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 -- 172.21.15.63:0/3106434023 learned_addr learned my addr 172.21.15.63:0/3106434023 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19feffd700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 0x7f1a00116510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 -- 172.21.15.63:0/3106434023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 msgr2=0x7f1a00116510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 0x7f1a00116510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 -- 172.21.15.63:0/3106434023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0009940 con 0x7f1a000702a0 2024-04-24T22:50:50.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.101+0000 7f19ff7fe700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19f0008df0 tx=0x7f19f0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.102+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f000e040 con 0x7f1a000702a0 2024-04-24T22:50:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.102+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a001101b0 con 0x7f1a000702a0 2024-04-24T22:50:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.102+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19f0004610 con 0x7f1a000702a0 2024-04-24T22:50:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.102+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f001c8a0 con 0x7f1a000702a0 2024-04-24T22:50:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.102+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a001106f0 con 0x7f1a000702a0 2024-04-24T22:50:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.103+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a000719e0 con 0x7f1a000702a0 2024-04-24T22:50:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.103+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f19f0012410 con 0x7f1a000702a0 2024-04-24T22:50:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.104+0000 7f19f7fff700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 0x7f19ec070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.104+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f19f008fa40 con 0x7f1a000702a0 2024-04-24T22:50:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.104+0000 7f19feffd700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 0x7f19ec070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.104+0000 7f19feffd700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 0x7f19ec070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a00111870 tx=0x7f19e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.108+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f19f005c7a0 con 0x7f1a000702a0 2024-04-24T22:50:50.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.325+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a0004bf40 con 0x7f19ec06e260 2024-04-24T22:50:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.335+0000 7f19f7fff700 1 -- 172.21.15.63:0/3106434023 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1a0004bf40 con 0x7f19ec06e260 2024-04-24T22:50:50.337 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:50:50.337 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 44s ago 13m 17.5M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:50:50.337 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (14m) 44s ago 14m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af eb0f15f2ffe0 2024-04-24T22:50:50.337 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (12m) 29s ago 12m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 5bda9dcaa247 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (2m) 44s ago 13m 75.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (8m) 44s ago 8m 15.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (8m) 44s ago 8m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (8m) 29s ago 8m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (8m) 29s ago 8m 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (4m) 44s ago 15m 472M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (3m) 29s ago 12m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (59s) 44s ago 15m 28.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (34s) 29s ago 12m 23.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (3m) 44s ago 14m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (3m) 29s ago 12m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (12m) 44s ago 12m 214M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (11m) 44s ago 11m 216M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (11m) 44s ago 11m 181M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:50:50.338 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (10m) 29s ago 10m 235M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:50:50.339 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (10m) 29s ago 10m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:50:50.339 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (9m) 29s ago 9m 193M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:50:50.339 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 44s ago 13m 56.8M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:50:50.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.337+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 msgr2=0x7f19ec070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.337+0000 7f1a06251700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 0x7f19ec070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a00111870 tx=0x7f19e800b040 comp rx=0 tx=0).stop 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.337+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 msgr2=0x7f1a00115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.337+0000 7f1a06251700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19f0008df0 tx=0x7f19f0004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 shutdown_connections 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f19ec06e260 0x7f19ec070710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a000702a0 0x7f1a00115fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 --2- 172.21.15.63:0/3106434023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00070bb0 0x7f1a00116510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 >> 172.21.15.63:0/3106434023 conn(0x7f1a000f8980 msgr2=0x7f1a000f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 shutdown_connections 2024-04-24T22:50:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.338+0000 7f1a06251700 1 -- 172.21.15.63:0/3106434023 wait complete. 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.441+0000 7f33eb1dc700 1 -- 172.21.15.63:0/3484062200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e41026b0 msgr2=0x7f33e40fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.441+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/3484062200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e41026b0 0x7f33e40fbd20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33d0004940 tx=0x7f33d0009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 -- 172.21.15.63:0/3484062200 shutdown_connections 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/3484062200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e41026b0 0x7f33e40fbd20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/3484062200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e4101d10 0x7f33e41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 -- 172.21.15.63:0/3484062200 >> 172.21.15.63:0/3484062200 conn(0x7f33e40f7860 msgr2=0x7f33e40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 -- 172.21.15.63:0/3484062200 shutdown_connections 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 -- 172.21.15.63:0/3484062200 wait complete. 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 Processor -- start 2024-04-24T22:50:50.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.442+0000 7f33eb1dc700 1 -- start start 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33eb1dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33eb1dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 0x7f33e41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33eb1dc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41035f0 con 0x7f33e4101d10 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33eb1dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4103760 con 0x7f33e41026b0 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49254/0 (socket says 172.21.15.63:49254) 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 -- 172.21.15.63:0/1861605800 learned_addr learned my addr 172.21.15.63:0/1861605800 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e3fff700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 0x7f33e41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 -- 172.21.15.63:0/1861605800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 msgr2=0x7f33e41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 0x7f33e41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 -- 172.21.15.63:0/1861605800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d0005040 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e3fff700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 0x7f33e41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e8f78700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33d800b790 tx=0x7f33d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.443+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33d800f770 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.444+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4119ef0 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.444+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33d800fda0 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.444+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33d800d310 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.444+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e411a240 con 0x7f33e4101d10 2024-04-24T22:50:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.445+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e40fd460 con 0x7f33e4101d10 2024-04-24T22:50:50.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.445+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f33d8010420 con 0x7f33e4101d10 2024-04-24T22:50:50.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.446+0000 7f33e17fa700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 0x7f33d4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.446+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f33d808dc70 con 0x7f33e4101d10 2024-04-24T22:50:50.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.446+0000 7f33e3fff700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 0x7f33d4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.446+0000 7f33e3fff700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 0x7f33d4070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009fd0 tx=0x7f33d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.450+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f33d8056fb0 con 0x7f33e4101d10 2024-04-24T22:50:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:50 smithi192 ceph-mon[80463]: Reconfiguring mds.cephfs.smithi192.mnfuxk (monmap changed)... 2024-04-24T22:50:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:50 smithi192 ceph-mon[80463]: Reconfiguring daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:50:50.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:50 smithi192 ceph-mon[80463]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.748+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f33e4104050 con 0x7f33e4101d10 2024-04-24T22:50:50.750 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:50 smithi063 ceph-mon[111213]: Reconfiguring mds.cephfs.smithi192.mnfuxk (monmap changed)... 2024-04-24T22:50:50.750 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:50 smithi063 ceph-mon[111213]: Reconfiguring daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:50:50.750 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:50 smithi063 ceph-mon[111213]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.748+0000 7f33e17fa700 1 -- 172.21.15.63:0/1861605800 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f33d805a670 con 0x7f33e4101d10 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:50:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 msgr2=0x7f33d4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 0x7f33d4070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009fd0 tx=0x7f33d0009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 msgr2=0x7f33e4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33d800b790 tx=0x7f33d8009250 comp rx=0 tx=0).stop 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 shutdown_connections 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33d406e270 0x7f33d4070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4101d10 0x7f33e4102b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 --2- 172.21.15.63:0/1861605800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e41026b0 0x7f33e41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 >> 172.21.15.63:0/1861605800 conn(0x7f33e40f7860 msgr2=0x7f33e4066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 shutdown_connections 2024-04-24T22:50:50.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.751+0000 7f33eb1dc700 1 -- 172.21.15.63:0/1861605800 wait complete. 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- 172.21.15.63:0/537094101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 msgr2=0x7ff2cc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/537094101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc070300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004970 tx=0x7ff2c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- 172.21.15.63:0/537094101 shutdown_connections 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/537094101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 0x7ff2cc070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/537094101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc070300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- 172.21.15.63:0/537094101 >> 172.21.15.63:0/537094101 conn(0x7ff2cc0f8760 msgr2=0x7ff2cc0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- 172.21.15.63:0/537094101 shutdown_connections 2024-04-24T22:50:50.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- 172.21.15.63:0/537094101 wait complete. 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 Processor -- start 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.856+0000 7ff2d25a0700 1 -- start start 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d25a0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 0x7ff2cc11a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d25a0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d25a0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc11aba0 con 0x7ff2cc071f30 2024-04-24T22:50:50.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d25a0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc113f70 con 0x7ff2cc070840 2024-04-24T22:50:50.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49274/0 (socket says 172.21.15.63:49274) 2024-04-24T22:50:50.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 -- 172.21.15.63:0/3557697951 learned_addr learned my addr 172.21.15.63:0/3557697951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:50.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d159e700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 0x7ff2cc11a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 -- 172.21.15.63:0/3557697951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 msgr2=0x7ff2cc11a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 0x7ff2cc11a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 -- 172.21.15.63:0/3557697951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8005040 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d0d9d700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc002a50 tx=0x7ff2bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc007cc0 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2bc00f040 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc114250 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.857+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc00da70 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.858+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc114790 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.859+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc071a40 con 0x7ff2cc071f30 2024-04-24T22:50:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.859+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff2bc00dc10 con 0x7ff2cc071f30 2024-04-24T22:50:50.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.859+0000 7ff2c1ffb700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 0x7ff2b8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:50.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.859+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff2bc08d3c0 con 0x7ff2cc071f30 2024-04-24T22:50:50.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.859+0000 7ff2d159e700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 0x7ff2b8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:50.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.860+0000 7ff2d159e700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 0x7ff2b8070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004f50 tx=0x7ff2c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:50.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:50.864+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff2bc05a0a0 con 0x7ff2cc071f30 2024-04-24T22:50:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.119+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2cc0fc3a0 con 0x7ff2cc071f30 2024-04-24T22:50:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.119+0000 7ff2c1ffb700 1 -- 172.21.15.63:0/3557697951 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff2bc059c30 con 0x7ff2cc071f30 2024-04-24T22:50:51.121 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:50:51.122 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:50:51.123 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:50:51.126 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:51.127 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 msgr2=0x7ff2b8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 0x7ff2b8070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004f50 tx=0x7ff2c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 msgr2=0x7ff2cc11a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc002a50 tx=0x7ff2bc0046b0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 shutdown_connections 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.121+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc070840 0x7ff2cc11a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.122+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff2b806e270 0x7ff2b8070720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.122+0000 7ff2d25a0700 1 --2- 172.21.15.63:0/3557697951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2cc071f30 0x7ff2cc11a550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.122+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 >> 172.21.15.63:0/3557697951 conn(0x7ff2cc0f8760 msgr2=0x7ff2cc0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.122+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 shutdown_connections 2024-04-24T22:50:51.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.122+0000 7ff2d25a0700 1 -- 172.21.15.63:0/3557697951 wait complete. 2024-04-24T22:50:51.129 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- 172.21.15.63:0/1088377717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 msgr2=0x7f90780fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 --2- 172.21.15.63:0/1088377717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f90780fc1b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f906c004940 tx=0x7f906c009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- 172.21.15.63:0/1088377717 shutdown_connections 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 --2- 172.21.15.63:0/1088377717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f90780fc1b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 --2- 172.21.15.63:0/1088377717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 0x7f9078102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- 172.21.15.63:0/1088377717 >> 172.21.15.63:0/1088377717 conn(0x7f9078071d00 msgr2=0x7f9078072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- 172.21.15.63:0/1088377717 shutdown_connections 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- 172.21.15.63:0/1088377717 wait complete. 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 Processor -- start 2024-04-24T22:50:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.227+0000 7f907d5e6700 1 -- start start 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f907d5e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f907d5e6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 0x7f9078103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f907d5e6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781035f0 con 0x7f90780fbd40 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f907d5e6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078103730 con 0x7f9078101d90 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49300/0 (socket says 172.21.15.63:49300) 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 -- 172.21.15.63:0/2618601520 learned_addr learned my addr 172.21.15.63:0/2618601520 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f90767fc700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 0x7f9078103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 -- 172.21.15.63:0/2618601520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 msgr2=0x7f9078103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 0x7f9078103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 -- 172.21.15.63:0/2618601520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c005040 con 0x7f90780fbd40 2024-04-24T22:50:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f9076ffd700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb40 tx=0x7f906800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.228+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9068007cc0 con 0x7f90780fbd40 2024-04-24T22:50:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.229+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f906800f040 con 0x7f90780fbd40 2024-04-24T22:50:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.229+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9078103980 con 0x7f90780fbd40 2024-04-24T22:50:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.229+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906800ca10 con 0x7f90780fbd40 2024-04-24T22:50:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.229+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907811a1a0 con 0x7f90780fbd40 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.229+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90781063c0 con 0x7f90780fbd40 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.230+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f906800cc80 con 0x7f90780fbd40 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.231+0000 7f905f7fe700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 0x7f9060070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.231+0000 7f90767fc700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 0x7f9060070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.231+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f906808d930 con 0x7f90780fbd40 2024-04-24T22:50:51.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.231+0000 7f90767fc700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 0x7f9060070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f906c009b30 tx=0x7f906c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.235+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9068056b90 con 0x7f90780fbd40 2024-04-24T22:50:51.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.460+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f907804bed0 con 0x7f906006e260 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.462+0000 7f905f7fe700 1 -- 172.21.15.63:0/2618601520 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f907804bed0 con 0x7f906006e260 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "4/21 daemons upgraded", 2024-04-24T22:50:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T22:50:51.465 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:50:51.465 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 msgr2=0x7f9060070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 0x7f9060070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f906c009b30 tx=0x7f906c009e30 comp rx=0 tx=0).stop 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 msgr2=0x7f9078102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb40 tx=0x7f906800b040 comp rx=0 tx=0).stop 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 shutdown_connections 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f906006e260 0x7f9060070710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90780fbd40 0x7f9078102b20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 --2- 172.21.15.63:0/2618601520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9078101d90 0x7f9078103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 >> 172.21.15.63:0/2618601520 conn(0x7f9078071d00 msgr2=0x7f9078065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 shutdown_connections 2024-04-24T22:50:51.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.464+0000 7f907d5e6700 1 -- 172.21.15.63:0/2618601520 wait complete. 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- 172.21.15.63:0/1771555807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0102de0 msgr2=0x7f95a01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/1771555807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0102de0 0x7f95a01031b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9598009ce0 tx=0x7f9598004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- 172.21.15.63:0/1771555807 shutdown_connections 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/1771555807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0103780 0x7f95a00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/1771555807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0102de0 0x7f95a01031b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- 172.21.15.63:0/1771555807 >> 172.21.15.63:0/1771555807 conn(0x7f95a00f8990 msgr2=0x7f95a00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- 172.21.15.63:0/1771555807 shutdown_connections 2024-04-24T22:50:51.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- 172.21.15.63:0/1771555807 wait complete. 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 Processor -- start 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.568+0000 7f95a4ec6700 1 -- start start 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0102de0 0x7f95a0113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a01146b0 con 0x7f95a0103780 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a010dc90 con 0x7f95a0102de0 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49306/0 (socket says 172.21.15.63:49306) 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 -- 172.21.15.63:0/3915950304 learned_addr learned my addr 172.21.15.63:0/3915950304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959e59c700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0102de0 0x7f95a0113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 -- 172.21.15.63:0/3915950304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0102de0 msgr2=0x7f95a0113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0102de0 0x7f95a0113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 -- 172.21.15.63:0/3915950304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9598009940 con 0x7f95a0103780 2024-04-24T22:50:51.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f959dd9b700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9588002a50 tx=0x7f9588004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9588007cc0 con 0x7f95a0103780 2024-04-24T22:50:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f958800f040 con 0x7f95a0103780 2024-04-24T22:50:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958800d9c0 con 0x7f95a0103780 2024-04-24T22:50:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a010df70 con 0x7f95a0103780 2024-04-24T22:50:51.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.569+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a010e530 con 0x7f95a0103780 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.570+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a01000f0 con 0x7f95a0103780 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.571+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9588009730 con 0x7f95a0103780 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.571+0000 7f9596ffd700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 0x7f958c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.571+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f958808d9e0 con 0x7f95a0103780 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.571+0000 7f959e59c700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 0x7f958c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:50:51.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.572+0000 7f959e59c700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 0x7f958c070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f959800b740 tx=0x7f9598004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:50:51.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.576+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f958805a740 con 0x7f95a0103780 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: Reconfiguring mds.cephfs.smithi192.ozemqi (monmap changed)... 2024-04-24T22:50:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: Reconfiguring daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:50:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1861605800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:51.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:51 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3557697951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:50:51.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.869+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95a00fe530 con 0x7f95a0103780 2024-04-24T22:50:51.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.870+0000 7f9596ffd700 1 -- 172.21.15.63:0/3915950304 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f958805a2d0 con 0x7f95a0103780 2024-04-24T22:50:51.872 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:51.872 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:50:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:50:51.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 msgr2=0x7f958c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 0x7f958c070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f959800b740 tx=0x7f9598004d10 comp rx=0 tx=0).stop 2024-04-24T22:50:51.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 msgr2=0x7f95a0114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:50:51.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9588002a50 tx=0x7f9588004600 comp rx=0 tx=0).stop 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 shutdown_connections 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a0102de0 0x7f95a0113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f958c06e270 0x7f958c070720 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 --2- 172.21.15.63:0/3915950304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95a0103780 0x7f95a0114060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 >> 172.21.15.63:0/3915950304 conn(0x7f95a00f8990 msgr2=0x7f95a0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 shutdown_connections 2024-04-24T22:50:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:50:51.872+0000 7f95a4ec6700 1 -- 172.21.15.63:0/3915950304 wait complete. 2024-04-24T22:50:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: Reconfiguring mds.cephfs.smithi192.ozemqi (monmap changed)... 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: Reconfiguring daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1861605800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:51.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:51 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3557697951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3915950304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-04-24T22:50:53.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-04-24T22:50:53.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-04-24T22:50:53.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-04-24T22:50:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3915950304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-04-24T22:50:53.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-04-24T22:50:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:53 smithi063 ceph-mon[111213]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:53 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all mon 2024-04-24T22:50:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:53 smithi192 ceph-mon[80463]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:54.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:53 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all mon 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: Upgrade: Updating crash.smithi063 (1/2) 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:55.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:55 smithi063 ceph-mon[111213]: Deploying daemon crash.smithi063 on smithi063 2024-04-24T22:50:55.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:55.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: Upgrade: Updating crash.smithi063 (1/2) 2024-04-24T22:50:55.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:55.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:50:55.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:50:55.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:55 smithi192 ceph-mon[80463]: Deploying daemon crash.smithi063 on smithi063 2024-04-24T22:50:57.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:57 smithi063 ceph-mon[111213]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:57 smithi192 ceph-mon[80463]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:59.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:59 smithi192 ceph-mon[80463]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:59.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:59 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:59.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:50:59 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:59.491 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:59 smithi063 ceph-mon[111213]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:50:59.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:59 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:50:59.492 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:50:59 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: Upgrade: Updating crash.smithi192 (2/2) 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: Deploying daemon crash.smithi192 on smithi192 2024-04-24T22:51:00.904 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: Upgrade: Updating crash.smithi192 (2/2) 2024-04-24T22:51:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T22:51:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: Deploying daemon crash.smithi192 on smithi192 2024-04-24T22:51:01.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:02.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:01 smithi192 ceph-mon[80463]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:02.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:01 smithi063 ceph-mon[111213]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:03 smithi063 ceph-mon[111213]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:03 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:03 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:04.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:03 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:04.144 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:03 smithi192 ceph-mon[80463]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:04.184 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:03 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:04.185 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:03 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:04.185 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:03 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:05 smithi192 ceph-mon[80463]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:06.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:05 smithi063 ceph-mon[111213]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:07.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:07 smithi063 ceph-mon[111213]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:07.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:07.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:07.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:07 smithi192 ceph-mon[80463]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:07.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:07.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:08 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:08 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:08 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:08.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:08 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:09.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:09 smithi192 ceph-mon[80463]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:09.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:09 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:09.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:09 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:09 smithi063 ceph-mon[111213]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:09 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:09 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:10 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:10 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:10.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:10 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:10.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:10 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:11 smithi063 ceph-mon[111213]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:11.829 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:11 smithi192 ceph-mon[80463]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:11.829 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:11.829 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:13.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.917 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-04-24T22:51:13.918 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-04-24T22:51:13.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-04-24T22:51:13.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-04-24T22:51:13.919 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-04-24T22:51:13.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T22:51:14.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:14 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all crash 2024-04-24T22:51:14.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:14 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T22:51:14.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:14 smithi063 ceph-mon[111213]: Upgrade: osd.0 is safe to restart 2024-04-24T22:51:14.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:14 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all crash 2024-04-24T22:51:14.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:14 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T22:51:14.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:14 smithi192 ceph-mon[80463]: Upgrade: osd.0 is safe to restart 2024-04-24T22:51:15.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:15 smithi063 ceph-mon[111213]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:15.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:15.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:51:15.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:15.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:15 smithi192 ceph-mon[80463]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T22:51:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:16.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:16 smithi063 ceph-mon[111213]: Upgrade: Updating osd.0 2024-04-24T22:51:16.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:16 smithi063 ceph-mon[111213]: Deploying daemon osd.0 on smithi063 2024-04-24T22:51:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:16 smithi192 ceph-mon[80463]: Upgrade: Updating osd.0 2024-04-24T22:51:16.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:16 smithi192 ceph-mon[80463]: Deploying daemon osd.0 on smithi063 2024-04-24T22:51:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:17 smithi192 ceph-mon[80463]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:18.009 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:17 smithi063 ceph-mon[111213]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:18.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:18 smithi063 systemd[1]: Stopping Ceph osd.0 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:51:19.303 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:18 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[45208]: 2024-04-24T22:51:18.829+0000 7fd69b334700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:51:19.304 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:18 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[45208]: 2024-04-24T22:51:18.829+0000 7fd69b334700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-24T22:51:19.304 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:18 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[45208]: 2024-04-24T22:51:18.829+0000 7fd69b334700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:51:19.882 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:19 smithi063 ceph-mon[111213]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:19.882 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:19 smithi063 ceph-mon[111213]: osd.0 marked itself down and dead 2024-04-24T22:51:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:19 smithi192 ceph-mon[80463]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:19 smithi192 ceph-mon[80463]: osd.0 marked itself down and dead 2024-04-24T22:51:20.290 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:20 smithi063 podman[122123]: 2024-04-24 22:51:20.037630532 +0000 UTC m=+1.317110604 container died 1d0a412fab32340099800767ad20a60e104a524a05296111ab30a6a2e708490b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T22:51:20.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:20 smithi063 podman[122123]: 2024-04-24 22:51:20.471444043 +0000 UTC m=+1.750924111 container cleanup 1d0a412fab32340099800767ad20a60e104a524a05296111ab30a6a2e708490b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T22:51:20.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:20 smithi063 bash[122123]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0 2024-04-24T22:51:20.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:20 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:51:20.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:20 smithi192 ceph-mon[80463]: osdmap e50: 6 total, 5 up, 6 in 2024-04-24T22:51:21.053 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:20 smithi063 podman[122142]: 2024-04-24 22:51:20.664718361 +0000 UTC m=+0.624854509 container remove 1d0a412fab32340099800767ad20a60e104a524a05296111ab30a6a2e708490b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T22:51:21.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:20 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:51:21.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:20 smithi063 ceph-mon[111213]: osdmap e50: 6 total, 5 up, 6 in 2024-04-24T22:51:21.337 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:20.98029963 +0000 UTC m=+0.018655189 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:51:21.337 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:21.12444994 +0000 UTC m=+0.162805472 container create 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:51:21.337 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:21.335634241 +0000 UTC m=+0.373989777 container init 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:51:21.602 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:21.339660394 +0000 UTC m=+0.378015925 container start 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T22:51:21.602 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:21.418699663 +0000 UTC m=+0.457055203 container attach 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-24T22:51:21.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:21 smithi192 ceph-mon[80463]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:21.881 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:21 smithi192 ceph-mon[80463]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T22:51:21.939 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:21 smithi063 ceph-mon[111213]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:21.940 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:21 smithi063 ceph-mon[111213]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T22:51:21.940 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:21 smithi063 podman[122239]: 2024-04-24 22:51:21.645467882 +0000 UTC m=+0.683823421 container died 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:51:21.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- 172.21.15.63:0/2206135818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 msgr2=0x7f83941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:21.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 --2- 172.21.15.63:0/2206135818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f83941031b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8390004940 tx=0x7f8390009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- 172.21.15.63:0/2206135818 shutdown_connections 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 --2- 172.21.15.63:0/2206135818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 0x7f83940fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 --2- 172.21.15.63:0/2206135818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f83941031b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- 172.21.15.63:0/2206135818 >> 172.21.15.63:0/2206135818 conn(0x7f83940f8950 msgr2=0x7f83940fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- 172.21.15.63:0/2206135818 shutdown_connections 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- 172.21.15.63:0/2206135818 wait complete. 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 Processor -- start 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.981+0000 7f839b718700 1 -- start start 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f839b718700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f839b718700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 0x7f83941140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f839b718700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8394114720 con 0x7f8394102de0 2024-04-24T22:51:21.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f839b718700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839410db30 con 0x7f8394103780 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38190/0 (socket says 172.21.15.63:38190) 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 -- 172.21.15.63:0/2510778126 learned_addr learned my addr 172.21.15.63:0/2510778126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f8398cb3700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 0x7f83941140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 -- 172.21.15.63:0/2510778126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 msgr2=0x7f83941140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 0x7f83941140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 -- 172.21.15.63:0/2510778126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390005040 con 0x7f8394102de0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f83994b4700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8390009150 tx=0x7f8390009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f839000e040 con 0x7f8394102de0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839410ddb0 con 0x7f8394102de0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8390013420 con 0x7f8394102de0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.982+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f839001c630 con 0x7f8394102de0 2024-04-24T22:51:21.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.983+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839410e370 con 0x7f8394102de0 2024-04-24T22:51:21.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.984+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83941000f0 con 0x7f8394102de0 2024-04-24T22:51:21.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.984+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8390013590 con 0x7f8394102de0 2024-04-24T22:51:21.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.985+0000 7f8389ffb700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 0x7f8380070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:21.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.985+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f839008efa0 con 0x7f8394102de0 2024-04-24T22:51:21.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.985+0000 7f8398cb3700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 0x7f8380070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:21.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.985+0000 7f8398cb3700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 0x7f8380070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fd0 tx=0x7f8384009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:21.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:21.990+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8390058280 con 0x7f8394102de0 2024-04-24T22:51:22.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.220+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8394109f00 con 0x7f838006e260 2024-04-24T22:51:22.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.221+0000 7f8389ffb700 1 -- 172.21.15.63:0/2510778126 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8394109f00 con 0x7f838006e260 2024-04-24T22:51:22.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 msgr2=0x7f8380070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 0x7f8380070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fd0 tx=0x7f8384009f20 comp rx=0 tx=0).stop 2024-04-24T22:51:22.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 msgr2=0x7f8394113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8390009150 tx=0x7f8390009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 shutdown_connections 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f838006e260 0x7f8380070710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8394102de0 0x7f8394113b90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 --2- 172.21.15.63:0/2510778126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8394103780 0x7f83941140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.223+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 >> 172.21.15.63:0/2510778126 conn(0x7f83940f8950 msgr2=0x7f83941073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.224+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 shutdown_connections 2024-04-24T22:51:22.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.224+0000 7f839b718700 1 -- 172.21.15.63:0/2510778126 wait complete. 2024-04-24T22:51:22.238 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 -- 172.21.15.63:0/3803302058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 msgr2=0x7ff384065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 --2- 172.21.15.63:0/3803302058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff384065ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff378004940 tx=0x7ff378009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 -- 172.21.15.63:0/3803302058 shutdown_connections 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 --2- 172.21.15.63:0/3803302058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff384065ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 --2- 172.21.15.63:0/3803302058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 0x7ff384103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 -- 172.21.15.63:0/3803302058 >> 172.21.15.63:0/3803302058 conn(0x7ff384071d00 msgr2=0x7ff384072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 -- 172.21.15.63:0/3803302058 shutdown_connections 2024-04-24T22:51:22.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.344+0000 7ff389131700 1 -- 172.21.15.63:0/3803302058 wait complete. 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 Processor -- start 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 -- start start 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 0x7ff3840fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840fc830 con 0x7ff384065640 2024-04-24T22:51:22.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.345+0000 7ff389131700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840fc970 con 0x7ff384102e50 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38208/0 (socket says 172.21.15.63:38208) 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 -- 172.21.15.63:0/1970749313 learned_addr learned my addr 172.21.15.63:0/1970749313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff38259c700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 0x7ff3840fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 -- 172.21.15.63:0/1970749313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 msgr2=0x7ff3840fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 0x7ff3840fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 -- 172.21.15.63:0/1970749313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378005040 con 0x7ff384065640 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff382d9d700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff37400eb40 tx=0x7ff37400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff374007cc0 con 0x7ff384065640 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3840fcbc0 con 0x7ff384065640 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff37400f040 con 0x7ff384065640 2024-04-24T22:51:22.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.346+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37400ca10 con 0x7ff384065640 2024-04-24T22:51:22.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.347+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38411a150 con 0x7ff384065640 2024-04-24T22:51:22.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.348+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384100100 con 0x7ff384065640 2024-04-24T22:51:22.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.348+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff37400cb70 con 0x7ff384065640 2024-04-24T22:51:22.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.349+0000 7ff36b7fe700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 0x7ff36c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.349+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff37408d7e0 con 0x7ff384065640 2024-04-24T22:51:22.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.349+0000 7ff38259c700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 0x7ff36c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.350+0000 7ff38259c700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 0x7ff36c070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff378009b30 tx=0x7ff378009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:22.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.354+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff374056ac0 con 0x7ff384065640 2024-04-24T22:51:22.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 podman[122265]: 2024-04-24 22:51:22.214104113 +0000 UTC m=+0.563298807 container remove 36c6c77bc918c5894f1c63bcf9b1c873b7c970ba7e1d57b93f9b0aa2bbf8ce5e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T22:51:22.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-24T22:51:22.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 systemd[1]: Stopped Ceph osd.0 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:51:22.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.0.service: Consumed 53.928s CPU time. 2024-04-24T22:51:22.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 systemd[1]: Starting Ceph osd.0 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:51:22.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.614+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38404bed0 con 0x7ff36c06e260 2024-04-24T22:51:22.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.615+0000 7ff36b7fe700 1 -- 172.21.15.63:0/1970749313 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff38404bed0 con 0x7ff36c06e260 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 msgr2=0x7ff36c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 0x7ff36c070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff378009b30 tx=0x7ff378009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 msgr2=0x7ff3840fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff37400eb40 tx=0x7ff37400b040 comp rx=0 tx=0).stop 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 shutdown_connections 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff36c06e260 0x7ff36c070710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff384065640 0x7ff3840fbdb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 --2- 172.21.15.63:0/1970749313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff384102e50 0x7ff3840fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 >> 172.21.15.63:0/1970749313 conn(0x7ff384071d00 msgr2=0x7ff3840faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 shutdown_connections 2024-04-24T22:51:22.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.617+0000 7ff389131700 1 -- 172.21.15.63:0/1970749313 wait complete. 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 -- 172.21.15.63:0/1981080839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64102de0 msgr2=0x7fdb641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 --2- 172.21.15.63:0/1981080839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64102de0 0x7fdb641031b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb54004940 tx=0x7fdb54009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 -- 172.21.15.63:0/1981080839 shutdown_connections 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 --2- 172.21.15.63:0/1981080839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64103780 0x7fdb640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 --2- 172.21.15.63:0/1981080839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64102de0 0x7fdb641031b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 -- 172.21.15.63:0/1981080839 >> 172.21.15.63:0/1981080839 conn(0x7fdb640f8990 msgr2=0x7fdb640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 -- 172.21.15.63:0/1981080839 shutdown_connections 2024-04-24T22:51:22.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.723+0000 7fdb69c10700 1 -- 172.21.15.63:0/1981080839 wait complete. 2024-04-24T22:51:22.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 Processor -- start 2024-04-24T22:51:22.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 -- start start 2024-04-24T22:51:22.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 0x7fdb64113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb64114500 con 0x7fdb64103780 2024-04-24T22:51:22.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb69c10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410db50 con 0x7fdb64102de0 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38234/0 (socket says 172.21.15.63:38234) 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb62ffd700 1 -- 172.21.15.63:0/2963926404 learned_addr learned my addr 172.21.15.63:0/2963926404 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb637fe700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 0x7fdb64113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb62ffd700 1 -- 172.21.15.63:0/2963926404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 msgr2=0x7fdb64113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.724+0000 7fdb62ffd700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 0x7fdb64113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb62ffd700 1 -- 172.21.15.63:0/2963926404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb54005040 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb637fe700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 0x7fdb64113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb62ffd700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb58002a50 tx=0x7fdb58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb58007cc0 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb5800f040 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6410de30 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5800da40 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.725+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6410e3f0 con 0x7fdb64103780 2024-04-24T22:51:22.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.726+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb641000f0 con 0x7fdb64103780 2024-04-24T22:51:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.726+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fdb580097c0 con 0x7fdb64103780 2024-04-24T22:51:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.727+0000 7fdb4bfff700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 0x7fdb4c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.727+0000 7fdb637fe700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 0x7fdb4c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.727+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb5808db10 con 0x7fdb64103780 2024-04-24T22:51:22.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.728+0000 7fdb637fe700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 0x7fdb4c070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb54004560 tx=0x7fdb54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:22.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.732+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb5805a870 con 0x7fdb64103780 2024-04-24T22:51:22.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:22 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-24T22:51:22.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 podman[122414]: 2024-04-24 22:51:22.597011799 +0000 UTC m=+0.022844285 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:51:22.874 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 podman[122414]: 2024-04-24 22:51:22.741013041 +0000 UTC m=+0.166845500 container create 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T22:51:22.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:22 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-24T22:51:22.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.958+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb6410efc0 con 0x7fdb4c06e270 2024-04-24T22:51:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.968+0000 7fdb4bfff700 1 -- 172.21.15.63:0/2963926404 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdb6410efc0 con 0x7fdb4c06e270 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 14s ago 14m 28.1M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (25s) 14s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (20s) 16s ago 13m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (2m) 14s ago 14m 75.9M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (8m) 14s ago 8m 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:51:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (8m) 14s ago 8m 13.8M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (8m) 16s ago 8m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (8m) 16s ago 8m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (4m) 14s ago 15m 495M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (4m) 16s ago 13m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (91s) 14s ago 16m 42.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (66s) 16s ago 13m 34.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (3m) 14s ago 14m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (3m) 16s ago 13m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (12m) 14s ago 12m 215M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1d0a412fab32 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (12m) 14s ago 12m 217M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (11m) 14s ago 11m 182M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (11m) 16s ago 11m 236M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (10m) 16s ago 10m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (10m) 16s ago 10m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:51:22.971 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 14s ago 13m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:51:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 msgr2=0x7fdb4c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 0x7fdb4c070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb54004560 tx=0x7fdb54009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 msgr2=0x7fdb64113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb58002a50 tx=0x7fdb58004600 comp rx=0 tx=0).stop 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 shutdown_connections 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb64102de0 0x7fdb64113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb4c06e270 0x7fdb4c070720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 --2- 172.21.15.63:0/2963926404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb64103780 0x7fdb64113eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 >> 172.21.15.63:0/2963926404 conn(0x7fdb640f8990 msgr2=0x7fdb64107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 shutdown_connections 2024-04-24T22:51:22.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:22.970+0000 7fdb69c10700 1 -- 172.21.15.63:0/2963926404 wait complete. 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 -- 172.21.15.63:0/3348441938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 msgr2=0x7f4c1c0ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 --2- 172.21.15.63:0/3348441938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c0ff010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c04004940 tx=0x7f4c04009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 -- 172.21.15.63:0/3348441938 shutdown_connections 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 --2- 172.21.15.63:0/3348441938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c0ff010 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 --2- 172.21.15.63:0/3348441938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 0x7f4c1c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 -- 172.21.15.63:0/3348441938 >> 172.21.15.63:0/3348441938 conn(0x7f4c1c0fabb0 msgr2=0x7f4c1c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 -- 172.21.15.63:0/3348441938 shutdown_connections 2024-04-24T22:51:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 -- 172.21.15.63:0/3348441938 wait complete. 2024-04-24T22:51:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.076+0000 7f4c2238d700 1 Processor -- start 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c2238d700 1 -- start start 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c2238d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 0x7f4c1c115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c2238d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c2238d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c116950 con 0x7f4c1c1059a0 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c2238d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c10fd20 con 0x7f4c1c105000 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38252/0 (socket says 172.21.15.63:38252) 2024-04-24T22:51:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 -- 172.21.15.63:0/1938406421 learned_addr learned my addr 172.21.15.63:0/1938406421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1bfff700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 0x7f4c1c115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 -- 172.21.15.63:0/1938406421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 msgr2=0x7f4c1c115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 0x7f4c1c115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.077+0000 7f4c1b7fe700 1 -- 172.21.15.63:0/1938406421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04005040 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c1b7fe700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009100 tx=0x7f4c04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0400e040 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c10ffa0 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c04004290 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0401c3f0 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.078+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c110560 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.079+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c100750 con 0x7f4c1c1059a0 2024-04-24T22:51:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.080+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4c04013690 con 0x7f4c1c1059a0 2024-04-24T22:51:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.080+0000 7f4c18ff9700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 0x7f4c08070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.080+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4c0408eda0 con 0x7f4c1c1059a0 2024-04-24T22:51:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.080+0000 7f4c1bfff700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 0x7f4c08070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.081+0000 7f4c1bfff700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 0x7f4c08070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c004f40 tx=0x7f4c0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.085+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4c0405bba0 con 0x7f4c1c1059a0 2024-04-24T22:51:23.303 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 podman[122414]: 2024-04-24 22:51:22.986617615 +0000 UTC m=+0.412450076 container init 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T22:51:23.303 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:22 smithi063 podman[122414]: 2024-04-24 22:51:22.990289601 +0000 UTC m=+0.416122061 container start 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T22:51:23.303 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 podman[122414]: 2024-04-24 22:51:23.060740986 +0000 UTC m=+0.486573468 container attach 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T22:51:23.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.389+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c1c04bed0 con 0x7f4c1c1059a0 2024-04-24T22:51:23.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.390+0000 7f4c18ff9700 1 -- 172.21.15.63:0/1938406421 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f4c0405b730 con 0x7f4c1c1059a0 2024-04-24T22:51:23.392 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:51:23.392 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:51:23.392 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:51:23.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:23.393 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:51:23.393 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:51:23.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:23.393 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:51:23.394 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 9, 2024-04-24T22:51:23.395 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:51:23.395 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:51:23.395 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:51:23.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 msgr2=0x7f4c08070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 0x7f4c08070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c004f40 tx=0x7f4c0c00b040 comp rx=0 tx=0).stop 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 msgr2=0x7f4c1c116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009100 tx=0x7f4c04009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 shutdown_connections 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c1c105000 0x7f4c1c115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4c0806e260 0x7f4c08070710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 --2- 172.21.15.63:0/1938406421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c1c1059a0 0x7f4c1c116300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.392+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 >> 172.21.15.63:0/1938406421 conn(0x7f4c1c0fabb0 msgr2=0x7f4c1c0fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.393+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 shutdown_connections 2024-04-24T22:51:23.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.393+0000 7f4c2238d700 1 -- 172.21.15.63:0/1938406421 wait complete. 2024-04-24T22:51:23.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 -- 172.21.15.63:0/3269800446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fd1b0 msgr2=0x7fe5a40fd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 --2- 172.21.15.63:0/3269800446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a40fd580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004940 tx=0x7fe5a0009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:23.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 -- 172.21.15.63:0/3269800446 shutdown_connections 2024-04-24T22:51:23.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 --2- 172.21.15.63:0/3269800446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a40fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 --2- 172.21.15.63:0/3269800446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a40fd580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 -- 172.21.15.63:0/3269800446 >> 172.21.15.63:0/3269800446 conn(0x7fe5a40f89b0 msgr2=0x7fe5a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 -- 172.21.15.63:0/3269800446 shutdown_connections 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.501+0000 7fe5aba17700 1 -- 172.21.15.63:0/3269800446 wait complete. 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 Processor -- start 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 -- start start 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a4113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a41144c0 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5aba17700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a410dac0 con 0x7fe5a40fd1b0 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5a8fb2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5a8fb2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38276/0 (socket says 172.21.15.63:38276) 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5a8fb2700 1 -- 172.21.15.63:0/957841026 learned_addr learned my addr 172.21.15.63:0/957841026 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:23.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.502+0000 7fe5a97b3700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a4113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe5a8fb2700 1 -- 172.21.15.63:0/957841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fd1b0 msgr2=0x7fe5a4113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe5a8fb2700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a4113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe5a8fb2700 1 -- 172.21.15.63:0/957841026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0005040 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe5a8fb2700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe594002a50 tx=0x7fe5940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe594007cc0 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a410dda0 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.503+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe59400f040 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.504+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe59400da00 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.504+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a410e2a0 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.505+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a40fe550 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.505+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fe59400db60 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.505+0000 7fe599ffb700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 0x7fe5900707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.506+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe59408daa0 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.506+0000 7fe5a97b3700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 0x7fe5900707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.506+0000 7fe5a97b3700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 0x7fe5900707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004f90 tx=0x7fe5a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.514+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe59405b860 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.655 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:51:23.655 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 bash[122414]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate[122452]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T22:51:23.656 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:23 smithi063 podman[122414]: 2024-04-24 22:51:23.488935722 +0000 UTC m=+0.914768186 container died 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T22:51:23.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.771+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe5a410ef80 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.771+0000 7fe599ffb700 1 -- 172.21.15.63:0/957841026 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fe594064590 con 0x7fe5a40fe9b0 2024-04-24T22:51:23.773 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:51:23.773 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:51:23.773 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:51:23.774 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:51:23.775 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:23.776 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.773+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 msgr2=0x7fe5900707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.773+0000 7fe5aba17700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 0x7fe5900707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004f90 tx=0x7fe5a0009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.773+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 msgr2=0x7fe5a4113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe594002a50 tx=0x7fe5940046b0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 shutdown_connections 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40fd1b0 0x7fe5a4113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe59006e340 0x7fe5900707f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 --2- 172.21.15.63:0/957841026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5a40fe9b0 0x7fe5a4113e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 >> 172.21.15.63:0/957841026 conn(0x7fe5a40f89b0 msgr2=0x7fe5a40fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:23.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 shutdown_connections 2024-04-24T22:51:23.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.774+0000 7fe5aba17700 1 -- 172.21.15.63:0/957841026 wait complete. 2024-04-24T22:51:23.778 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/3092799127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14103780 msgr2=0x7f0e140fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/3092799127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14103780 0x7f0e140fcdf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e10004940 tx=0x7f0e10009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/3092799127 shutdown_connections 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/3092799127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14103780 0x7f0e140fcdf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/3092799127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14102de0 0x7f0e141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/3092799127 >> 172.21.15.63:0/3092799127 conn(0x7f0e140f8990 msgr2=0x7f0e140fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/3092799127 shutdown_connections 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.880+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/3092799127 wait complete. 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 Processor -- start 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 -- start start 2024-04-24T22:51:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14103780 0x7f0e14113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141144e0 con 0x7f0e14102de0 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1c4e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1410dae0 con 0x7f0e14103780 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1a285700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1a285700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38286/0 (socket says 172.21.15.63:38286) 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e1a285700 1 -- 172.21.15.63:0/1137509098 learned_addr learned my addr 172.21.15.63:0/1137509098 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.881+0000 7f0e19a84700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14103780 0x7f0e14113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1a285700 1 -- 172.21.15.63:0/1137509098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14103780 msgr2=0x7f0e14113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1a285700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14103780 0x7f0e14113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1a285700 1 -- 172.21.15.63:0/1137509098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10005040 con 0x7f0e14102de0 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1a285700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c8c0 tx=0x7f0e040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0400a760 con 0x7f0e14102de0 2024-04-24T22:51:23.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0400ad90 con 0x7f0e14102de0 2024-04-24T22:51:23.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1410ddc0 con 0x7f0e14102de0 2024-04-24T22:51:23.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e040113a0 con 0x7f0e14102de0 2024-04-24T22:51:23.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.882+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1410e350 con 0x7f0e14102de0 2024-04-24T22:51:23.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.883+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140fe530 con 0x7f0e14102de0 2024-04-24T22:51:23.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.883+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0e04012400 con 0x7f0e14102de0 2024-04-24T22:51:23.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.884+0000 7f0e0affd700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 0x7f0e00070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:23.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.884+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0e0408dc90 con 0x7f0e14102de0 2024-04-24T22:51:23.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.884+0000 7f0e19a84700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 0x7f0e00070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:23.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.884+0000 7f0e19a84700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 0x7f0e00070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0e1410edc0 tx=0x7f0e10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:23.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:23.889+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0e0405ab00 con 0x7f0e14102de0 2024-04-24T22:51:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:23 smithi192 ceph-mon[80463]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:23 smithi192 ceph-mon[80463]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:23 smithi192 ceph-mon[80463]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:23 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1938406421' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-mon[111213]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-mon[111213]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-mon[111213]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:23 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1938406421' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:24.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.118+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e14109ec0 con 0x7f0e0006e270 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.120+0000 7f0e0affd700 1 -- 172.21.15.63:0/1137509098 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0e14109ec0 con 0x7f0e0006e270 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:51:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "6/21 daemons upgraded", 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:51:24.122 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:51:24.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 msgr2=0x7f0e00070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 0x7f0e00070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0e1410edc0 tx=0x7f0e10009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 msgr2=0x7f0e14113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c8c0 tx=0x7f0e040044c0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 shutdown_connections 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0e0006e270 0x7f0e00070720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e14102de0 0x7f0e14113950 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 --2- 172.21.15.63:0/1137509098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e14103780 0x7f0e14113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 >> 172.21.15.63:0/1137509098 conn(0x7f0e140f8990 msgr2=0x7f0e140f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 shutdown_connections 2024-04-24T22:51:24.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.122+0000 7f0e1c4e9700 1 -- 172.21.15.63:0/1137509098 wait complete. 2024-04-24T22:51:24.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 -- 172.21.15.63:0/1655984208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 msgr2=0x7f95600fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 --2- 172.21.15.63:0/1655984208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95600fce00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9550004940 tx=0x7f9550009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 -- 172.21.15.63:0/1655984208 shutdown_connections 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 --2- 172.21.15.63:0/1655984208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95600fce00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 --2- 172.21.15.63:0/1655984208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f95601031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 -- 172.21.15.63:0/1655984208 >> 172.21.15.63:0/1655984208 conn(0x7f95600f8980 msgr2=0x7f95600fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 -- 172.21.15.63:0/1655984208 shutdown_connections 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.243+0000 7f95648bd700 1 -- 172.21.15.63:0/1655984208 wait complete. 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 Processor -- start 2024-04-24T22:51:24.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 -- start start 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f9560113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560114720 con 0x7f9560103790 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f95648bd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956010db10 con 0x7f9560102df0 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f955dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f955dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38308/0 (socket says 172.21.15.63:38308) 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.244+0000 7f955dd9b700 1 -- 172.21.15.63:0/3273873784 learned_addr learned my addr 172.21.15.63:0/3273873784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955e59c700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f9560113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955dd9b700 1 -- 172.21.15.63:0/3273873784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 msgr2=0x7f9560113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955dd9b700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f9560113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955dd9b700 1 -- 172.21.15.63:0/3273873784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550005040 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955e59c700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f9560113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f955dd9b700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9550009100 tx=0x7f9550009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955000e040 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956010dd90 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956010e350 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9550013420 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.245+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9550025640 con 0x7f9560103790 2024-04-24T22:51:24.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.246+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400052b0 con 0x7f9560103790 2024-04-24T22:51:24.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.247+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f955000baf0 con 0x7f9560103790 2024-04-24T22:51:24.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.247+0000 7f954effd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 0x7f9548070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:24.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.247+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f955008ee20 con 0x7f9560103790 2024-04-24T22:51:24.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.247+0000 7f955e59c700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 0x7f9548070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:24.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.248+0000 7f955e59c700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 0x7f9548070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95540041d0 tx=0x7f9554009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:24.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.252+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f955005bc20 con 0x7f9560103790 2024-04-24T22:51:24.358 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 podman[122617]: 2024-04-24 22:51:24.097988653 +0000 UTC m=+0.599900337 container remove 48f91f35537cdd96e2f8c96a7467b225f175993b9cf147fb2d4e529cae37cc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-activate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:51:24.359 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 podman[122727]: 2024-04-24 22:51:24.227615398 +0000 UTC m=+0.019472695 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:51:24.359 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 podman[122727]: 2024-04-24 22:51:24.35768741 +0000 UTC m=+0.149544708 container create 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T22:51:24.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.546+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9540005830 con 0x7f9560103790 2024-04-24T22:51:24.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.547+0000 7f954effd700 1 -- 172.21.15.63:0/3273873784 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1517 (secure 0 0 0) 0x7f955005b7b0 con 0x7f9560103790 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: osd.0 (root=default,host=smithi063) is down 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-24T22:51:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-24T22:51:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-24T22:51:24.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 msgr2=0x7f9548070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 0x7f9548070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95540041d0 tx=0x7f9554009040 comp rx=0 tx=0).stop 2024-04-24T22:51:24.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 msgr2=0x7f95601140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:24.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9550009100 tx=0x7f9550009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 shutdown_connections 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9560102df0 0x7f9560113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f954806e260 0x7f9548070710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 --2- 172.21.15.63:0/3273873784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9560103790 0x7f95601140d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 >> 172.21.15.63:0/3273873784 conn(0x7f95600f8980 msgr2=0x7f95600f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 shutdown_connections 2024-04-24T22:51:24.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:24.549+0000 7f95648bd700 1 -- 172.21.15.63:0/3273873784 wait complete. 2024-04-24T22:51:24.660 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 podman[122727]: 2024-04-24 22:51:24.578329377 +0000 UTC m=+0.370186679 container init 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-24T22:51:24.660 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 podman[122727]: 2024-04-24 22:51:24.581490422 +0000 UTC m=+0.373347727 container start 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T22:51:24.660 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 ceph-osd[122768]: -- 172.21.15.63:0/3148963672 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d764bf5680 con 0x55d764bbf000 2024-04-24T22:51:24.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:24 smithi192 ceph-mon[80463]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:24.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:24 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/957841026' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:51:24.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:24 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3273873784' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:51:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:24 smithi063 ceph-mon[111213]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:24 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/957841026' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:51:25.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:24 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3273873784' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:51:25.053 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 bash[122727]: 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc 2024-04-24T22:51:25.053 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:24 smithi063 systemd[1]: Started Ceph osd.0 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:51:25.553 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T22:51:25.118+0000 7f8e9272e7c0 -1 Falling back to public interface 2024-04-24T22:51:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-mon[111213]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-mon[111213]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:25 smithi192 ceph-mon[80463]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:25 smithi192 ceph-mon[80463]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:27 smithi063 ceph-mon[111213]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:28.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:27 smithi192 ceph-mon[80463]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:30.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:29 smithi063 ceph-mon[111213]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:30.053 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:29 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T22:51:29.973+0000 7f8e9272e7c0 -1 osd.0 49 log_to_monitors true 2024-04-24T22:51:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:29 smithi192 ceph-mon[80463]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:30 smithi192 ceph-mon[80463]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:51:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:30 smithi063 ceph-mon[111213]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T22:51:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:32.090 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T22:51:31.811+0000 7f8e7fb7e700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:51:32.090 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:32.090 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:51:32.091 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T22:51:32.091 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:51:32.091 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:32.091 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:32.091 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:31 smithi063 ceph-mon[111213]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:32.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:31 smithi192 ceph-mon[80463]: from='osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146]' entity='osd.0' 2024-04-24T22:51:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:32 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:51:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:32 smithi192 ceph-mon[80463]: osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146] boot 2024-04-24T22:51:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:32 smithi192 ceph-mon[80463]: osdmap e53: 6 total, 6 up, 6 in 2024-04-24T22:51:33.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:32 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:51:33.227 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:32 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:51:33.227 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:32 smithi063 ceph-mon[111213]: osd.0 [v2:172.21.15.63:6802/3067429146,v1:172.21.15.63:6803/3067429146] boot 2024-04-24T22:51:33.227 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:32 smithi063 ceph-mon[111213]: osdmap e53: 6 total, 6 up, 6 in 2024-04-24T22:51:33.227 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:32 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T22:51:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:33 smithi192 ceph-mon[80463]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-24T22:51:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:33 smithi192 ceph-mon[80463]: mgrmap e41: smithi063.upcnvd(active, since 92s), standbys: smithi192.ctmwwc 2024-04-24T22:51:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:33 smithi192 ceph-mon[80463]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T22:51:34.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:33 smithi063 ceph-mon[111213]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-24T22:51:34.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:33 smithi063 ceph-mon[111213]: mgrmap e41: smithi063.upcnvd(active, since 92s), standbys: smithi192.ctmwwc 2024-04-24T22:51:34.263 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:33 smithi063 ceph-mon[111213]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T22:51:35.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:34 smithi063 ceph-mon[111213]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-24T22:51:35.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:34 smithi192 ceph-mon[80463]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-24T22:51:36.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:36.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:35 smithi063 ceph-mon[111213]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T22:51:36.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:36.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:36.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:36.429 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:35 smithi192 ceph-mon[80463]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T22:51:37.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:36 smithi063 ceph-mon[111213]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T22:51:37.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:36 smithi192 ceph-mon[80463]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T22:51:38.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:37 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-04-24T22:51:38.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:37 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-04-24T22:51:39.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:38 smithi063 ceph-mon[111213]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 427 B/s wr, 0 op/s 2024-04-24T22:51:39.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:38 smithi192 ceph-mon[80463]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 427 B/s wr, 0 op/s 2024-04-24T22:51:41.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:41 smithi192 ceph-mon[80463]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:41.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:41 smithi063 ceph-mon[111213]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:43.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:43 smithi192 ceph-mon[80463]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:43.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:43 smithi063 ceph-mon[111213]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:45 smithi063 ceph-mon[111213]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:45.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:45.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:45 smithi192 ceph-mon[80463]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:45.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:45.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:51:45.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:47.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:47 smithi192 ceph-mon[80463]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:47 smithi063 ceph-mon[111213]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:49.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:49 smithi192 ceph-mon[80463]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:49 smithi063 ceph-mon[111213]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:51.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:51 smithi063 ceph-mon[111213]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:51.414 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:51.415 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:51 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:51.415 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:51 smithi063 ceph-mon[111213]: Upgrade: osd.1 is safe to restart 2024-04-24T22:51:51.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:51 smithi192 ceph-mon[80463]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:51.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:51.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:51 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T22:51:51.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:51 smithi192 ceph-mon[80463]: Upgrade: osd.1 is safe to restart 2024-04-24T22:51:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:52 smithi063 ceph-mon[111213]: Upgrade: Updating osd.1 2024-04-24T22:51:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:51:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:52 smithi063 ceph-mon[111213]: Deploying daemon osd.1 on smithi063 2024-04-24T22:51:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:52 smithi192 ceph-mon[80463]: Upgrade: Updating osd.1 2024-04-24T22:51:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:51:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T22:51:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:51:53.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:52 smithi192 ceph-mon[80463]: Deploying daemon osd.1 on smithi063 2024-04-24T22:51:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:53 smithi063 ceph-mon[111213]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:53 smithi192 ceph-mon[80463]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- 172.21.15.63:0/3427638429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 msgr2=0x7fc50c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 --2- 172.21.15.63:0/3427638429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0fce10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc004940 tx=0x7fc4fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- 172.21.15.63:0/3427638429 shutdown_connections 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 --2- 172.21.15.63:0/3427638429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0fce10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 --2- 172.21.15.63:0/3427638429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 0x7fc50c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- 172.21.15.63:0/3427638429 >> 172.21.15.63:0/3427638429 conn(0x7fc50c0f89b0 msgr2=0x7fc50c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- 172.21.15.63:0/3427638429 shutdown_connections 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- 172.21.15.63:0/3427638429 wait complete. 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 Processor -- start 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.659+0000 7fc51476a700 1 -- start start 2024-04-24T22:51:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc51476a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 0x7fc50c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc51476a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc51476a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0721b0 con 0x7fc50c1037a0 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc51476a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c070ce0 con 0x7fc50c102e00 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44294/0 (socket says 172.21.15.63:44294) 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 -- 172.21.15.63:0/182572276 learned_addr learned my addr 172.21.15.63:0/182572276 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc512506700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 0x7fc50c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 -- 172.21.15.63:0/182572276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 msgr2=0x7fc50c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 0x7fc50c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 -- 172.21.15.63:0/182572276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc005040 con 0x7fc50c1037a0 2024-04-24T22:51:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc511d05700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009100 tx=0x7fc4fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4fc00e040 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.660+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c070f60 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.661+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4fc004290 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.661+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4fc01c3f0 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.661+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c11a0b0 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.661+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c1052d0 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.662+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc4fc013690 con 0x7fc50c1037a0 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.662+0000 7fc502ffd700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 0x7fc4f80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.663+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc4fc08ebf0 con 0x7fc50c1037a0 2024-04-24T22:51:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.663+0000 7fc512506700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 0x7fc4f80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.663+0000 7fc512506700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 0x7fc4f80705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc508004f40 tx=0x7fc50800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:54.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.667+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc4fc058040 con 0x7fc50c1037a0 2024-04-24T22:51:54.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.897+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc50c109f50 con 0x7fc4f806e140 2024-04-24T22:51:54.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.898+0000 7fc502ffd700 1 -- 172.21.15.63:0/182572276 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc50c109f50 con 0x7fc4f806e140 2024-04-24T22:51:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.900+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 msgr2=0x7fc4f80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.900+0000 7fc51476a700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 0x7fc4f80705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc508004f40 tx=0x7fc50800b040 comp rx=0 tx=0).stop 2024-04-24T22:51:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 msgr2=0x7fc50c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009100 tx=0x7fc4fc009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 shutdown_connections 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c102e00 0x7fc50c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc4f806e140 0x7fc4f80705f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 --2- 172.21.15.63:0/182572276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc50c1037a0 0x7fc50c0707a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 >> 172.21.15.63:0/182572276 conn(0x7fc50c0f89b0 msgr2=0x7fc50c102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 shutdown_connections 2024-04-24T22:51:54.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:54.901+0000 7fc51476a700 1 -- 172.21.15.63:0/182572276 wait complete. 2024-04-24T22:51:54.913 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 -- 172.21.15.63:0/1362196140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 msgr2=0x7f36a40fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 --2- 172.21.15.63:0/1362196140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a40fe230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3694004940 tx=0x7f3694009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 -- 172.21.15.63:0/1362196140 shutdown_connections 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 --2- 172.21.15.63:0/1362196140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40fe800 0x7f36a4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 --2- 172.21.15.63:0/1362196140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a40fe230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.021+0000 7f36a88cd700 1 -- 172.21.15.63:0/1362196140 >> 172.21.15.63:0/1362196140 conn(0x7f36a40f7870 msgr2=0x7f36a40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 -- 172.21.15.63:0/1362196140 shutdown_connections 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 -- 172.21.15.63:0/1362196140 wait complete. 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 Processor -- start 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 -- start start 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40fe800 0x7f36a4113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4114480 con 0x7f36a40fde60 2024-04-24T22:51:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a88cd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a410da80 con 0x7f36a40fe800 2024-04-24T22:51:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44312/0 (socket says 172.21.15.63:44312) 2024-04-24T22:51:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.022+0000 7f36a259c700 1 -- 172.21.15.63:0/4178728400 learned_addr learned my addr 172.21.15.63:0/4178728400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a259c700 1 -- 172.21.15.63:0/4178728400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40fe800 msgr2=0x7f36a4113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a259c700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40fe800 0x7f36a4113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a259c700 1 -- 172.21.15.63:0/4178728400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694005040 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a259c700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3694009380 tx=0x7f3694009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369400e040 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a410dd60 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3694009af0 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369401c630 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.023+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a410e2a0 con 0x7f36a40fde60 2024-04-24T22:51:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.024+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a4066ad0 con 0x7f36a40fde60 2024-04-24T22:51:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.025+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f369400bbc0 con 0x7f36a40fde60 2024-04-24T22:51:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.025+0000 7f3692ffd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 0x7f368c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.025+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f369408edd0 con 0x7f36a40fde60 2024-04-24T22:51:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.025+0000 7f36a1d9b700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 0x7f368c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.026+0000 7f36a1d9b700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 0x7f368c0705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f36a410ec40 tx=0x7f3698009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.030+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f369405bd50 con 0x7f36a40fde60 2024-04-24T22:51:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.265+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36a4109e90 con 0x7f368c06e140 2024-04-24T22:51:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.267+0000 7f3692ffd700 1 -- 172.21.15.63:0/4178728400 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f36a4109e90 con 0x7f368c06e140 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 msgr2=0x7f368c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 0x7f368c0705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f36a410ec40 tx=0x7f3698009ce0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 msgr2=0x7f36a41138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3694009380 tx=0x7f3694009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 shutdown_connections 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f368c06e140 0x7f368c0705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36a40fde60 0x7f36a41138f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 --2- 172.21.15.63:0/4178728400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40fe800 0x7f36a4113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 >> 172.21.15.63:0/4178728400 conn(0x7f36a40f7870 msgr2=0x7f36a40f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 shutdown_connections 2024-04-24T22:51:55.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.269+0000 7f36a88cd700 1 -- 172.21.15.63:0/4178728400 wait complete. 2024-04-24T22:51:55.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 -- 172.21.15.63:0/1541563016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594103780 msgr2=0x7f35940fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 --2- 172.21.15.63:0/1541563016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594103780 0x7f35940fcdf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f357c004940 tx=0x7f357c009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 -- 172.21.15.63:0/1541563016 shutdown_connections 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 --2- 172.21.15.63:0/1541563016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594103780 0x7f35940fcdf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 --2- 172.21.15.63:0/1541563016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594102de0 0x7f35941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 -- 172.21.15.63:0/1541563016 >> 172.21.15.63:0/1541563016 conn(0x7f35940f8990 msgr2=0x7f35940fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 -- 172.21.15.63:0/1541563016 shutdown_connections 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.381+0000 7f3598cdb700 1 -- 172.21.15.63:0/1541563016 wait complete. 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 Processor -- start 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 -- start start 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594102de0 0x7f35941138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594114470 con 0x7f3594103780 2024-04-24T22:51:55.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3598cdb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410da70 con 0x7f3594102de0 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3591d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3591d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44328/0 (socket says 172.21.15.63:44328) 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f3591d9b700 1 -- 172.21.15.63:0/3935422576 learned_addr learned my addr 172.21.15.63:0/3935422576 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.382+0000 7f359259c700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594102de0 0x7f35941138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3591d9b700 1 -- 172.21.15.63:0/3935422576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594102de0 msgr2=0x7f35941138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3591d9b700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594102de0 0x7f35941138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3591d9b700 1 -- 172.21.15.63:0/3935422576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0048f0 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3591d9b700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b0d0 tx=0x7f357c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357c021070 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357c013c00 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c005040 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357c00e070 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.383+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359410e0c0 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.384+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35940fe530 con 0x7f3594103780 2024-04-24T22:51:55.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.385+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f357c01c960 con 0x7f3594103780 2024-04-24T22:51:55.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.385+0000 7f358affd700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 0x7f35800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.385+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f357c08ee50 con 0x7f3594103780 2024-04-24T22:51:55.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.386+0000 7f359259c700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 0x7f35800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.386+0000 7f359259c700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 0x7f35800705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f358c007d60 tx=0x7f358c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.390+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f357c05bdd0 con 0x7f3594103780 2024-04-24T22:51:55.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:55 smithi063 systemd[1]: Stopping Ceph osd.1 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:51:55.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.633+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3594109ec0 con 0x7f358006e140 2024-04-24T22:51:55.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.643+0000 7f358affd700 1 -- 172.21.15.63:0/3935422576 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3594109ec0 con 0x7f358006e140 2024-04-24T22:51:55.645 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:51:55.645 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 25s ago 14m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (57s) 25s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (52s) 49s ago 14m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 25s ago 14m 75.9M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (9m) 25s ago 9m 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (9m) 25s ago 9m 13.9M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (9m) 49s ago 9m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (9m) 49s ago 9m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (5m) 25s ago 16m 501M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (4m) 49s ago 13m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:51:55.646 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 25s ago 16m 45.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (99s) 49s ago 13m 34.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (4m) 25s ago 15m 15.7M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 49s ago 13m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (31s) 25s ago 13m 15.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (12m) 25s ago 12m 218M 1557M 17.2.7-902-gc61472c8 0de407c1b3af cf01811a119f 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (12m) 25s ago 12m 183M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (11m) 49s ago 11m 236M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (11m) 49s ago 11m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (10m) 49s ago 10m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:51:55.647 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 25s ago 14m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 msgr2=0x7f35800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 0x7f35800705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f358c007d60 tx=0x7f358c00b040 comp rx=0 tx=0).stop 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 msgr2=0x7f3594113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b0d0 tx=0x7f357c009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 shutdown_connections 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3594102de0 0x7f35941138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f358006e140 0x7f35800705f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 --2- 172.21.15.63:0/3935422576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3594103780 0x7f3594113e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 >> 172.21.15.63:0/3935422576 conn(0x7f35940f8990 msgr2=0x7f35940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 shutdown_connections 2024-04-24T22:51:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.646+0000 7f3598cdb700 1 -- 172.21.15.63:0/3935422576 wait complete. 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 -- 172.21.15.63:0/3343927264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fde10 msgr2=0x7fab300fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 --2- 172.21.15.63:0/3343927264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fde10 0x7fab300fe1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab20004940 tx=0x7fab20009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 -- 172.21.15.63:0/3343927264 shutdown_connections 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 --2- 172.21.15.63:0/3343927264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fe7b0 0x7fab30106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 --2- 172.21.15.63:0/3343927264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fde10 0x7fab300fe1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.748+0000 7fab356b5700 1 -- 172.21.15.63:0/3343927264 >> 172.21.15.63:0/3343927264 conn(0x7fab300f78a0 msgr2=0x7fab300f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:55.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 -- 172.21.15.63:0/3343927264 shutdown_connections 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 -- 172.21.15.63:0/3343927264 wait complete. 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 Processor -- start 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 -- start start 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fde10 0x7fab300fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300fdc20 con 0x7fab300fe7b0 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.749+0000 7fab356b5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300fc6c0 con 0x7fab300fde10 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44334/0 (socket says 172.21.15.63:44334) 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 -- 172.21.15.63:0/858601313 learned_addr learned my addr 172.21.15.63:0/858601313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2effd700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fde10 0x7fab300fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 -- 172.21.15.63:0/858601313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fde10 msgr2=0x7fab300fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:55.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fde10 0x7fab300fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 -- 172.21.15.63:0/858601313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20005040 con 0x7fab300fe7b0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab2e7fc700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b790 tx=0x7fab24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2400f770 con 0x7fab300fe7b0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2400fda0 con 0x7fab300fe7b0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2400d310 con 0x7fab300fe7b0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab300fc9a0 con 0x7fab300fe7b0 2024-04-24T22:51:55.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.750+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3011a120 con 0x7fab300fe7b0 2024-04-24T22:51:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.751+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30066b30 con 0x7fab300fe7b0 2024-04-24T22:51:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.752+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fab24010420 con 0x7fab300fe7b0 2024-04-24T22:51:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.752+0000 7fab177fe700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 0x7fab180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.752+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fab2408d9c0 con 0x7fab300fe7b0 2024-04-24T22:51:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.752+0000 7fab2effd700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 0x7fab180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.753+0000 7fab2effd700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 0x7fab180705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab20004f90 tx=0x7fab20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:55.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:55.757+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fab24056eb0 con 0x7fab300fe7b0 2024-04-24T22:51:55.876 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:55 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[52914]: 2024-04-24T22:51:55.586+0000 7f0ff6fb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:51:55.876 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:55 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[52914]: 2024-04-24T22:51:55.586+0000 7f0ff6fb1700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-24T22:51:55.876 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:55 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[52914]: 2024-04-24T22:51:55.586+0000 7f0ff6fb1700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:51:56.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.066+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab3004c710 con 0x7fab300fe7b0 2024-04-24T22:51:56.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.067+0000 7fab177fe700 1 -- 172.21.15.63:0/858601313 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fab2405a570 con 0x7fab300fe7b0 2024-04-24T22:51:56.069 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:51:56.069 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:51:56.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:51:56.070 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:51:56.071 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-24T22:51:56.072 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-24T22:51:56.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:51:56.072 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:51:56.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 msgr2=0x7fab180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 0x7fab180705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab20004f90 tx=0x7fab20009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 msgr2=0x7fab300fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b790 tx=0x7fab24009250 comp rx=0 tx=0).stop 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 shutdown_connections 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab300fde10 0x7fab300fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fab1806e140 0x7fab180705f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 --2- 172.21.15.63:0/858601313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab300fe7b0 0x7fab300fc180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 >> 172.21.15.63:0/858601313 conn(0x7fab300f78a0 msgr2=0x7fab30103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 shutdown_connections 2024-04-24T22:51:56.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.069+0000 7fab356b5700 1 -- 172.21.15.63:0/858601313 wait complete. 2024-04-24T22:51:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:55 smithi192 ceph-mon[80463]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:55 smithi192 ceph-mon[80463]: osd.1 marked itself down and dead 2024-04-24T22:51:56.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:55 smithi063 ceph-mon[111213]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:56.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:55 smithi063 ceph-mon[111213]: osd.1 marked itself down and dead 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 -- 172.21.15.63:0/431452867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744102de0 msgr2=0x7f57441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 --2- 172.21.15.63:0/431452867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744102de0 0x7f57441031b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5738004940 tx=0x7f5738009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 -- 172.21.15.63:0/431452867 shutdown_connections 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 --2- 172.21.15.63:0/431452867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744103780 0x7f57440fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 --2- 172.21.15.63:0/431452867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744102de0 0x7f57441031b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 -- 172.21.15.63:0/431452867 >> 172.21.15.63:0/431452867 conn(0x7f57440f8950 msgr2=0x7f57440fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 -- 172.21.15.63:0/431452867 shutdown_connections 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.178+0000 7f574b0ca700 1 -- 172.21.15.63:0/431452867 wait complete. 2024-04-24T22:51:56.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.179+0000 7f574b0ca700 1 Processor -- start 2024-04-24T22:51:56.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.179+0000 7f574b0ca700 1 -- start start 2024-04-24T22:51:56.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f574b0ca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744102de0 0x7f5744113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f574b0ca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f574b0ca700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744114710 con 0x7f5744103780 2024-04-24T22:51:56.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f574b0ca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574410db50 con 0x7f5744102de0 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44354/0 (socket says 172.21.15.63:44354) 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 -- 172.21.15.63:0/4208242462 learned_addr learned my addr 172.21.15.63:0/4208242462 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5748e66700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744102de0 0x7f5744113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 -- 172.21.15.63:0/4208242462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744102de0 msgr2=0x7f5744113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744102de0 0x7f5744113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.180+0000 7f5743fff700 1 -- 172.21.15.63:0/4208242462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738005040 con 0x7f5744103780 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f5743fff700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5730002a50 tx=0x7f5730004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5730007cc0 con 0x7f5744103780 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f573000f040 con 0x7f5744103780 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574410de30 con 0x7f5744103780 2024-04-24T22:51:56.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574410e3f0 con 0x7f5744103780 2024-04-24T22:51:56.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.181+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573000d9c0 con 0x7f5744103780 2024-04-24T22:51:56.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.182+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57441000f0 con 0x7f5744103780 2024-04-24T22:51:56.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.183+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5730009730 con 0x7f5744103780 2024-04-24T22:51:56.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.183+0000 7f57417fa700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 0x7f5734070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.184+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f573008d7a0 con 0x7f5744103780 2024-04-24T22:51:56.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.184+0000 7f5748e66700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 0x7f5734070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.184+0000 7f5748e66700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 0x7f5734070600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f573800b3b0 tx=0x7f5738009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.189+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f573005a6a0 con 0x7f5744103780 2024-04-24T22:51:56.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.442+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f574410f1d0 con 0x7f5744103780 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.443+0000 7f57417fa700 1 -- 172.21.15.63:0/4208242462 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f573005a230 con 0x7f5744103780 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:51:56.445 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:51:56.446 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:51:56.447 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:56.448 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:56.449 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.445+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 msgr2=0x7f5734070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.445+0000 7f574b0ca700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 0x7f5734070600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f573800b3b0 tx=0x7f5738009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.445+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 msgr2=0x7f57441140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.445+0000 7f574b0ca700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5730002a50 tx=0x7f5730004600 comp rx=0 tx=0).stop 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 shutdown_connections 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5744102de0 0x7f5744113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f573406e150 0x7f5734070600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 --2- 172.21.15.63:0/4208242462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5744103780 0x7f57441140c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 >> 172.21.15.63:0/4208242462 conn(0x7f57440f8950 msgr2=0x7f57441073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 shutdown_connections 2024-04-24T22:51:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.446+0000 7f574b0ca700 1 -- 172.21.15.63:0/4208242462 wait complete. 2024-04-24T22:51:56.452 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:51:56.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 -- 172.21.15.63:0/1007017896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 msgr2=0x7f31f40fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 --2- 172.21.15.63:0/1007017896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fb970 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31e4004940 tx=0x7f31e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 -- 172.21.15.63:0/1007017896 shutdown_connections 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 --2- 172.21.15.63:0/1007017896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 --2- 172.21.15.63:0/1007017896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fb970 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 -- 172.21.15.63:0/1007017896 >> 172.21.15.63:0/1007017896 conn(0x7f31f4073350 msgr2=0x7f31f4073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 -- 172.21.15.63:0/1007017896 shutdown_connections 2024-04-24T22:51:56.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 -- 172.21.15.63:0/1007017896 wait complete. 2024-04-24T22:51:56.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.552+0000 7f31fc164700 1 Processor -- start 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31fc164700 1 -- start start 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31fc164700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31fc164700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f40fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31fc164700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40fe480 con 0x7f31f40fb5a0 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31fc164700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40fcfb0 con 0x7f31f40fbf40 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44384/0 (socket says 172.21.15.63:44384) 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 -- 172.21.15.63:0/2115739231 learned_addr learned my addr 172.21.15.63:0/2115739231 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f96ff700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f40fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 -- 172.21.15.63:0/2115739231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 msgr2=0x7f31f40fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f40fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 -- 172.21.15.63:0/2115739231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4005040 con 0x7f31f40fb5a0 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f96ff700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f40fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:51:56.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31f9f00700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b3b0 tx=0x7f31e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.553+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e400e040 con 0x7f31f40fb5a0 2024-04-24T22:51:56.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.554+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31e400bbc0 con 0x7f31f40fb5a0 2024-04-24T22:51:56.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.554+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f40fd230 con 0x7f31f40fb5a0 2024-04-24T22:51:56.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.554+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e401c3f0 con 0x7f31f40fb5a0 2024-04-24T22:51:56.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.554+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f411a090 con 0x7f31f40fb5a0 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.555+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f40fe670 con 0x7f31f40fb5a0 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.555+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f31e400bd30 con 0x7f31f40fb5a0 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.555+0000 7f31ea7fc700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 0x7f31e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.556+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f31e408ed20 con 0x7f31f40fb5a0 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.556+0000 7f31f96ff700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 0x7f31e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.556+0000 7f31f96ff700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 0x7f31e00705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009fd0 tx=0x7f31f0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.560+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f31e40580f0 con 0x7f31f40fb5a0 2024-04-24T22:51:56.770 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:56 smithi063 podman[128017]: 2024-04-24 22:51:56.433053723 +0000 UTC m=+0.959418672 container died cf01811a119f79dec7f8cc181f7c6f5e2b74f0e900c53aeffccd3746b4eb5875 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T22:51:56.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.787+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31f4101120 con 0x7f31e006e140 2024-04-24T22:51:56.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.788+0000 7f31ea7fc700 1 -- 172.21.15.63:0/2115739231 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f31f4101120 con 0x7f31e006e140 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:51:56.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:51:56.791 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:51:56.791 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "7/21 daemons upgraded", 2024-04-24T22:51:56.791 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:51:56.791 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:51:56.791 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:51:56.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 msgr2=0x7f31e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 0x7f31e00705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009fd0 tx=0x7f31f0009ee0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 msgr2=0x7f31f40fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b3b0 tx=0x7f31e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 shutdown_connections 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f31e006e140 0x7f31e00705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f40fb5a0 0x7f31f40fc530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 --2- 172.21.15.63:0/2115739231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31f40fbf40 0x7f31f40fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 >> 172.21.15.63:0/2115739231 conn(0x7f31f4073350 msgr2=0x7f31f4100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 shutdown_connections 2024-04-24T22:51:56.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.790+0000 7f31fc164700 1 -- 172.21.15.63:0/2115739231 wait complete. 2024-04-24T22:51:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.901+0000 7fdc92ffd700 1 -- 172.21.15.63:0/3018527812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8400bc70 con 0x7fdc94109db0 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.902+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3018527812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109db0 msgr2=0x7fdc94103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.902+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3018527812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109db0 0x7fdc94103420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc84004940 tx=0x7fdc84009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3018527812 shutdown_connections 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3018527812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109db0 0x7fdc94103420 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc84004940 tx=0x7fdc84009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3018527812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc94109410 0x7fdc941097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3018527812 >> 172.21.15.63:0/3018527812 conn(0x7fdc94072c00 msgr2=0x7fdc94075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3018527812 shutdown_connections 2024-04-24T22:51:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.903+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3018527812 wait complete. 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.904+0000 7fdc9a6b3700 1 Processor -- start 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.904+0000 7fdc9a6b3700 1 -- start start 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.904+0000 7fdc9a6b3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 0x7fdc9411a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc9a6b3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 0x7fdc94114350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc9a6b3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9411ad30 con 0x7fdc94109410 2024-04-24T22:51:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc9a6b3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc94114890 con 0x7fdc9411a720 2024-04-24T22:51:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc937fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 0x7fdc94114350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc937fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 0x7fdc94114350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.63:40286/0 (socket says 172.21.15.63:40286) 2024-04-24T22:51:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc937fe700 1 -- 172.21.15.63:0/3975515221 learned_addr learned my addr 172.21.15.63:0/3975515221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:51:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.905+0000 7fdc93fff700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 0x7fdc9411a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.906+0000 7fdc93fff700 1 -- 172.21.15.63:0/3975515221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 msgr2=0x7fdc94114350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.906+0000 7fdc93fff700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 0x7fdc94114350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.906+0000 7fdc93fff700 1 -- 172.21.15.63:0/3975515221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc84005040 con 0x7fdc94109410 2024-04-24T22:51:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.906+0000 7fdc93fff700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 0x7fdc9411a1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800b530 tx=0x7fdc880090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.907+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8800f6f0 con 0x7fdc94109410 2024-04-24T22:51:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.907+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc8800fd20 con 0x7fdc94109410 2024-04-24T22:51:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.907+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc94114a30 con 0x7fdc94109410 2024-04-24T22:51:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.907+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc94114ff0 con 0x7fdc94109410 2024-04-24T22:51:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.907+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc88011dd0 con 0x7fdc94109410 2024-04-24T22:51:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.909+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc780052b0 con 0x7fdc94109410 2024-04-24T22:51:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.909+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc8800f850 con 0x7fdc94109410 2024-04-24T22:51:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.910+0000 7fdc90ff9700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 0x7fdc7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:51:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.910+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdc8808d4d0 con 0x7fdc94109410 2024-04-24T22:51:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.910+0000 7fdc937fe700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 0x7fdc7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:51:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.910+0000 7fdc937fe700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 0x7fdc7c0705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b3b0 tx=0x7fdc84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:51:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:56.914+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdc8805a4f0 con 0x7fdc94109410 2024-04-24T22:51:57.020 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128017]: 2024-04-24 22:51:57.001980882 +0000 UTC m=+1.528345830 container cleanup cf01811a119f79dec7f8cc181f7c6f5e2b74f0e900c53aeffccd3746b4eb5875 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T22:51:57.021 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 bash[128017]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: osdmap e55: 6 total, 5 up, 6 in 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/858601313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:57.021 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:56 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/4208242462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:51:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:51:57.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: osdmap e55: 6 total, 5 up, 6 in 2024-04-24T22:51:57.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/858601313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:51:57.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:56 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/4208242462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:51:57.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.224+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdc78005560 con 0x7fdc94109410 2024-04-24T22:51:57.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.224+0000 7fdc90ff9700 1 -- 172.21.15.63:0/3975515221 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1091 (secure 0 0 0) 0x7fdc8805a080 con 0x7fdc94109410 2024-04-24T22:51:57.226 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 9 pgs peering 2024-04-24T22:51:57.226 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:51:57.226 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: osd.1 (root=default,host=smithi063) is down 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.6 is stuck peering for 4m, current state peering, last acting [3,4] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.9 is stuck peering for 4m, current state peering, last acting [4,0] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.a is stuck peering for 5m, current state peering, last acting [4,3] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.d is stuck peering for 5m, current state peering, last acting [3,2] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.15 is stuck peering for 4m, current state peering, last acting [3,0] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 3.0 is stuck peering for 4m, current state peering, last acting [2,4] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 3.2 is stuck peering for 4m, current state peering, last acting [5,3] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 3.4 is stuck peering for 4m, current state peering, last acting [2,3] 2024-04-24T22:51:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: pg 3.19 is stuck peering for 4m, current state peering, last acting [4,3] 2024-04-24T22:51:57.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 msgr2=0x7fdc7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:57.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 0x7fdc7c0705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b3b0 tx=0x7fdc84009e30 comp rx=0 tx=0).stop 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 msgr2=0x7fdc9411a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 0x7fdc9411a1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800b530 tx=0x7fdc880090f0 comp rx=0 tx=0).stop 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 shutdown_connections 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdc7c06e140 0x7fdc7c0705f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc94109410 0x7fdc9411a1e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 --2- 172.21.15.63:0/3975515221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc9411a720 0x7fdc94114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.226+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 >> 172.21.15.63:0/3975515221 conn(0x7fdc94072c00 msgr2=0x7fdc94074ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.227+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 shutdown_connections 2024-04-24T22:51:57.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:51:57.227+0000 7fdc9a6b3700 1 -- 172.21.15.63:0/3975515221 wait complete. 2024-04-24T22:51:57.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128080]: 2024-04-24 22:51:57.2956784 +0000 UTC m=+0.857720541 container remove cf01811a119f79dec7f8cc181f7c6f5e2b74f0e900c53aeffccd3746b4eb5875 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux ) 2024-04-24T22:51:57.900 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:57 smithi063 ceph-mon[111213]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:57.900 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:57 smithi063 ceph-mon[111213]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:57.900 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:57 smithi063 ceph-mon[111213]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-24T22:51:57.900 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:57 smithi063 ceph-mon[111213]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T22:51:57.900 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:57 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3975515221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:51:57.901 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128223]: 2024-04-24 22:51:57.539186542 +0000 UTC m=+0.018933565 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:51:57.901 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128223]: 2024-04-24 22:51:57.671677289 +0000 UTC m=+0.151424302 container create 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T22:51:57.901 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128223]: 2024-04-24 22:51:57.899558048 +0000 UTC m=+0.379305062 container init 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T22:51:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:57 smithi192 ceph-mon[80463]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:51:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:57 smithi192 ceph-mon[80463]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:51:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:57 smithi192 ceph-mon[80463]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-24T22:51:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:57 smithi192 ceph-mon[80463]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T22:51:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:57 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3975515221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:51:58.208 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128223]: 2024-04-24 22:51:57.903217156 +0000 UTC m=+0.382964178 container start 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:51:58.208 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:57 smithi063 podman[128223]: 2024-04-24 22:51:57.999654122 +0000 UTC m=+0.479401135 container attach 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:51:58.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:58 smithi063 podman[128223]: 2024-04-24 22:51:58.207703413 +0000 UTC m=+0.687450427 container died 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T22:51:59.023 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:58 smithi063 podman[128249]: 2024-04-24 22:51:58.778262116 +0000 UTC m=+0.565192201 container remove 78fd09b8f6a0bd798ddb921c27e17188123fad25e2d73aecc56e65a033765e48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T22:51:59.023 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:58 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-24T22:51:59.023 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:58 smithi063 systemd[1]: Stopped Ceph osd.1 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:51:59.023 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:58 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.1.service: Consumed 58.560s CPU time. 2024-04-24T22:51:59.303 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 systemd[1]: Starting Ceph osd.1 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:51:59.303 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.145595154 +0000 UTC m=+0.018879597 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:51:59.303 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.254532357 +0000 UTC m=+0.127816761 container create 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T22:51:59.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.490672647 +0000 UTC m=+0.363957060 container init 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T22:51:59.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.493961546 +0000 UTC m=+0.367245951 container start 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:51:59.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.557128669 +0000 UTC m=+0.430413082 container attach 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T22:52:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:51:59 smithi192 ceph-mon[80463]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-mon[111213]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:52:00.304 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 bash[128351]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate[128366]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T22:52:00.305 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:51:59 smithi063 podman[128351]: 2024-04-24 22:51:59.97893654 +0000 UTC m=+0.852220955 container died 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:52:00.888 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:00 smithi063 podman[128489]: 2024-04-24 22:52:00.627594318 +0000 UTC m=+0.638166296 container remove 0c0d1955894cdf9178e74927ddbf2040a87bacd48e76966611fd32a1bb304504 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T22:52:00.888 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:00 smithi063 podman[128553]: 2024-04-24 22:52:00.747556538 +0000 UTC m=+0.019035210 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:52:00.888 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:00 smithi063 podman[128553]: 2024-04-24 22:52:00.887004207 +0000 UTC m=+0.158482902 container create 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T22:52:01.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:01 smithi063 ceph-mon[111213]: pgmap v70: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:01.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:01.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:01 smithi192 ceph-mon[80463]: pgmap v70: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:01.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 podman[128553]: 2024-04-24 22:52:01.139962107 +0000 UTC m=+0.411440762 container init 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T22:52:01.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 podman[128553]: 2024-04-24 22:52:01.143604032 +0000 UTC m=+0.415082676 container start 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:52:01.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 ceph-osd[128572]: -- 172.21.15.63:0/1391164473 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b23ddd680 con 0x564b23da6c00 2024-04-24T22:52:01.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 bash[128553]: 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e 2024-04-24T22:52:01.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 systemd[1]: Started Ceph osd.1 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:52:02.053 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:01 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T22:52:01.682+0000 7f88a26ef7c0 -1 Falling back to public interface 2024-04-24T22:52:02.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:02 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:02.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:02 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:02.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:02 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:02.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:02 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:02.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:02 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:02.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:02 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:03 smithi063 ceph-mon[111213]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:03 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:03 smithi063 ceph-mon[111213]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-24T22:52:03.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:03 smithi192 ceph-mon[80463]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:03.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:03 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:03.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:03 smithi192 ceph-mon[80463]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-24T22:52:05.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:05 smithi063 ceph-mon[111213]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:05.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:05 smithi192 ceph-mon[80463]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:07.053 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:06 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T22:52:06.565+0000 7f88a26ef7c0 -1 osd.1 54 log_to_monitors true 2024-04-24T22:52:07.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:07 smithi192 ceph-mon[80463]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:07.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:07.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:07.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:07 smithi192 ceph-mon[80463]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:52:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:07 smithi063 ceph-mon[111213]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:07 smithi063 ceph-mon[111213]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T22:52:08.534 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T22:52:08.279+0000 7f888fb3f700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:08.535 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:08 smithi063 ceph-mon[111213]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' 2024-04-24T22:52:08.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T22:52:08.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T22:52:08.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:52:08.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:08.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:08.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:08 smithi192 ceph-mon[80463]: from='osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179]' entity='osd.1' 2024-04-24T22:52:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:09 smithi192 ceph-mon[80463]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:09 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:52:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:09 smithi192 ceph-mon[80463]: osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179] boot 2024-04-24T22:52:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:09 smithi192 ceph-mon[80463]: osdmap e58: 6 total, 6 up, 6 in 2024-04-24T22:52:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:09 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:52:10.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:09 smithi063 ceph-mon[111213]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:10.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:09 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:52:10.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:09 smithi063 ceph-mon[111213]: osd.1 [v2:172.21.15.63:6810/2869549179,v1:172.21.15.63:6811/2869549179] boot 2024-04-24T22:52:10.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:09 smithi063 ceph-mon[111213]: osdmap e58: 6 total, 6 up, 6 in 2024-04-24T22:52:10.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:09 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T22:52:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:11 smithi063 ceph-mon[111213]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:11 smithi063 ceph-mon[111213]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T22:52:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:11 smithi192 ceph-mon[80463]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:11.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:11 smithi192 ceph-mon[80463]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T22:52:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:12.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:12 smithi063 ceph-mon[111213]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-24T22:52:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:12.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:12.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:12 smithi192 ceph-mon[80463]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-24T22:52:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:13 smithi063 ceph-mon[111213]: pgmap v79: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-24T22:52:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:13 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:13.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:13 smithi192 ceph-mon[80463]: pgmap v79: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-24T22:52:13.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:13 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:14.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:14 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded) 2024-04-24T22:52:14.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:14 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded) 2024-04-24T22:52:15.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:15 smithi192 ceph-mon[80463]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:15.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:15.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:15.704 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:15 smithi063 ceph-mon[111213]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:15.704 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:15.704 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:17.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:17 smithi192 ceph-mon[80463]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:17.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:17 smithi063 ceph-mon[111213]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:19 smithi192 ceph-mon[80463]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:19 smithi063 ceph-mon[111213]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:21.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:21 smithi192 ceph-mon[80463]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:21.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:21 smithi063 ceph-mon[111213]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:23 smithi192 ceph-mon[80463]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:23 smithi063 ceph-mon[111213]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:25.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:25 smithi192 ceph-mon[80463]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:25 smithi063 ceph-mon[111213]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:27.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:27 smithi063 ceph-mon[111213]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:27.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:27.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:27 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:27.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:27 smithi063 ceph-mon[111213]: Upgrade: osd.2 is safe to restart 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 -- 172.21.15.63:0/695070762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 msgr2=0x7fd4dc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/695070762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc1031b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc004940 tx=0x7fd4cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 -- 172.21.15.63:0/695070762 shutdown_connections 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/695070762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 0x7fd4dc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/695070762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc1031b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 -- 172.21.15.63:0/695070762 >> 172.21.15.63:0/695070762 conn(0x7fd4dc0f8950 msgr2=0x7fd4dc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 -- 172.21.15.63:0/695070762 shutdown_connections 2024-04-24T22:52:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.338+0000 7fd4e29cb700 1 -- 172.21.15.63:0/695070762 wait complete. 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 Processor -- start 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 -- start start 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 0x7fd4dc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc070dc0 con 0x7fd4dc102de0 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4e29cb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc070f00 con 0x7fd4dc103780 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4dbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4dbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59336/0 (socket says 172.21.15.63:59336) 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.339+0000 7fd4dbfff700 1 -- 172.21.15.63:0/3774629252 learned_addr learned my addr 172.21.15.63:0/3774629252 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4db7fe700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 0x7fd4dc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4dbfff700 1 -- 172.21.15.63:0/3774629252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 msgr2=0x7fd4dc0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4dbfff700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 0x7fd4dc0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4dbfff700 1 -- 172.21.15.63:0/3774629252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc005040 con 0x7fd4dc102de0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4dbfff700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b3b0 tx=0x7fd4cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4cc00e040 con 0x7fd4dc102de0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc119e00 con 0x7fd4dc102de0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4cc00bbc0 con 0x7fd4dc102de0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4cc01c3f0 con 0x7fd4dc102de0 2024-04-24T22:52:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.340+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc11a2e0 con 0x7fd4dc102de0 2024-04-24T22:52:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.341+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc1000f0 con 0x7fd4dc102de0 2024-04-24T22:52:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.342+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd4cc00bd30 con 0x7fd4dc102de0 2024-04-24T22:52:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.343+0000 7fd4d8ff9700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 0x7fd4c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.343+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd4cc08ed20 con 0x7fd4dc102de0 2024-04-24T22:52:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.343+0000 7fd4db7fe700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 0x7fd4c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.343+0000 7fd4db7fe700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 0x7fd4c40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0006fd0 tx=0x7fd4d0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:27.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.348+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd4cc0580f0 con 0x7fd4dc102de0 2024-04-24T22:52:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:27 smithi192 ceph-mon[80463]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:27 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T22:52:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:27 smithi192 ceph-mon[80463]: Upgrade: osd.2 is safe to restart 2024-04-24T22:52:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.584+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4dc04bed0 con 0x7fd4c406e140 2024-04-24T22:52:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.585+0000 7fd4d8ff9700 1 -- 172.21.15.63:0/3774629252 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd4dc04bed0 con 0x7fd4c406e140 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 msgr2=0x7fd4c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 0x7fd4c40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0006fd0 tx=0x7fd4d0008040 comp rx=0 tx=0).stop 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 msgr2=0x7fd4dc070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b3b0 tx=0x7fd4cc009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 shutdown_connections 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4c406e140 0x7fd4c40705f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4dc102de0 0x7fd4dc070260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 --2- 172.21.15.63:0/3774629252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc103780 0x7fd4dc0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 >> 172.21.15.63:0/3774629252 conn(0x7fd4dc0f8950 msgr2=0x7fd4dc1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:27.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 shutdown_connections 2024-04-24T22:52:27.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.587+0000 7fd4e29cb700 1 -- 172.21.15.63:0/3774629252 wait complete. 2024-04-24T22:52:27.600 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:52:27.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/4244591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4101c70 msgr2=0x7fe1c4065640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/4244591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4101c70 0x7fe1c4065640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4004940 tx=0x7fe1b4009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/4244591868 shutdown_connections 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/4244591868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4065b80 0x7fe1c4065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/4244591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4101c70 0x7fe1c4065640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.693+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/4244591868 >> 172.21.15.63:0/4244591868 conn(0x7fe1c4071d00 msgr2=0x7fe1c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/4244591868 shutdown_connections 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/4244591868 wait complete. 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 Processor -- start 2024-04-24T22:52:27.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 -- start start 2024-04-24T22:52:27.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 0x7fe1c411a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c411ae10 con 0x7fe1c4065b80 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1cb9e2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c41141e0 con 0x7fe1c4101c70 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1c977e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1c977e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59350/0 (socket says 172.21.15.63:59350) 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.694+0000 7fe1c977e700 1 -- 172.21.15.63:0/1444135758 learned_addr learned my addr 172.21.15.63:0/1444135758 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c8f7d700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 0x7fe1c411a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c977e700 1 -- 172.21.15.63:0/1444135758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 msgr2=0x7fe1c411a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c977e700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 0x7fe1c411a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c977e700 1 -- 172.21.15.63:0/1444135758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4005040 con 0x7fe1c4065b80 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c8f7d700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 0x7fe1c411a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1c977e700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009150 tx=0x7fe1b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b400e040 con 0x7fe1c4065b80 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c4114460 con 0x7fe1c4065b80 2024-04-24T22:52:27.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1b4013420 con 0x7fe1c4065b80 2024-04-24T22:52:27.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b401c630 con 0x7fe1c4065b80 2024-04-24T22:52:27.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.695+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c41149a0 con 0x7fe1c4065b80 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.697+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c40fc330 con 0x7fe1c4065b80 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.697+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe1b4013590 con 0x7fe1c4065b80 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.697+0000 7fe1b9ffb700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 0x7fe1b0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.698+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe1b408ed50 con 0x7fe1c4065b80 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.698+0000 7fe1c8f7d700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 0x7fe1b0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:27.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.698+0000 7fe1c8f7d700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 0x7fe1b0070600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:27.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.703+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe1b405bcd0 con 0x7fe1c4065b80 2024-04-24T22:52:27.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.934+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1c404bed0 con 0x7fe1b006e150 2024-04-24T22:52:27.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.935+0000 7fe1b9ffb700 1 -- 172.21.15.63:0/1444135758 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe1c404bed0 con 0x7fe1b006e150 2024-04-24T22:52:27.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 msgr2=0x7fe1b0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 0x7fe1b0070600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 msgr2=0x7fe1c411a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009150 tx=0x7fe1b4009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 shutdown_connections 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe1b006e150 0x7fe1b0070600 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1c4065b80 0x7fe1c411a280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 --2- 172.21.15.63:0/1444135758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c4101c70 0x7fe1c411a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 >> 172.21.15.63:0/1444135758 conn(0x7fe1c4071d00 msgr2=0x7fe1c40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 shutdown_connections 2024-04-24T22:52:27.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:27.938+0000 7fe1cb9e2700 1 -- 172.21.15.63:0/1444135758 wait complete. 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 -- 172.21.15.63:0/2288012839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 msgr2=0x7f7308102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 --2- 172.21.15.63:0/2288012839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f7308102160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009ce0 tx=0x7f72f8004d10 comp rx=0 tx=0).stop 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 -- 172.21.15.63:0/2288012839 shutdown_connections 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 --2- 172.21.15.63:0/2288012839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 0x7f73080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 --2- 172.21.15.63:0/2288012839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f7308102160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.043+0000 7f730cb41700 1 -- 172.21.15.63:0/2288012839 >> 172.21.15.63:0/2288012839 conn(0x7f7308071d00 msgr2=0x7f7308072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 -- 172.21.15.63:0/2288012839 shutdown_connections 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 -- 172.21.15.63:0/2288012839 wait complete. 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 Processor -- start 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 -- start start 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 0x7f7308106b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308107630 con 0x7f7308101d90 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f730cb41700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308107770 con 0x7f73080fbd40 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f7305d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f7305d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59376/0 (socket says 172.21.15.63:59376) 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.044+0000 7f7305d9b700 1 -- 172.21.15.63:0/4005778036 learned_addr learned my addr 172.21.15.63:0/4005778036 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f730659c700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 0x7f7308106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f7305d9b700 1 -- 172.21.15.63:0/4005778036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 msgr2=0x7f7308106b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f7305d9b700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 0x7f7308106b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f7305d9b700 1 -- 172.21.15.63:0/4005778036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8009940 con 0x7f7308101d90 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f7305d9b700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c830 tx=0x7f72f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f00106f0 con 0x7f7308101d90 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730811a050 con 0x7f7308101d90 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72f0010d20 con 0x7f7308101d90 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f00113a0 con 0x7f7308101d90 2024-04-24T22:52:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.045+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730811a4d0 con 0x7f7308101d90 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.046+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73080feff0 con 0x7f7308101d90 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.047+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f72f0012400 con 0x7f7308101d90 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.047+0000 7f72feffd700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 0x7f72f4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.047+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f72f008da20 con 0x7f7308101d90 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.047+0000 7f730659c700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 0x7f72f4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.048+0000 7f730659c700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 0x7f72f4070600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b740 tx=0x7f72f8004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.052+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f72f0056e70 con 0x7f7308101d90 2024-04-24T22:52:28.193 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:28 smithi063 ceph-mon[111213]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:28.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.275+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7308109ef0 con 0x7f72f406e150 2024-04-24T22:52:28.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.286+0000 7f72feffd700 1 -- 172.21.15.63:0/4005778036 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7308109ef0 con 0x7f72f406e150 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (4m) 21s ago 15m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (90s) 21s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (85s) 82s ago 14m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 21s ago 15m 81.1M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (9m) 21s ago 9m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (10m) 21s ago 10m 14.0M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (9m) 82s ago 9m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (9m) 82s ago 9m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:52:28.288 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (5m) 21s ago 16m 502M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (5m) 82s ago 14m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 21s ago 17m 46.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (2m) 82s ago 14m 34.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (4m) 21s ago 15m 15.7M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 82s ago 14m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (64s) 21s ago 13m 90.1M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (27s) 21s ago 13m 14.8M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (12m) 21s ago 12m 181M 1557M 17.2.7-902-gc61472c8 0de407c1b3af c27ca20a4c78 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (12m) 82s ago 12m 236M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (11m) 82s ago 11m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (11m) 82s ago 11m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:52:28.289 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 21s ago 14m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 msgr2=0x7f72f4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 0x7f72f4070600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b740 tx=0x7f72f8004d10 comp rx=0 tx=0).stop 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 msgr2=0x7f73081070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c830 tx=0x7f72f0004470 comp rx=0 tx=0).stop 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 shutdown_connections 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73080fbd40 0x7f7308106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f72f406e150 0x7f72f4070600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 --2- 172.21.15.63:0/4005778036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7308101d90 0x7f73081070a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 >> 172.21.15.63:0/4005778036 conn(0x7f7308071d00 msgr2=0x7f73080fb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 shutdown_connections 2024-04-24T22:52:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.288+0000 7f730cb41700 1 -- 172.21.15.63:0/4005778036 wait complete. 2024-04-24T22:52:28.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 -- 172.21.15.63:0/216799520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638102670 msgr2=0x7f36380fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 --2- 172.21.15.63:0/216799520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638102670 0x7f36380fb600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f362c004940 tx=0x7f362c009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:28.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 -- 172.21.15.63:0/216799520 shutdown_connections 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 --2- 172.21.15.63:0/216799520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638102670 0x7f36380fb600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 --2- 172.21.15.63:0/216799520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638101cd0 0x7f36381020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 -- 172.21.15.63:0/216799520 >> 172.21.15.63:0/216799520 conn(0x7f3638071460 msgr2=0x7f3638071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 -- 172.21.15.63:0/216799520 shutdown_connections 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.402+0000 7f363cc46700 1 -- 172.21.15.63:0/216799520 wait complete. 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 Processor -- start 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 -- start start 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638102670 0x7f3638116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638116950 con 0x7f3638101cd0 2024-04-24T22:52:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363cc46700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363810fd20 con 0x7f3638102670 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59402/0 (socket says 172.21.15.63:59402) 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f363659c700 1 -- 172.21.15.63:0/871284006 learned_addr learned my addr 172.21.15.63:0/871284006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.403+0000 7f3635d9b700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638102670 0x7f3638116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363659c700 1 -- 172.21.15.63:0/871284006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638102670 msgr2=0x7f3638116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363659c700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638102670 0x7f3638116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363659c700 1 -- 172.21.15.63:0/871284006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c005040 con 0x7f3638101cd0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363659c700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3628002a80 tx=0x7f36280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3628007cc0 con 0x7f3638101cd0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3638110000 con 0x7f3638101cd0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f362800f040 con 0x7f3638101cd0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36280096f0 con 0x7f3638101cd0 2024-04-24T22:52:28.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.404+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3638110540 con 0x7f3638101cd0 2024-04-24T22:52:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.405+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3638065200 con 0x7f3638101cd0 2024-04-24T22:52:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.406+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f362800d690 con 0x7f3638101cd0 2024-04-24T22:52:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.406+0000 7f3626ffd700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 0x7f36200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.406+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f362808d280 con 0x7f3638101cd0 2024-04-24T22:52:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.406+0000 7f3635d9b700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 0x7f36200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.407+0000 7f3635d9b700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 0x7f36200705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3638111660 tx=0x7f362c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.411+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f362805a180 con 0x7f3638101cd0 2024-04-24T22:52:28.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:28 smithi192 ceph-mon[80463]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:28.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.727+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f363804bed0 con 0x7f3638101cd0 2024-04-24T22:52:28.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.728+0000 7f3626ffd700 1 -- 172.21.15.63:0/871284006 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f3628059d10 con 0x7f3638101cd0 2024-04-24T22:52:28.729 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:52:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:52:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-24T22:52:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:52:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:52:28.731 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 msgr2=0x7f36200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 0x7f36200705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3638111660 tx=0x7f362c009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 msgr2=0x7f3638115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3628002a80 tx=0x7f36280046b0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 shutdown_connections 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f362006e140 0x7f36200705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3638101cd0 0x7f3638115dc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 --2- 172.21.15.63:0/871284006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3638102670 0x7f3638116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 >> 172.21.15.63:0/871284006 conn(0x7f3638071460 msgr2=0x7f36380f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 shutdown_connections 2024-04-24T22:52:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.730+0000 7f363cc46700 1 -- 172.21.15.63:0/871284006 wait complete. 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 -- 172.21.15.63:0/1826994834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc102de0 msgr2=0x7f6ebc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/1826994834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc102de0 0x7f6ebc1031b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004940 tx=0x7f6eb4009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 -- 172.21.15.63:0/1826994834 shutdown_connections 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/1826994834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc103780 0x7f6ebc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/1826994834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc102de0 0x7f6ebc1031b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 -- 172.21.15.63:0/1826994834 >> 172.21.15.63:0/1826994834 conn(0x7f6ebc0f8990 msgr2=0x7f6ebc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 -- 172.21.15.63:0/1826994834 shutdown_connections 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 -- 172.21.15.63:0/1826994834 wait complete. 2024-04-24T22:52:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.851+0000 7f6ec20a6700 1 Processor -- start 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ec20a6700 1 -- start start 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ec20a6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc102de0 0x7f6ebc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ec20a6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ec20a6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc114500 con 0x7f6ebc103780 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ec20a6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc10db50 con 0x7f6ebc102de0 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37266/0 (socket says 172.21.15.63:37266) 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 -- 172.21.15.63:0/4088795966 learned_addr learned my addr 172.21.15.63:0/4088795966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:28.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebb7fe700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc102de0 0x7f6ebc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 -- 172.21.15.63:0/4088795966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc102de0 msgr2=0x7f6ebc113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc102de0 0x7f6ebc113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.852+0000 7f6ebaffd700 1 -- 172.21.15.63:0/4088795966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb4005040 con 0x7f6ebc103780 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6ebaffd700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4002a50 tx=0x7f6ea4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea4007cc0 con 0x7f6ebc103780 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc10de30 con 0x7f6ebc103780 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ea400f040 con 0x7f6ebc103780 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea400d9c0 con 0x7f6ebc103780 2024-04-24T22:52:28.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.853+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc10e3f0 con 0x7f6ebc103780 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.854+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc1000f0 con 0x7f6ebc103780 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.854+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6ea4009730 con 0x7f6ebc103780 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.855+0000 7f6eb3fff700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 0x7f6ea80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.855+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6ea408d8e0 con 0x7f6ebc103780 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.855+0000 7f6ebb7fe700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 0x7f6ea80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.855+0000 7f6ebb7fe700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 0x7f6ea80705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004f90 tx=0x7f6eb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:28.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:28.859+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6ea405a740 con 0x7f6ebc103780 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.119+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ebc0fe530 con 0x7f6ebc103780 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.119+0000 7f6eb3fff700 1 -- 172.21.15.63:0/4088795966 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6ea405a2d0 con 0x7f6ebc103780 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:52:29.121 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:52:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:52:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 msgr2=0x7f6ea80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 0x7f6ea80705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004f90 tx=0x7f6eb4009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 msgr2=0x7f6ebc113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4002a50 tx=0x7f6ea4004600 comp rx=0 tx=0).stop 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 shutdown_connections 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ebc102de0 0x7f6ebc113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6ea806e140 0x7f6ea80705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 --2- 172.21.15.63:0/4088795966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ebc103780 0x7f6ebc113eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 >> 172.21.15.63:0/4088795966 conn(0x7f6ebc0f8990 msgr2=0x7f6ebc107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 shutdown_connections 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.121+0000 7f6ec20a6700 1 -- 172.21.15.63:0/4088795966 wait complete. 2024-04-24T22:52:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 -- 172.21.15.63:0/295118978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 msgr2=0x7f18b0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 --2- 172.21.15.63:0/295118978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18ac004940 tx=0x7f18ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 -- 172.21.15.63:0/295118978 shutdown_connections 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 --2- 172.21.15.63:0/295118978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b00fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 --2- 172.21.15.63:0/295118978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102140 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 -- 172.21.15.63:0/295118978 >> 172.21.15.63:0/295118978 conn(0x7f18b00f78e0 msgr2=0x7f18b00f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 -- 172.21.15.63:0/295118978 shutdown_connections 2024-04-24T22:52:29.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.230+0000 7f18b7539700 1 -- 172.21.15.63:0/295118978 wait complete. 2024-04-24T22:52:29.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 Processor -- start 2024-04-24T22:52:29.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 -- start start 2024-04-24T22:52:29.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b0103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0104b20 con 0x7f18b0101d70 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b7539700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0103650 con 0x7f18b0102710 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b52d5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b52d5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37278/0 (socket says 172.21.15.63:37278) 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b52d5700 1 -- 172.21.15.63:0/2046823700 learned_addr learned my addr 172.21.15.63:0/2046823700 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.231+0000 7f18b4ad4700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b0103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b4ad4700 1 -- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 msgr2=0x7f18b0102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b4ad4700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b4ad4700 1 -- 172.21.15.63:0/2046823700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac005040 con 0x7f18b0102710 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b52d5700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b4ad4700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b0103110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18b0064cf0 tx=0x7f18a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:29.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a0004e90 con 0x7f18b0102710 2024-04-24T22:52:29.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18a000f040 con 0x7f18b0102710 2024-04-24T22:52:29.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a000ca20 con 0x7f18b0102710 2024-04-24T22:52:29.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.232+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b01038d0 con 0x7f18b0102710 2024-04-24T22:52:29.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.233+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0119f40 con 0x7f18b0102710 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.233+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00fd4c0 con 0x7f18b0102710 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.234+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f18a0007cc0 con 0x7f18b0102710 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.234+0000 7f18a5ffb700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 0x7f189c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.235+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f18a008d420 con 0x7f18b0102710 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.235+0000 7f18b52d5700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 0x7f189c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.235+0000 7f18b52d5700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 0x7f189c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b0d0 tx=0x7f18ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:29.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.240+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f18a0056880 con 0x7f18b0102710 2024-04-24T22:52:29.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.470+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18b0105d20 con 0x7f189c06e140 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: Upgrade: Updating osd.2 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:29.472 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:52:29.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:29.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: Deploying daemon osd.2 on smithi063 2024-04-24T22:52:29.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/871284006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:29.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:29 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/4088795966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:52:29.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.471+0000 7f18a5ffb700 1 -- 172.21.15.63:0/2046823700 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f18b0105d20 con 0x7f189c06e140 2024-04-24T22:52:29.473 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:52:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "8/21 daemons upgraded", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:52:29.474 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 msgr2=0x7f189c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 0x7f189c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b0d0 tx=0x7f18ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 msgr2=0x7f18b0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b0103110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18b0064cf0 tx=0x7f18a000b040 comp rx=0 tx=0).stop 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 shutdown_connections 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f189c06e140 0x7f189c0705f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18b0101d70 0x7f18b0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 --2- 172.21.15.63:0/2046823700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18b0102710 0x7f18b0103110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 >> 172.21.15.63:0/2046823700 conn(0x7f18b00f78e0 msgr2=0x7f18b0066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:29.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 shutdown_connections 2024-04-24T22:52:29.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.473+0000 7f18b7539700 1 -- 172.21.15.63:0/2046823700 wait complete. 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 -- 172.21.15.63:0/4112358544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 msgr2=0x7fc5481020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/4112358544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5481020a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc538004940 tx=0x7fc538009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 -- 172.21.15.63:0/4112358544 shutdown_connections 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/4112358544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 0x7fc5480fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/4112358544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5481020a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 -- 172.21.15.63:0/4112358544 >> 172.21.15.63:0/4112358544 conn(0x7fc548071460 msgr2=0x7fc548071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 -- 172.21.15.63:0/4112358544 shutdown_connections 2024-04-24T22:52:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.580+0000 7fc54f3f5700 1 -- 172.21.15.63:0/4112358544 wait complete. 2024-04-24T22:52:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 Processor -- start 2024-04-24T22:52:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 -- start start 2024-04-24T22:52:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 0x7fc5480fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480fd2b0 con 0x7fc548101cd0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54f3f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480fd3f0 con 0x7fc548102670 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54d191700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54d191700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37306/0 (socket says 172.21.15.63:37306) 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.581+0000 7fc54d191700 1 -- 172.21.15.63:0/2334284230 learned_addr learned my addr 172.21.15.63:0/2334284230 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54c990700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 0x7fc5480fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54d191700 1 -- 172.21.15.63:0/2334284230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 msgr2=0x7fc5480fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54d191700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 0x7fc5480fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54d191700 1 -- 172.21.15.63:0/2334284230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538005040 con 0x7fc548101cd0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54d191700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc53800b8b0 tx=0x7fc538009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc53800e040 con 0x7fc548101cd0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc538004290 con 0x7fc548101cd0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc53801c3f0 con 0x7fc548101cd0 2024-04-24T22:52:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5480fd640 con 0x7fc548101cd0 2024-04-24T22:52:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.582+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54811a2d0 con 0x7fc548101cd0 2024-04-24T22:52:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.583+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548065200 con 0x7fc548101cd0 2024-04-24T22:52:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.584+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5380137b0 con 0x7fc548101cd0 2024-04-24T22:52:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.584+0000 7fc53dffb700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:52:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.584+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc53808ee00 con 0x7fc548101cd0 2024-04-24T22:52:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.584+0000 7fc54c990700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:52:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.585+0000 7fc54c990700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 0x7fc5340705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc544007fd0 tx=0x7fc544009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:52:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.589+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc538058250 con 0x7fc548101cd0 2024-04-24T22:52:29.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:29.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:29.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: Upgrade: Updating osd.2 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: Deploying daemon osd.2 on smithi063 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/871284006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:29.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:29 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/4088795966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:52:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.886+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc548109f40 con 0x7fc548101cd0 2024-04-24T22:52:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.886+0000 7fc53dffb700 1 -- 172.21.15.63:0/2334284230 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc53805b910 con 0x7fc548101cd0 2024-04-24T22:52:29.888 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:52:29.888 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:52:29.888 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 msgr2=0x7fc5340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 0x7fc5340705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc544007fd0 tx=0x7fc544009040 comp rx=0 tx=0).stop 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 msgr2=0x7fc5480fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc53800b8b0 tx=0x7fc538009e30 comp rx=0 tx=0).stop 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 shutdown_connections 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc53406e140 0x7fc5340705f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548101cd0 0x7fc5480fc7e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 --2- 172.21.15.63:0/2334284230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc548102670 0x7fc5480fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:52:29.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.888+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 >> 172.21.15.63:0/2334284230 conn(0x7fc548071460 msgr2=0x7fc5480f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:52:29.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.889+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 shutdown_connections 2024-04-24T22:52:29.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:52:29.889+0000 7fc54f3f5700 1 -- 172.21.15.63:0/2334284230 wait complete. 2024-04-24T22:52:30.429 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:30 smithi063 ceph-mon[111213]: from='client.44181 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:30.429 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:30 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2334284230' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:52:30.429 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:30.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:30 smithi192 ceph-mon[80463]: from='client.44181 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:52:30.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:30 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2334284230' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:52:30.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:31.465 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:31 smithi063 ceph-mon[111213]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:31.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:31 smithi192 ceph-mon[80463]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:31.803 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:31 smithi063 systemd[1]: Stopping Ceph osd.2 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:52:32.286 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:31 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:52:31.936+0000 7efd230d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:52:32.286 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:31 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:52:31.936+0000 7efd230d6700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-24T22:52:32.286 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:31 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[60266]: 2024-04-24T22:52:31.936+0000 7efd230d6700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:52:32.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:32 smithi063 ceph-mon[111213]: osd.2 marked itself down and dead 2024-04-24T22:52:32.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:32 smithi192 ceph-mon[80463]: osd.2 marked itself down and dead 2024-04-24T22:52:33.130 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:32 smithi063 podman[133927]: 2024-04-24 22:52:32.856304577 +0000 UTC m=+1.037765737 container died c27ca20a4c7809c68ff9ebe0ec61e0944daa6440ce87240b1bbbe72ff3b7bddb (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:52:33.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:33 smithi063 podman[133927]: 2024-04-24 22:52:33.319751322 +0000 UTC m=+1.501212484 container cleanup c27ca20a4c7809c68ff9ebe0ec61e0944daa6440ce87240b1bbbe72ff3b7bddb (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c) 2024-04-24T22:52:33.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:33 smithi063 bash[133927]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2 2024-04-24T22:52:33.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:33 smithi063 ceph-mon[111213]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:33.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:33 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:52:33.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:33 smithi063 ceph-mon[111213]: osdmap e60: 6 total, 5 up, 6 in 2024-04-24T22:52:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:33 smithi192 ceph-mon[80463]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:33 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:52:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:33 smithi192 ceph-mon[80463]: osdmap e60: 6 total, 5 up, 6 in 2024-04-24T22:52:33.898 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:33 smithi063 podman[133945]: 2024-04-24 22:52:33.587741997 +0000 UTC m=+0.733550710 container remove c27ca20a4c7809c68ff9ebe0ec61e0944daa6440ce87240b1bbbe72ff3b7bddb (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424) 2024-04-24T22:52:34.200 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:33 smithi063 podman[134039]: 2024-04-24 22:52:33.800714279 +0000 UTC m=+0.018801641 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:52:34.200 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:33 smithi063 podman[134039]: 2024-04-24 22:52:33.905155067 +0000 UTC m=+0.123242415 container create dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T22:52:34.200 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:34 smithi063 podman[134039]: 2024-04-24 22:52:34.116120437 +0000 UTC m=+0.334207787 container init dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T22:52:34.200 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:34 smithi063 podman[134039]: 2024-04-24 22:52:34.119348147 +0000 UTC m=+0.337435500 container start dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:52:34.200 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:34 smithi063 podman[134039]: 2024-04-24 22:52:34.199404526 +0000 UTC m=+0.417491887 container attach dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:52:34.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:34 smithi063 ceph-mon[111213]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T22:52:34.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:34 smithi063 podman[134039]: 2024-04-24 22:52:34.428129317 +0000 UTC m=+0.646216663 container died dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:52:34.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:34 smithi192 ceph-mon[80463]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T22:52:35.302 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134065]: 2024-04-24 22:52:35.044705963 +0000 UTC m=+0.610854517 container remove dfbd7c2806284144718162880c81be550d9cb57e90bcb0bc4eee6328b9bf8e7b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T22:52:35.302 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-24T22:52:35.302 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 systemd[1]: Stopped Ceph osd.2 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:52:35.302 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.2.service: Consumed 48.553s CPU time. 2024-04-24T22:52:35.302 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 systemd[1]: Starting Ceph osd.2 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:52:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:35 smithi063 ceph-mon[111213]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:35.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:35 smithi063 ceph-mon[111213]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T22:52:35.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134171]: 2024-04-24 22:52:35.404734201 +0000 UTC m=+0.018781454 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:52:35.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134171]: 2024-04-24 22:52:35.512774964 +0000 UTC m=+0.126822248 container create 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T22:52:35.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:35 smithi192 ceph-mon[80463]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-24T22:52:35.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:35 smithi192 ceph-mon[80463]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T22:52:36.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134171]: 2024-04-24 22:52:35.765474017 +0000 UTC m=+0.379521256 container init 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T22:52:36.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134171]: 2024-04-24 22:52:35.768678796 +0000 UTC m=+0.382726034 container start 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T22:52:36.053 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:35 smithi063 podman[134171]: 2024-04-24 22:52:35.857109956 +0000 UTC m=+0.471157205 container attach 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T22:52:36.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 20/264 objects degraded (7.576%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:36.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 bash[134171]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate[134186]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T22:52:36.554 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 podman[134171]: 2024-04-24 22:52:36.254817182 +0000 UTC m=+0.868864421 container died 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T22:52:36.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:36 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 20/264 objects degraded (7.576%), 9 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:37.179 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:36 smithi063 podman[134309]: 2024-04-24 22:52:36.844202659 +0000 UTC m=+0.584003314 container remove 3657e833445bd1795b2b18f8de1cd9f3ab08879fd71592916ca2a457e772dded (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T22:52:37.180 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 podman[134374]: 2024-04-24 22:52:36.972332167 +0000 UTC m=+0.019228498 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:52:37.180 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 podman[134374]: 2024-04-24 22:52:37.069946869 +0000 UTC m=+0.116843180 container create 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T22:52:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:37 smithi063 ceph-mon[111213]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 20/264 objects degraded (7.576%) 2024-04-24T22:52:37.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 podman[134374]: 2024-04-24 22:52:37.305515957 +0000 UTC m=+0.352412269 container init 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T22:52:37.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 podman[134374]: 2024-04-24 22:52:37.308999799 +0000 UTC m=+0.355896111 container start 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:52:37.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 ceph-osd[134393]: -- 172.21.15.63:0/4186357140 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56146c0c5680 con 0x56146c08ec00 2024-04-24T22:52:37.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 bash[134374]: 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 2024-04-24T22:52:37.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 systemd[1]: Started Ceph osd.2 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:52:37.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:37 smithi192 ceph-mon[80463]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 20/264 objects degraded (7.576%) 2024-04-24T22:52:38.303 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:37 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T22:52:37.854+0000 7f766f3317c0 -1 Falling back to public interface 2024-04-24T22:52:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:38.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:38.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:38.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:39 smithi063 ceph-mon[111213]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:39 smithi063 ceph-mon[111213]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T22:52:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:39 smithi192 ceph-mon[80463]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:39 smithi192 ceph-mon[80463]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T22:52:41.729 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:41 smithi192 ceph-mon[80463]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:41.729 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:41 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:41 smithi063 ceph-mon[111213]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:41 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:42.303 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:42 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T22:52:42.056+0000 7f766f3317c0 -1 osd.2 59 log_to_monitors true 2024-04-24T22:52:42.803 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 22:52:42 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T22:52:42.495+0000 7f7660789700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:52:42.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:42 smithi063 ceph-mon[111213]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:52:42.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:42 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:42.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:42 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:42.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:42 smithi192 ceph-mon[80463]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T22:52:42.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:42 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:42.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:42 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:43 smithi063 ceph-mon[111213]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:43 smithi063 ceph-mon[111213]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:52:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:43 smithi063 ceph-mon[111213]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T22:52:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:43 smithi063 ceph-mon[111213]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:52:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:43 smithi192 ceph-mon[80463]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:43 smithi192 ceph-mon[80463]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T22:52:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:43 smithi192 ceph-mon[80463]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T22:52:43.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:43 smithi192 ceph-mon[80463]: from='osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-04-24T22:52:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:52:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347] boot 2024-04-24T22:52:44.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: osdmap e63: 6 total, 6 up, 6 in 2024-04-24T22:52:44.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:52:44.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:44.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:44.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:44 smithi063 ceph-mon[111213]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: osd.2 [v2:172.21.15.63:6818/2703892347,v1:172.21.15.63:6819/2703892347] boot 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: osdmap e63: 6 total, 6 up, 6 in 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:44.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:44 smithi192 ceph-mon[80463]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T22:52:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:45 smithi063 ceph-mon[111213]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:45 smithi192 ceph-mon[80463]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:52:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:52:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:46 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 21/264 objects degraded (7.955%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:47.042 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:46 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 21/264 objects degraded (7.955%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T22:52:47.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: pgmap v101: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: pgmap v101: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:52:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:47.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:52:47.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:52:47.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:52:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:48 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:52:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:48 smithi192 ceph-mon[80463]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-04-24T22:52:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:48 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:52:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:48 smithi063 ceph-mon[111213]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-04-24T22:52:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:49 smithi192 ceph-mon[80463]: pgmap v102: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:49 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/264 objects degraded (7.955%), 6 pgs degraded) 2024-04-24T22:52:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:49 smithi063 ceph-mon[111213]: pgmap v102: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:49 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/264 objects degraded (7.955%), 6 pgs degraded) 2024-04-24T22:52:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:51 smithi192 ceph-mon[80463]: pgmap v103: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:51 smithi063 ceph-mon[111213]: pgmap v103: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:53 smithi192 ceph-mon[80463]: pgmap v104: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:53 smithi063 ceph-mon[111213]: pgmap v104: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:55 smithi192 ceph-mon[80463]: pgmap v105: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:55 smithi063 ceph-mon[111213]: pgmap v105: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:52:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:57 smithi063 ceph-mon[111213]: pgmap v106: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:52:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:57 smithi192 ceph-mon[80463]: pgmap v106: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:00.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:52:59 smithi063 ceph-mon[111213]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:00.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 -- 172.21.15.63:0/2246308880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 msgr2=0x7fc4b0101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 --2- 172.21.15.63:0/2246308880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0101a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0004940 tx=0x7fc4a0009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 -- 172.21.15.63:0/2246308880 shutdown_connections 2024-04-24T22:53:00.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 --2- 172.21.15.63:0/2246308880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0101a30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 --2- 172.21.15.63:0/2246308880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 0x7fc4b00fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 -- 172.21.15.63:0/2246308880 >> 172.21.15.63:0/2246308880 conn(0x7fc4b0073350 msgr2=0x7fc4b0073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 -- 172.21.15.63:0/2246308880 shutdown_connections 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.009+0000 7fc4b869f700 1 -- 172.21.15.63:0/2246308880 wait complete. 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 Processor -- start 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 -- start start 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 0x7fc4b0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01145d0 con 0x7fc4b00fbf40 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b869f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b010dbb0 con 0x7fc4b00fb5a0 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46800/0 (socket says 172.21.15.63:46800) 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 -- 172.21.15.63:0/952386933 learned_addr learned my addr 172.21.15.63:0/952386933 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:00.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b643b700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 0x7fc4b0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 -- 172.21.15.63:0/952386933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 msgr2=0x7fc4b0113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 0x7fc4b0113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.010+0000 7fc4b5c3a700 1 -- 172.21.15.63:0/952386933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0005040 con 0x7fc4b00fbf40 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4b5c3a700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000b3b0 tx=0x7fc4a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a000e040 con 0x7fc4b00fbf40 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4a0013420 con 0x7fc4b00fbf40 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a001c630 con 0x7fc4b00fbf40 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b010de30 con 0x7fc4b00fbf40 2024-04-24T22:53:00.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.011+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b010e370 con 0x7fc4b00fbf40 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.012+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00fe670 con 0x7fc4b00fbf40 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.013+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc4a0013590 con 0x7fc4b00fbf40 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.013+0000 7fc4a6ffd700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 0x7fc49c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.013+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc4a008eb00 con 0x7fc4b00fbf40 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.013+0000 7fc4b643b700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 0x7fc49c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.014+0000 7fc4b643b700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 0x7fc49c0705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac004f40 tx=0x7fc4ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.018+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc4a005bbd0 con 0x7fc4b00fbf40 2024-04-24T22:53:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:52:59 smithi192 ceph-mon[80463]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:00.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.250+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b0109f70 con 0x7fc49c06e140 2024-04-24T22:53:00.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.251+0000 7fc4a6ffd700 1 -- 172.21.15.63:0/952386933 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc4b0109f70 con 0x7fc49c06e140 2024-04-24T22:53:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 msgr2=0x7fc49c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 0x7fc49c0705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac004f40 tx=0x7fc4ac00b040 comp rx=0 tx=0).stop 2024-04-24T22:53:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 msgr2=0x7fc4b0113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000b3b0 tx=0x7fc4a0009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 shutdown_connections 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b00fb5a0 0x7fc4b0113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc49c06e140 0x7fc49c0705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 --2- 172.21.15.63:0/952386933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b00fbf40 0x7fc4b0113f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 >> 172.21.15.63:0/952386933 conn(0x7fc4b0073350 msgr2=0x7fc4b0100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 shutdown_connections 2024-04-24T22:53:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.253+0000 7fc4b869f700 1 -- 172.21.15.63:0/952386933 wait complete. 2024-04-24T22:53:00.266 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.359+0000 7f3794e38700 1 -- 172.21.15.63:0/3898888461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790103780 msgr2=0x7f37900fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.359+0000 7f3794e38700 1 --2- 172.21.15.63:0/3898888461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790103780 0x7f37900fcdf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3784004940 tx=0x7f3784009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- 172.21.15.63:0/3898888461 shutdown_connections 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 --2- 172.21.15.63:0/3898888461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790103780 0x7f37900fcdf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 --2- 172.21.15.63:0/3898888461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790102de0 0x7f37901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- 172.21.15.63:0/3898888461 >> 172.21.15.63:0/3898888461 conn(0x7f37900f8990 msgr2=0x7f37900fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- 172.21.15.63:0/3898888461 shutdown_connections 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- 172.21.15.63:0/3898888461 wait complete. 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 Processor -- start 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- start start 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790103780 0x7f3790113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901144e0 con 0x7f3790102de0 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f3794e38700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379010dae0 con 0x7f3790103780 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f378e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f378e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46824/0 (socket says 172.21.15.63:46824) 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.360+0000 7f378e59c700 1 -- 172.21.15.63:0/949880140 learned_addr learned my addr 172.21.15.63:0/949880140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f378dd9b700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790103780 0x7f3790113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f378e59c700 1 -- 172.21.15.63:0/949880140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790103780 msgr2=0x7f3790113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f378e59c700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790103780 0x7f3790113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f378e59c700 1 -- 172.21.15.63:0/949880140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784005040 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f378e59c700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f378000c8c0 tx=0x7f37800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f378000a980 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f378000ce60 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379010ddc0 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37800115e0 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.361+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379010e380 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.362+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37900fe530 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.363+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3780012400 con 0x7f3790102de0 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.363+0000 7f377effd700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 0x7f37780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.363+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f378008dab0 con 0x7f3790102de0 2024-04-24T22:53:00.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.363+0000 7f378dd9b700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 0x7f37780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.364+0000 7f378dd9b700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 0x7f37780705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f379010eca0 tx=0x7f3784009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.368+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f378005aad0 con 0x7f3790102de0 2024-04-24T22:53:00.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.598+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3790109ec0 con 0x7f377806e140 2024-04-24T22:53:00.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.599+0000 7f377effd700 1 -- 172.21.15.63:0/949880140 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3790109ec0 con 0x7f377806e140 2024-04-24T22:53:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 msgr2=0x7f37780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 0x7f37780705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f379010eca0 tx=0x7f3784009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 msgr2=0x7f3790113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f378000c8c0 tx=0x7f37800044c0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 shutdown_connections 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f377806e140 0x7f37780705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3790102de0 0x7f3790113950 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 --2- 172.21.15.63:0/949880140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3790103780 0x7f3790113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 >> 172.21.15.63:0/949880140 conn(0x7f37900f8990 msgr2=0x7f37900f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 shutdown_connections 2024-04-24T22:53:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.601+0000 7f3794e38700 1 -- 172.21.15.63:0/949880140 wait complete. 2024-04-24T22:53:00.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 -- 172.21.15.63:0/725515155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 msgr2=0x7f9c700fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 --2- 172.21.15.63:0/725515155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c700fc1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9c60004940 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 -- 172.21.15.63:0/725515155 shutdown_connections 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 --2- 172.21.15.63:0/725515155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c700fc1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 --2- 172.21.15.63:0/725515155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 0x7f9c70102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 -- 172.21.15.63:0/725515155 >> 172.21.15.63:0/725515155 conn(0x7f9c70071d00 msgr2=0x7f9c70072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 -- 172.21.15.63:0/725515155 shutdown_connections 2024-04-24T22:53:00.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 -- 172.21.15.63:0/725515155 wait complete. 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.707+0000 7f9c782d6700 1 Processor -- start 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c782d6700 1 -- start start 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c782d6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c782d6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 0x7f9c701070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c782d6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c701076c0 con 0x7f9c700fbd40 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c782d6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70107800 con 0x7f9c70101d90 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46848/0 (socket says 172.21.15.63:46848) 2024-04-24T22:53:00.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 -- 172.21.15.63:0/1941406104 learned_addr learned my addr 172.21.15.63:0/1941406104 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c75871700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 0x7f9c701070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 -- 172.21.15.63:0/1941406104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 msgr2=0x7f9c701070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 0x7f9c701070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 -- 172.21.15.63:0/1941406104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60005040 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.708+0000 7f9c76072700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00bbb0 tx=0x7f9c6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.709+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c0092e0 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.709+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c6c00cbd0 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.709+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c70119ea0 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.709+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c007400 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.709+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7011a380 con 0x7f9c700fbd40 2024-04-24T22:53:00.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.710+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c700fd430 con 0x7f9c700fbd40 2024-04-24T22:53:00.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.710+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c6c012070 con 0x7f9c700fbd40 2024-04-24T22:53:00.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.711+0000 7f9c66ffd700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 0x7f9c5c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:00.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.711+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9c6c08d790 con 0x7f9c700fbd40 2024-04-24T22:53:00.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.711+0000 7f9c75871700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 0x7f9c5c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:00.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.711+0000 7f9c75871700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 0x7f9c5c0705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:00.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.715+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9c6c056b60 con 0x7f9c700fbd40 2024-04-24T22:53:00.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.930+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c70108250 con 0x7f9c5c06e140 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.940+0000 7f9c66ffd700 1 -- 172.21.15.63:0/1941406104 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9c70108250 con 0x7f9c5c06e140 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (4m) 18s ago 16m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (2m) 18s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (118s) 114s ago 15m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (4m) 18s ago 15m 81.1M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (10m) 18s ago 10m 15.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (10m) 18s ago 10m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (10m) 114s ago 10m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:53:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (10m) 114s ago 10m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (6m) 18s ago 17m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (5m) 114s ago 15m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 18s ago 17m 47.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (2m) 114s ago 14m 34.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 18s ago 16m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 114s ago 15m 15.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (96s) 18s ago 14m 91.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (60s) 18s ago 13m 91.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (23s) 18s ago 13m 12.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (12m) 114s ago 12m 236M 2581M 17.2.7-902-gc61472c8 0de407c1b3af d6ce52fabc69 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (12m) 114s ago 12m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (12m) 114s ago 12m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:53:00.943 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 18s ago 15m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 msgr2=0x7f9c5c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 0x7f9c5c0705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 msgr2=0x7f9c70106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00bbb0 tx=0x7f9c6c009840 comp rx=0 tx=0).stop 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 shutdown_connections 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c5c06e140 0x7f9c5c0705f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c700fbd40 0x7f9c70106b60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 --2- 172.21.15.63:0/1941406104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c70101d90 0x7f9c701070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 >> 172.21.15.63:0/1941406104 conn(0x7f9c70071d00 msgr2=0x7f9c700fa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 shutdown_connections 2024-04-24T22:53:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:00.942+0000 7f9c782d6700 1 -- 172.21.15.63:0/1941406104 wait complete. 2024-04-24T22:53:01.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 -- 172.21.15.63:0/2536929712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 msgr2=0x7f7e8c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 --2- 172.21.15.63:0/2536929712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c0fc1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7e74004940 tx=0x7f7e74009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:01.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 -- 172.21.15.63:0/2536929712 shutdown_connections 2024-04-24T22:53:01.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 --2- 172.21.15.63:0/2536929712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c0fc1b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 --2- 172.21.15.63:0/2536929712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 0x7f7e8c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 -- 172.21.15.63:0/2536929712 >> 172.21.15.63:0/2536929712 conn(0x7f7e8c071d00 msgr2=0x7f7e8c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 -- 172.21.15.63:0/2536929712 shutdown_connections 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.046+0000 7f7e91464700 1 -- 172.21.15.63:0/2536929712 wait complete. 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 Processor -- start 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 -- start start 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 0x7f7e8c112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c113620 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e91464700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c10ca10 con 0x7f7e8c101d90 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46874/0 (socket says 172.21.15.63:46874) 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 -- 172.21.15.63:0/188619984 learned_addr learned my addr 172.21.15.63:0/188619984 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:01.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8a7fc700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 0x7f7e8c112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 -- 172.21.15.63:0/188619984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 msgr2=0x7f7e8c112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 0x7f7e8c112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 -- 172.21.15.63:0/188619984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74005040 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.047+0000 7f7e8affd700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400cb10 tx=0x7f7e84007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.048+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e84004200 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.048+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8400acc0 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.048+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e84003d90 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.048+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8c10ccf0 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.048+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8c10d2b0 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.049+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8c0fd430 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.049+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7e84003430 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.050+0000 7f7e837fe700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 0x7f7e780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.050+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7e8408e010 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.050+0000 7f7e8a7fc700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 0x7f7e780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.050+0000 7f7e8a7fc700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 0x7f7e780705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400b3b0 tx=0x7f7e74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.055+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7e8405af90 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:01 smithi063 ceph-mon[111213]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:01 smithi063 ceph-mon[111213]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:01 smithi063 ceph-mon[111213]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.351+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e8c108e50 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.351+0000 7f7e837fe700 1 -- 172.21.15.63:0/188619984 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f7e8405ab20 con 0x7f7e8c0fbd40 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:53:01.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-24T22:53:01.354 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 7 2024-04-24T22:53:01.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:53:01.355 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:53:01.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 msgr2=0x7f7e780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 0x7f7e780705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400b3b0 tx=0x7f7e74009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 msgr2=0x7f7e8c112a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400cb10 tx=0x7f7e84007f90 comp rx=0 tx=0).stop 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 shutdown_connections 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7e7806e140 0x7f7e780705f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e8c0fbd40 0x7f7e8c112a90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 --2- 172.21.15.63:0/188619984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e8c101d90 0x7f7e8c112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.353+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 >> 172.21.15.63:0/188619984 conn(0x7f7e8c071d00 msgr2=0x7f7e8c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.354+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 shutdown_connections 2024-04-24T22:53:01.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.354+0000 7f7e91464700 1 -- 172.21.15.63:0/188619984 wait complete. 2024-04-24T22:53:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:01 smithi192 ceph-mon[80463]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:01 smithi192 ceph-mon[80463]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:01 smithi192 ceph-mon[80463]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- 172.21.15.63:0/3014568713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 msgr2=0x7f62540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 --2- 172.21.15.63:0/3014568713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62540fce10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6248004940 tx=0x7f6248009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- 172.21.15.63:0/3014568713 shutdown_connections 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 --2- 172.21.15.63:0/3014568713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62540fce10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 --2- 172.21.15.63:0/3014568713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f62541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- 172.21.15.63:0/3014568713 >> 172.21.15.63:0/3014568713 conn(0x7f62540f89b0 msgr2=0x7f62540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- 172.21.15.63:0/3014568713 shutdown_connections 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- 172.21.15.63:0/3014568713 wait complete. 2024-04-24T22:53:01.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 Processor -- start 2024-04-24T22:53:01.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.458+0000 7f625ad87700 1 -- start start 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f625ad87700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f6254113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f625ad87700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f625ad87700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254114740 con 0x7f62541037a0 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f625ad87700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625410db30 con 0x7f6254102e00 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46894/0 (socket says 172.21.15.63:46894) 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 -- 172.21.15.63:0/1236406197 learned_addr learned my addr 172.21.15.63:0/1236406197 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6258b23700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f6254113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 -- 172.21.15.63:0/1236406197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 msgr2=0x7f6254113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f6254113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 -- 172.21.15.63:0/1236406197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248005040 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6258b23700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f6254113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.459+0000 7f6253fff700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6248009100 tx=0x7f6248009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.460+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624800e040 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.460+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6248013420 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.460+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624801c610 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.460+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625410ddb0 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.460+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625410e370 con 0x7f62541037a0 2024-04-24T22:53:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.461+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62541052d0 con 0x7f62541037a0 2024-04-24T22:53:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.461+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6248013590 con 0x7f62541037a0 2024-04-24T22:53:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.462+0000 7f62517fa700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 0x7f623c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.462+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f624808ed70 con 0x7f62541037a0 2024-04-24T22:53:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.462+0000 7f6258b23700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 0x7f623c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.462+0000 7f6258b23700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 0x7f623c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f624c004f50 tx=0x7f624c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.467+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f624805bcf0 con 0x7f62541037a0 2024-04-24T22:53:01.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.718+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f625410f200 con 0x7f62541037a0 2024-04-24T22:53:01.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.718+0000 7f62517fa700 1 -- 172.21.15.63:0/1236406197 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f624805b880 con 0x7f62541037a0 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:53:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:53:01.721 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:01.722 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:01.723 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:01.723 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 msgr2=0x7f623c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 0x7f623c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f624c004f50 tx=0x7f624c00b040 comp rx=0 tx=0).stop 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 msgr2=0x7f62541140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6248009100 tx=0x7f6248009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 shutdown_connections 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6254102e00 0x7f6254113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f623c06e140 0x7f623c0705f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 --2- 172.21.15.63:0/1236406197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62541037a0 0x7f62541140f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 >> 172.21.15.63:0/1236406197 conn(0x7f62540f89b0 msgr2=0x7f62541021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 shutdown_connections 2024-04-24T22:53:01.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.720+0000 7f625ad87700 1 -- 172.21.15.63:0/1236406197 wait complete. 2024-04-24T22:53:01.725 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 -- 172.21.15.63:0/4287778911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 msgr2=0x7f51200fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 --2- 172.21.15.63:0/4287778911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fe1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5114004940 tx=0x7f5114009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 -- 172.21.15.63:0/4287778911 shutdown_connections 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 --2- 172.21.15.63:0/4287778911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 0x7f5120106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 --2- 172.21.15.63:0/4287778911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fe1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 -- 172.21.15.63:0/4287778911 >> 172.21.15.63:0/4287778911 conn(0x7f51200f7870 msgr2=0x7f51200f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 -- 172.21.15.63:0/4287778911 shutdown_connections 2024-04-24T22:53:01.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.827+0000 7f5126d11700 1 -- 172.21.15.63:0/4287778911 wait complete. 2024-04-24T22:53:01.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 Processor -- start 2024-04-24T22:53:01.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 -- start start 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 0x7f51200fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200fdc00 con 0x7f51200fddf0 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5126d11700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200fc730 con 0x7f51200fe790 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46908/0 (socket says 172.21.15.63:46908) 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 -- 172.21.15.63:0/982556555 learned_addr learned my addr 172.21.15.63:0/982556555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f511ffff700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 0x7f51200fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 -- 172.21.15.63:0/982556555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 msgr2=0x7f51200fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 0x7f51200fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.828+0000 7f5124aad700 1 -- 172.21.15.63:0/982556555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114005040 con 0x7f51200fddf0 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f5124aad700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5114021070 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51200fc9b0 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5114013c00 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511400e070 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.829+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120119fb0 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.830+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5120066ab0 con 0x7f51200fddf0 2024-04-24T22:53:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.830+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f511401c9d0 con 0x7f51200fddf0 2024-04-24T22:53:01.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.831+0000 7f511d7fa700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 0x7f51100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:01.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.831+0000 7f511ffff700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 0x7f51100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:01.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.831+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f511408eea0 con 0x7f51200fddf0 2024-04-24T22:53:01.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.832+0000 7f511ffff700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 0x7f51100705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51200fd3e0 tx=0x7f510c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:01.836+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f51140582f0 con 0x7f51200fddf0 2024-04-24T22:53:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.068+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51200fd260 con 0x7f511006e140 2024-04-24T22:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.069+0000 7f511d7fa700 1 -- 172.21.15.63:0/982556555 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f510c00f040 con 0x7f511006e140 2024-04-24T22:53:02.070 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:53:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:53:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:53:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "9/21 daemons upgraded", 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:53:02.071 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 msgr2=0x7f51100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 0x7f51100705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51200fd3e0 tx=0x7f510c00b040 comp rx=0 tx=0).stop 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 msgr2=0x7f51200fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 shutdown_connections 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f511006e140 0x7f51100705f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200fddf0 0x7f51200fbcb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 --2- 172.21.15.63:0/982556555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51200fe790 0x7f51200fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 >> 172.21.15.63:0/982556555 conn(0x7f51200f7870 msgr2=0x7f51200f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 shutdown_connections 2024-04-24T22:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.071+0000 7f5126d11700 1 -- 172.21.15.63:0/982556555 wait complete. 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 -- 172.21.15.63:0/490826600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600fbb60 msgr2=0x7f70600fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 --2- 172.21.15.63:0/490826600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600fbb60 0x7f70600fbfd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f704c004940 tx=0x7f704c009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 -- 172.21.15.63:0/490826600 shutdown_connections 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 --2- 172.21.15.63:0/490826600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600fbb60 0x7f70600fbfd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 --2- 172.21.15.63:0/490826600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7060100a90 0x7f7060100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 -- 172.21.15.63:0/490826600 >> 172.21.15.63:0/490826600 conn(0x7f7060071b40 msgr2=0x7f7060071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 -- 172.21.15.63:0/490826600 shutdown_connections 2024-04-24T22:53:02.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.177+0000 7f70659a2700 1 -- 172.21.15.63:0/490826600 wait complete. 2024-04-24T22:53:02.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 Processor -- start 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 -- start start 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 0x7f706010eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706010f600 con 0x7f7060100a90 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70659a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706010f740 con 0x7f70600fbb60 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46940/0 (socket says 172.21.15.63:46940) 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 -- 172.21.15.63:0/3605280328 learned_addr learned my addr 172.21.15.63:0/3605280328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70649a0700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 0x7f706010eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 -- 172.21.15.63:0/3605280328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 msgr2=0x7f706010eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 0x7f706010eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f705ffff700 1 -- 172.21.15.63:0/3605280328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c005040 con 0x7f7060100a90 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.178+0000 7f70649a0700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 0x7f706010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:02.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f705ffff700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f704c009150 tx=0x7f704c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:02.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704c00e040 con 0x7f7060100a90 2024-04-24T22:53:02.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7060117a00 con 0x7f7060100a90 2024-04-24T22:53:02.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f704c013420 con 0x7f7060100a90 2024-04-24T22:53:02.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7060117f90 con 0x7f7060100a90 2024-04-24T22:53:02.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.179+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704c01c630 con 0x7f7060100a90 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.180+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70600fd200 con 0x7f7060100a90 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.180+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f704c013590 con 0x7f7060100a90 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.181+0000 7f705d7fa700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 0x7f7050070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.181+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f704c08ed20 con 0x7f7060100a90 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.181+0000 7f70649a0700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 0x7f7050070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:02.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.181+0000 7f70649a0700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 0x7f7050070600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7054009b10 tx=0x7f7054008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:02.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.186+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f704c058170 con 0x7f7060100a90 2024-04-24T22:53:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:02 smithi192 ceph-mon[80463]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:02 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/188619984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:02 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1236406197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:53:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.477+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f706005f5c0 con 0x7f7060100a90 2024-04-24T22:53:02.480 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:02 smithi063 ceph-mon[111213]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:02.480 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:02 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/188619984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:02.480 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:02 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1236406197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:53:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.478+0000 7f705d7fa700 1 -- 172.21.15.63:0/3605280328 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f706005f5c0 con 0x7f7060100a90 2024-04-24T22:53:02.480 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:53:02.480 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:53:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:53:02.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.480+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 msgr2=0x7f7050070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.480+0000 7f70659a2700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 0x7f7050070600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7054009b10 tx=0x7f7054008040 comp rx=0 tx=0).stop 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.480+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 msgr2=0x7f706010f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.480+0000 7f70659a2700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f704c009150 tx=0x7f704c009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 shutdown_connections 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70600fbb60 0x7f706010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f705006e150 0x7f7050070600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 --2- 172.21.15.63:0/3605280328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7060100a90 0x7f706010f030 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 >> 172.21.15.63:0/3605280328 conn(0x7f7060071b40 msgr2=0x7f70600f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 shutdown_connections 2024-04-24T22:53:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:02.481+0000 7f70659a2700 1 -- 172.21.15.63:0/3605280328 wait complete. 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: Upgrade: osd.3 is safe to restart 2024-04-24T22:53:03.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:03 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3605280328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:53:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:03.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:53:03.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T22:53:03.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: Upgrade: osd.3 is safe to restart 2024-04-24T22:53:03.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:03 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3605280328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: Upgrade: Updating osd.3 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: Deploying daemon osd.3 on smithi192 2024-04-24T22:53:05.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:04 smithi192 ceph-mon[80463]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: Upgrade: Updating osd.3 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: Deploying daemon osd.3 on smithi192 2024-04-24T22:53:05.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:04 smithi063 ceph-mon[111213]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:07.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:07 smithi192 ceph-mon[80463]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:07.427 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:07 smithi192 systemd[1]: Stopping Ceph osd.3 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:53:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:07 smithi063 ceph-mon[111213]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:07.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:07 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[39433]: 2024-04-24T22:53:07.538+0000 7fb97a501700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:53:07.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:07 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[39433]: 2024-04-24T22:53:07.538+0000 7fb97a501700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-24T22:53:07.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:07 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[39433]: 2024-04-24T22:53:07.538+0000 7fb97a501700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:53:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:08 smithi192 ceph-mon[80463]: osd.3 marked itself down and dead 2024-04-24T22:53:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:08 smithi063 ceph-mon[111213]: osd.3 marked itself down and dead 2024-04-24T22:53:08.813 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:08 smithi192 podman[89666]: 2024-04-24 22:53:08.562200186 +0000 UTC m=+1.145006462 container died d6ce52fabc697d8062a1229f05cdb28efb8519d2cc09e30a9984810707e3e76e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=) 2024-04-24T22:53:09.145 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:08 smithi192 podman[89666]: 2024-04-24 22:53:08.905911556 +0000 UTC m=+1.488717840 container cleanup d6ce52fabc697d8062a1229f05cdb28efb8519d2cc09e30a9984810707e3e76e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=quincy-c61472c) 2024-04-24T22:53:09.145 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:08 smithi192 bash[89666]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3 2024-04-24T22:53:09.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:09 smithi192 ceph-mon[80463]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:09.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:09 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:53:09.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:09 smithi192 ceph-mon[80463]: osdmap e65: 6 total, 5 up, 6 in 2024-04-24T22:53:09.428 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:09 smithi192 podman[89685]: 2024-04-24 22:53:09.22454493 +0000 UTC m=+0.665056577 container remove d6ce52fabc697d8062a1229f05cdb28efb8519d2cc09e30a9984810707e3e76e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T22:53:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:09 smithi063 ceph-mon[111213]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:09 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:53:09.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:09 smithi063 ceph-mon[111213]: osdmap e65: 6 total, 5 up, 6 in 2024-04-24T22:53:09.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:09 smithi192 podman[89776]: 2024-04-24 22:53:09.558516839 +0000 UTC m=+0.101451039 container create 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:53:09.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:09 smithi192 podman[89776]: 2024-04-24 22:53:09.47479262 +0000 UTC m=+0.017726827 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:09.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:09 smithi192 podman[89776]: 2024-04-24 22:53:09.802855873 +0000 UTC m=+0.345790062 container init 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T22:53:09.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:09 smithi192 podman[89776]: 2024-04-24 22:53:09.805985837 +0000 UTC m=+0.348920033 container start 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T22:53:10.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:10 smithi192 ceph-mon[80463]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T22:53:10.427 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 podman[89776]: 2024-04-24 22:53:10.094728291 +0000 UTC m=+0.637662496 container attach 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T22:53:10.427 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 podman[89776]: 2024-04-24 22:53:10.098830597 +0000 UTC m=+0.641764798 container died 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T22:53:10.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:10 smithi063 ceph-mon[111213]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T22:53:11.141 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 podman[89802]: 2024-04-24 22:53:10.698435622 +0000 UTC m=+0.589590646 container remove 4a5da5f8b31abba09a0d5f8212e7f9f6ef930e1fc6688608a97f82153e86e8f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T22:53:11.141 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-24T22:53:11.141 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 systemd[1]: Stopped Ceph osd.3 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:53:11.141 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 2.064s CPU time. 2024-04-24T22:53:11.141 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:10 smithi192 systemd[1]: Starting Ceph osd.3 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:53:11.404 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-mon[80463]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:11.404 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.04284887 +0000 UTC m=+0.017822370 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:11.404 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.161092126 +0000 UTC m=+0.136065627 container create 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T22:53:11.405 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.402540181 +0000 UTC m=+0.377513680 container init 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:53:11.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:11 smithi063 ceph-mon[111213]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:11.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.406117248 +0000 UTC m=+0.381090756 container start 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T22:53:11.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.502623544 +0000 UTC m=+0.477597057 container attach 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T22:53:12.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 bash[89903]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate[89918]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T22:53:12.178 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:11 smithi192 podman[89903]: 2024-04-24 22:53:11.876012982 +0000 UTC m=+0.850986485 container died 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T22:53:12.515 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:12 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 40/264 objects degraded (15.152%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:12 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 40/264 objects degraded (15.152%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:12.856 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:12 smithi192 podman[90053]: 2024-04-24 22:53:12.515145091 +0000 UTC m=+0.634097682 container remove 33abc40aa25030456c7da999018692c1c6ce6e8fc3e0c4210be24b4f68db7da0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T22:53:12.856 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:12 smithi192 podman[90118]: 2024-04-24 22:53:12.632988374 +0000 UTC m=+0.018076228 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:12.856 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:12 smithi192 podman[90118]: 2024-04-24 22:53:12.741100284 +0000 UTC m=+0.126188136 container create 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T22:53:13.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:12 smithi192 podman[90118]: 2024-04-24 22:53:12.976532203 +0000 UTC m=+0.361620044 container init 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:53:13.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:12 smithi192 podman[90118]: 2024-04-24 22:53:12.97969058 +0000 UTC m=+0.364778422 container start 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T22:53:13.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-osd[90137]: -- 172.21.15.192:0/4054903852 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e359c31680 con 0x55e359bfb000 2024-04-24T22:53:13.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:13 smithi192 bash[90118]: 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada 2024-04-24T22:53:13.183 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:13 smithi192 systemd[1]: Started Ceph osd.3 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:53:13.527 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-mon[80463]: pgmap v116: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 40/264 objects degraded (15.152%) 2024-04-24T22:53:13.527 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:13.527 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:13.528 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:13 smithi063 ceph-mon[111213]: pgmap v116: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 40/264 objects degraded (15.152%) 2024-04-24T22:53:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:13.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:13 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:13.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:13 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T22:53:13.525+0000 7ff72468e7c0 -1 Falling back to public interface 2024-04-24T22:53:15.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:15 smithi063 ceph-mon[111213]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:15.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:15.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:15 smithi192 ceph-mon[80463]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:17.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:17 smithi063 ceph-mon[111213]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:17.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:17 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:17.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:17 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:17.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:17 smithi192 ceph-mon[80463]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:17.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:17 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:17.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:17 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-mon[80463]: from='osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:53:18.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-mon[80463]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:53:18.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T22:53:18.422+0000 7ff72468e7c0 -1 osd.3 64 log_to_monitors true 2024-04-24T22:53:18.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 22:53:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T22:53:18.462+0000 7ff715ae6700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:53:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:18 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:18 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:18 smithi063 ceph-mon[111213]: from='osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:53:18.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:18 smithi063 ceph-mon[111213]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T22:53:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:19 smithi192 ceph-mon[80463]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:19 smithi192 ceph-mon[80463]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:53:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:19 smithi192 ceph-mon[80463]: from='osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:19.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:19 smithi192 ceph-mon[80463]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T22:53:19.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:19 smithi192 ceph-mon[80463]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:19 smithi063 ceph-mon[111213]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:19 smithi063 ceph-mon[111213]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T22:53:19.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:19 smithi063 ceph-mon[111213]: from='osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:19.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:19 smithi063 ceph-mon[111213]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T22:53:19.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:19 smithi063 ceph-mon[111213]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:20.753 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:20 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:53:20.753 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:20 smithi192 ceph-mon[80463]: osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226] boot 2024-04-24T22:53:20.753 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:20 smithi192 ceph-mon[80463]: osdmap e68: 6 total, 6 up, 6 in 2024-04-24T22:53:20.753 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:20 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:53:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:20 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:53:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:20 smithi063 ceph-mon[111213]: osd.3 [v2:172.21.15.192:6800/534042226,v1:172.21.15.192:6801/534042226] boot 2024-04-24T22:53:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:20 smithi063 ceph-mon[111213]: osdmap e68: 6 total, 6 up, 6 in 2024-04-24T22:53:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:20 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:21 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:21.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:21 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:22.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:22 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:22.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:22 smithi063 ceph-mon[111213]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-24T22:53:22.906 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:22 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:22.907 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:22 smithi192 ceph-mon[80463]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-24T22:53:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:23 smithi063 ceph-mon[111213]: pgmap v124: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/264 objects degraded (17.803%) 2024-04-24T22:53:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:23 smithi192 ceph-mon[80463]: pgmap v124: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/264 objects degraded (17.803%) 2024-04-24T22:53:24.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:24 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 47/264 objects degraded (17.803%), 15 pgs degraded) 2024-04-24T22:53:24.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:24 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 47/264 objects degraded (17.803%), 15 pgs degraded) 2024-04-24T22:53:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:25 smithi063 ceph-mon[111213]: pgmap v125: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:25 smithi192 ceph-mon[80463]: pgmap v125: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:27 smithi063 ceph-mon[111213]: pgmap v126: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:27 smithi192 ceph-mon[80463]: pgmap v126: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:29 smithi063 ceph-mon[111213]: pgmap v127: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:29 smithi192 ceph-mon[80463]: pgmap v127: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:31 smithi192 ceph-mon[80463]: pgmap v128: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:31.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:31 smithi063 ceph-mon[111213]: pgmap v128: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:31.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:31.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:31.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 -- 172.21.15.63:0/4256480713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 msgr2=0x7fb9841091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 --2- 172.21.15.63:0/4256480713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb9841091a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb980004940 tx=0x7fb980009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 -- 172.21.15.63:0/4256480713 shutdown_connections 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 --2- 172.21.15.63:0/4256480713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb9841091a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 --2- 172.21.15.63:0/4256480713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 0x7fb9840fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 -- 172.21.15.63:0/4256480713 >> 172.21.15.63:0/4256480713 conn(0x7fb9840f7840 msgr2=0x7fb9840f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 -- 172.21.15.63:0/4256480713 shutdown_connections 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.602+0000 7fb98ba98700 1 -- 172.21.15.63:0/4256480713 wait complete. 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 Processor -- start 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 -- start start 2024-04-24T22:53:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 0x7fb984115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841168c0 con 0x7fb9840fc440 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb98ba98700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98410fc90 con 0x7fb9840fbb30 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb989033700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb989033700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39712/0 (socket says 172.21.15.63:39712) 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb989033700 1 -- 172.21.15.63:0/540416873 learned_addr learned my addr 172.21.15.63:0/540416873 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.603+0000 7fb989834700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 0x7fb984115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb989033700 1 -- 172.21.15.63:0/540416873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 msgr2=0x7fb984115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb989033700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 0x7fb984115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb989033700 1 -- 172.21.15.63:0/540416873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb980005040 con 0x7fb9840fc440 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb989033700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb98000b3b0 tx=0x7fb980009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98000e040 con 0x7fb9840fc440 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb980013420 con 0x7fb9840fc440 2024-04-24T22:53:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98410ff10 con 0x7fb9840fc440 2024-04-24T22:53:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98001c630 con 0x7fb9840fc440 2024-04-24T22:53:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.604+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9841104d0 con 0x7fb9840fc440 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.605+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984066750 con 0x7fb9840fc440 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.606+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb980013590 con 0x7fb9840fc440 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.606+0000 7fb97a7fc700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 0x7fb9700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.606+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb98008ebe0 con 0x7fb9840fc440 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.606+0000 7fb989834700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 0x7fb9700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.607+0000 7fb989834700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 0x7fb9700705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb974004f40 tx=0x7fb97400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.611+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb98005bc00 con 0x7fb9840fc440 2024-04-24T22:53:32.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.839+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb98410c100 con 0x7fb97006e140 2024-04-24T22:53:32.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.840+0000 7fb97a7fc700 1 -- 172.21.15.63:0/540416873 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb98410c100 con 0x7fb97006e140 2024-04-24T22:53:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 msgr2=0x7fb9700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 0x7fb9700705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb974004f40 tx=0x7fb97400b040 comp rx=0 tx=0).stop 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 msgr2=0x7fb984116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb98000b3b0 tx=0x7fb980009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 shutdown_connections 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9840fbb30 0x7fb984115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb97006e140 0x7fb9700705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 --2- 172.21.15.63:0/540416873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9840fc440 0x7fb984116270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 >> 172.21.15.63:0/540416873 conn(0x7fb9840f7840 msgr2=0x7fb9840f7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 shutdown_connections 2024-04-24T22:53:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.842+0000 7fb98ba98700 1 -- 172.21.15.63:0/540416873 wait complete. 2024-04-24T22:53:32.854 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 -- 172.21.15.63:0/4288449381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 msgr2=0x7f5324103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 --2- 172.21.15.63:0/4288449381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324103190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5314004940 tx=0x7f5314009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 -- 172.21.15.63:0/4288449381 shutdown_connections 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 --2- 172.21.15.63:0/4288449381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 0x7f53240fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 --2- 172.21.15.63:0/4288449381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324103190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 -- 172.21.15.63:0/4288449381 >> 172.21.15.63:0/4288449381 conn(0x7f53240f8930 msgr2=0x7f53240fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 -- 172.21.15.63:0/4288449381 shutdown_connections 2024-04-24T22:53:32.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.947+0000 7f532bf64700 1 -- 172.21.15.63:0/4288449381 wait complete. 2024-04-24T22:53:32.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 Processor -- start 2024-04-24T22:53:32.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 -- start start 2024-04-24T22:53:32.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 0x7f5324113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53241144f0 con 0x7f5324102dc0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f532bf64700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532410daf0 con 0x7f5324103760 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f5329d00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f5329d00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39734/0 (socket says 172.21.15.63:39734) 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f5329d00700 1 -- 172.21.15.63:0/4157393612 learned_addr learned my addr 172.21.15.63:0/4157393612 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.948+0000 7f53294ff700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 0x7f5324113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f5329d00700 1 -- 172.21.15.63:0/4157393612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 msgr2=0x7f5324113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f5329d00700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 0x7f5324113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f5329d00700 1 -- 172.21.15.63:0/4157393612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314005040 con 0x7f5324102dc0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f5329d00700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5314009150 tx=0x7f5314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531400e040 con 0x7f5324102dc0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5314013420 con 0x7f5324102dc0 2024-04-24T22:53:32.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531401c610 con 0x7f5324102dc0 2024-04-24T22:53:32.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532410dd70 con 0x7f5324102dc0 2024-04-24T22:53:32.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.949+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532410e330 con 0x7f5324102dc0 2024-04-24T22:53:32.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.950+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53240fe510 con 0x7f5324102dc0 2024-04-24T22:53:32.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.951+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f531400baf0 con 0x7f5324102dc0 2024-04-24T22:53:32.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.951+0000 7f531a7fc700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 0x7f53100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:32.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.951+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f531408ebb0 con 0x7f5324102dc0 2024-04-24T22:53:32.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.951+0000 7f53294ff700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 0x7f53100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:32.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.952+0000 7f53294ff700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 0x7f53100705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f532410ecb0 tx=0x7f532000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:32.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:32.956+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5314095080 con 0x7f5324102dc0 2024-04-24T22:53:33.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.184+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5324109f30 con 0x7f531006e140 2024-04-24T22:53:33.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.185+0000 7f531a7fc700 1 -- 172.21.15.63:0/4157393612 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5324109f30 con 0x7f531006e140 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 msgr2=0x7f53100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 0x7f53100705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f532410ecb0 tx=0x7f532000b040 comp rx=0 tx=0).stop 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 msgr2=0x7f5324113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5314009150 tx=0x7f5314009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 shutdown_connections 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f531006e140 0x7f53100705f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5324102dc0 0x7f5324113960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 --2- 172.21.15.63:0/4157393612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5324103760 0x7f5324113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.187+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 >> 172.21.15.63:0/4157393612 conn(0x7f53240f8930 msgr2=0x7f53240f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:33.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.188+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 shutdown_connections 2024-04-24T22:53:33.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.188+0000 7f532bf64700 1 -- 172.21.15.63:0/4157393612 wait complete. 2024-04-24T22:53:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 -- 172.21.15.63:0/1633662990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950065640 msgr2=0x7f5950065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 --2- 172.21.15.63:0/1633662990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950065640 0x7f5950065ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f593c004940 tx=0x7f593c009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 -- 172.21.15.63:0/1633662990 shutdown_connections 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 --2- 172.21.15.63:0/1633662990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950065640 0x7f5950065ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 --2- 172.21.15.63:0/1633662990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950102e50 0x7f5950103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 -- 172.21.15.63:0/1633662990 >> 172.21.15.63:0/1633662990 conn(0x7f5950071d00 msgr2=0x7f5950072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 -- 172.21.15.63:0/1633662990 shutdown_connections 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.292+0000 7f5956e0a700 1 -- 172.21.15.63:0/1633662990 wait complete. 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 Processor -- start 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 -- start start 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950065640 0x7f59500fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500fc880 con 0x7f5950102e50 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.293+0000 7f5956e0a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500fc9c0 con 0x7f5950065640 2024-04-24T22:53:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39740/0 (socket says 172.21.15.63:39740) 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 -- 172.21.15.63:0/2102965799 learned_addr learned my addr 172.21.15.63:0/2102965799 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f5954ba6700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950065640 0x7f59500fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 -- 172.21.15.63:0/2102965799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950065640 msgr2=0x7f59500fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950065640 0x7f59500fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 -- 172.21.15.63:0/2102965799 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c005040 con 0x7f5950102e50 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594ffff700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f593c00b8b0 tx=0x7f593c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593c00e040 con 0x7f5950102e50 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f593c004290 con 0x7f5950102e50 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500fcc10 con 0x7f5950102e50 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593c01c3f0 con 0x7f5950102e50 2024-04-24T22:53:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.294+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595011a2a0 con 0x7f5950102e50 2024-04-24T22:53:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.295+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950100100 con 0x7f5950102e50 2024-04-24T22:53:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.296+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f593c0137b0 con 0x7f5950102e50 2024-04-24T22:53:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.296+0000 7f594d7fa700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 0x7f59400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.296+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f593c08ede0 con 0x7f5950102e50 2024-04-24T22:53:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.296+0000 7f5954ba6700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 0x7f59400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.297+0000 7f5954ba6700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 0x7f59400705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5944007fd0 tx=0x7f5944009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.301+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f593c058230 con 0x7f5950102e50 2024-04-24T22:53:33.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.521+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5950109f00 con 0x7f594006e140 2024-04-24T22:53:33.523 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:33 smithi063 ceph-mon[111213]: pgmap v129: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:33.523 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:33 smithi063 ceph-mon[111213]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:33.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.532+0000 7f594d7fa700 1 -- 172.21.15.63:0/2102965799 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5950109f00 con 0x7f594006e140 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 51s ago 16m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (2m) 51s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (2m) 17s ago 15m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 51s ago 16m 81.1M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (11m) 51s ago 10m 15.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (11m) 51s ago 11m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (11m) 17s ago 11m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (10m) 17s ago 10m 16.3M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (6m) 51s ago 18m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:53:33.534 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (6m) 17s ago 15m 450M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 51s ago 18m 47.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (3m) 17s ago 15m 38.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 51s ago 16m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 17s ago 15m 16.7M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 51s ago 14m 91.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (92s) 51s ago 14m 91.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (56s) 51s ago 13m 12.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (20s) 17s ago 13m 15.1M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (12m) 17s ago 12m 212M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 18dcb9bd6573 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (12m) 17s ago 12m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:53:33.535 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 51s ago 15m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 msgr2=0x7f59400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 0x7f59400705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5944007fd0 tx=0x7f5944009040 comp rx=0 tx=0).stop 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 msgr2=0x7f59500fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f593c00b8b0 tx=0x7f593c009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 shutdown_connections 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5950065640 0x7f59500fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f594006e140 0x7f59400705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 --2- 172.21.15.63:0/2102965799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5950102e50 0x7f59500fc2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 >> 172.21.15.63:0/2102965799 conn(0x7f5950071d00 msgr2=0x7f59500fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:33.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 shutdown_connections 2024-04-24T22:53:33.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.534+0000 7f5956e0a700 1 -- 172.21.15.63:0/2102965799 wait complete. 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- 172.21.15.63:0/1100884547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fbf40 msgr2=0x7f32f8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 --2- 172.21.15.63:0/1100884547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fbf40 0x7f32f8101a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f32e8004940 tx=0x7f32e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- 172.21.15.63:0/1100884547 shutdown_connections 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 --2- 172.21.15.63:0/1100884547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fbf40 0x7f32f8101a30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 --2- 172.21.15.63:0/1100884547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fb5a0 0x7f32f80fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- 172.21.15.63:0/1100884547 >> 172.21.15.63:0/1100884547 conn(0x7f32f8073350 msgr2=0x7f32f8073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:33.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- 172.21.15.63:0/1100884547 shutdown_connections 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- 172.21.15.63:0/1100884547 wait complete. 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 Processor -- start 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- start start 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fbf40 0x7f32f8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8114530 con 0x7f32f80fb5a0 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.638+0000 7f32ffae1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f810db50 con 0x7f32f80fbf40 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39762/0 (socket says 172.21.15.63:39762) 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 -- 172.21.15.63:0/648188452 learned_addr learned my addr 172.21.15.63:0/648188452 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd07c700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fbf40 0x7f32f8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 -- 172.21.15.63:0/648188452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fbf40 msgr2=0x7f32f8113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fbf40 0x7f32f8113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 -- 172.21.15.63:0/648188452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8005040 con 0x7f32f80fb5a0 2024-04-24T22:53:33.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32fd87d700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c8c0 tx=0x7f32f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:33.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f400a980 con 0x7f32f80fb5a0 2024-04-24T22:53:33.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f400ce60 con 0x7f32f80fb5a0 2024-04-24T22:53:33.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f40115e0 con 0x7f32f80fb5a0 2024-04-24T22:53:33.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f810de30 con 0x7f32f80fb5a0 2024-04-24T22:53:33.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.639+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f810e370 con 0x7f32f80fb5a0 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.640+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f80fe670 con 0x7f32f80fb5a0 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.641+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f32f4012400 con 0x7f32f80fb5a0 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.641+0000 7f32ee7fc700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 0x7f32e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.641+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f32f408dab0 con 0x7f32f80fb5a0 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.641+0000 7f32fd07c700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 0x7f32e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:33.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.642+0000 7f32fd07c700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 0x7f32e40705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f32f810f240 tx=0x7f32e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:33.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.646+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f32f4056fa0 con 0x7f32f80fb5a0 2024-04-24T22:53:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:33 smithi192 ceph-mon[80463]: pgmap v129: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:33.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:33 smithi192 ceph-mon[80463]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.958+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32f8109f90 con 0x7f32f80fb5a0 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.958+0000 7f32ee7fc700 1 -- 172.21.15.63:0/648188452 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f32f40126b0 con 0x7f32f80fb5a0 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:53:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6, 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 8 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:53:33.961 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:53:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 msgr2=0x7f32e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 0x7f32e40705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f32f810f240 tx=0x7f32e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 msgr2=0x7f32f81139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c8c0 tx=0x7f32f40044c0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 shutdown_connections 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f32e406e140 0x7f32e40705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f80fb5a0 0x7f32f81139a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 --2- 172.21.15.63:0/648188452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f80fbf40 0x7f32f8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 >> 172.21.15.63:0/648188452 conn(0x7f32f8073350 msgr2=0x7f32f8100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 shutdown_connections 2024-04-24T22:53:33.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:33.961+0000 7f32ffae1700 1 -- 172.21.15.63:0/648188452 wait complete. 2024-04-24T22:53:34.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 -- 172.21.15.63:0/2934965836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28102de0 msgr2=0x7fdb281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 --2- 172.21.15.63:0/2934965836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28102de0 0x7fdb281031b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ce0 tx=0x7fdb18004d10 comp rx=0 tx=0).stop 2024-04-24T22:53:34.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 -- 172.21.15.63:0/2934965836 shutdown_connections 2024-04-24T22:53:34.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 --2- 172.21.15.63:0/2934965836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28103780 0x7fdb280fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 --2- 172.21.15.63:0/2934965836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28102de0 0x7fdb281031b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 -- 172.21.15.63:0/2934965836 >> 172.21.15.63:0/2934965836 conn(0x7fdb280f8990 msgr2=0x7fdb280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:34.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 -- 172.21.15.63:0/2934965836 shutdown_connections 2024-04-24T22:53:34.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.065+0000 7fdb3026b700 1 -- 172.21.15.63:0/2934965836 wait complete. 2024-04-24T22:53:34.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 Processor -- start 2024-04-24T22:53:34.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 -- start start 2024-04-24T22:53:34.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28102de0 0x7fdb28113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28114660 con 0x7fdb28103780 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb3026b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2810dc90 con 0x7fdb28102de0 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb2d806700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb2d806700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39778/0 (socket says 172.21.15.63:39778) 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.066+0000 7fdb2d806700 1 -- 172.21.15.63:0/1783563568 learned_addr learned my addr 172.21.15.63:0/1783563568 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb2e007700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28102de0 0x7fdb28113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb2d806700 1 -- 172.21.15.63:0/1783563568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28102de0 msgr2=0x7fdb28113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb2d806700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28102de0 0x7fdb28113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb2d806700 1 -- 172.21.15.63:0/1783563568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18009940 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb2d806700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c002a50 tx=0x7fdb1c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1c007cc0 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb1c00f040 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2810df70 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2810e530 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.067+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1c00da40 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.068+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb140052b0 con 0x7fdb28103780 2024-04-24T22:53:34.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.069+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb1c00dba0 con 0x7fdb28103780 2024-04-24T22:53:34.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.069+0000 7fdb26ffd700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 0x7fdb100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.069+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb1c08d810 con 0x7fdb28103780 2024-04-24T22:53:34.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.069+0000 7fdb2e007700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 0x7fdb100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.069+0000 7fdb2e007700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 0x7fdb100705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb18015040 tx=0x7fdb18004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.074+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb1c05a790 con 0x7fdb28103780 2024-04-24T22:53:34.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.326+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb14005d70 con 0x7fdb28103780 2024-04-24T22:53:34.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.327+0000 7fdb26ffd700 1 -- 172.21.15.63:0/1783563568 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fdb1c05a320 con 0x7fdb28103780 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:53:34.331 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:53:34.332 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:53:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:53:34.334 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:34.334 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:34.334 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 msgr2=0x7fdb100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 0x7fdb100705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb18015040 tx=0x7fdb18004d10 comp rx=0 tx=0).stop 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 msgr2=0x7fdb28114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c002a50 tx=0x7fdb1c004600 comp rx=0 tx=0).stop 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 shutdown_connections 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb28102de0 0x7fdb28113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdb1006e140 0x7fdb100705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 --2- 172.21.15.63:0/1783563568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb28103780 0x7fdb28114010 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 >> 172.21.15.63:0/1783563568 conn(0x7fdb280f8990 msgr2=0x7fdb28107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 shutdown_connections 2024-04-24T22:53:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.329+0000 7fdb3026b700 1 -- 172.21.15.63:0/1783563568 wait complete. 2024-04-24T22:53:34.336 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:53:34.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 -- 172.21.15.63:0/1174084638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20102de0 msgr2=0x7f9c201031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 --2- 172.21.15.63:0/1174084638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20102de0 0x7f9c201031b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c10004940 tx=0x7f9c10009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:34.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 -- 172.21.15.63:0/1174084638 shutdown_connections 2024-04-24T22:53:34.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 --2- 172.21.15.63:0/1174084638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20103780 0x7f9c200fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 --2- 172.21.15.63:0/1174084638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20102de0 0x7f9c201031b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 -- 172.21.15.63:0/1174084638 >> 172.21.15.63:0/1174084638 conn(0x7f9c200f8990 msgr2=0x7f9c200fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:34.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.434+0000 7f9c25db2700 1 -- 172.21.15.63:0/1174084638 shutdown_connections 2024-04-24T22:53:34.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 -- 172.21.15.63:0/1174084638 wait complete. 2024-04-24T22:53:34.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 Processor -- start 2024-04-24T22:53:34.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 -- start start 2024-04-24T22:53:34.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20102de0 0x7f9c201139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20114530 con 0x7f9c20103780 2024-04-24T22:53:34.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.435+0000 7f9c25db2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2010db30 con 0x7f9c20102de0 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39792/0 (socket says 172.21.15.63:39792) 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 -- 172.21.15.63:0/3584189530 learned_addr learned my addr 172.21.15.63:0/3584189530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 -- 172.21.15.63:0/3584189530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20102de0 msgr2=0x7f9c201139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20102de0 0x7f9c201139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 -- 172.21.15.63:0/3584189530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10005040 con 0x7f9c20103780 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c1effd700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c14002a80 tx=0x7f9c14004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c14007cc0 con 0x7f9c20103780 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2010de10 con 0x7f9c20103780 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c1400f040 con 0x7f9c20103780 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1400daf0 con 0x7f9c20103780 2024-04-24T22:53:34.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.436+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2010e3d0 con 0x7f9c20103780 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.437+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c201000f0 con 0x7f9c20103780 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.438+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c1400dcf0 con 0x7f9c20103780 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.438+0000 7f9c07fff700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 0x7f9c080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.438+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9c1408d0d0 con 0x7f9c20103780 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.438+0000 7f9c1f7fe700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 0x7f9c080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.439+0000 7f9c1f7fe700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 0x7f9c080705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c10004f90 tx=0x7f9c10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.443+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9c1405a050 con 0x7f9c20103780 2024-04-24T22:53:34.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.671+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c2010eff0 con 0x7f9c0806e140 2024-04-24T22:53:34.673 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:34 smithi063 ceph-mon[111213]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:34.673 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:34 smithi063 ceph-mon[111213]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:34.673 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:34 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/648188452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:34.674 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:34 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1783563568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.673+0000 7f9c07fff700 1 -- 172.21.15.63:0/3584189530 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9c2010eff0 con 0x7f9c0806e140 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:53:34.674 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "10/21 daemons upgraded", 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:53:34.675 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 msgr2=0x7f9c080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 0x7f9c080705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c10004f90 tx=0x7f9c10009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 msgr2=0x7f9c20113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c14002a80 tx=0x7f9c14004630 comp rx=0 tx=0).stop 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 shutdown_connections 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20102de0 0x7f9c201139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9c0806e140 0x7f9c080705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 --2- 172.21.15.63:0/3584189530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c20103780 0x7f9c20113ee0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 >> 172.21.15.63:0/3584189530 conn(0x7f9c200f8990 msgr2=0x7f9c201073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:34.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 shutdown_connections 2024-04-24T22:53:34.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.675+0000 7f9c25db2700 1 -- 172.21.15.63:0/3584189530 wait complete. 2024-04-24T22:53:34.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:34 smithi192 ceph-mon[80463]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:34.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:34 smithi192 ceph-mon[80463]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:34.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:34 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/648188452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:34.678 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:34 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1783563568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/1734969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 msgr2=0x7fc6f00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/1734969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f00fc1b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8004940 tx=0x7fc6d8009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/1734969609 shutdown_connections 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/1734969609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f00fc1b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/1734969609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/1734969609 >> 172.21.15.63:0/1734969609 conn(0x7fc6f0071d00 msgr2=0x7fc6f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:34.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/1734969609 shutdown_connections 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.779+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/1734969609 wait complete. 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 Processor -- start 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 -- start start 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f010f750 con 0x7fc6f00fbd40 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6f4cb3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f010f890 con 0x7fc6f0101d90 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39810/0 (socket says 172.21.15.63:39810) 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 -- 172.21.15.63:0/2400454140 learned_addr learned my addr 172.21.15.63:0/2400454140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6edd9b700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 -- 172.21.15.63:0/2400454140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 msgr2=0x7fc6f010f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f010f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 -- 172.21.15.63:0/2400454140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8005040 con 0x7fc6f00fbd40 2024-04-24T22:53:34.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6edd9b700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.780+0000 7fc6ee59c700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cac0 tx=0x7fc6e8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e800aa50 con 0x7fc6f00fbd40 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6e800abb0 con 0x7fc6f00fbd40 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0117b80 con 0x7fc6f00fbd40 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e80115e0 con 0x7fc6f00fbd40 2024-04-24T22:53:34.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f0118110 con 0x7fc6f00fbd40 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.781+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f00fd430 con 0x7fc6f00fbd40 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.782+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc6e800ad20 con 0x7fc6f00fbd40 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.782+0000 7fc6e6ffd700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 0x7fc6dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.783+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc6e808e240 con 0x7fc6f00fbd40 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.783+0000 7fc6edd9b700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 0x7fc6dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:53:34.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.783+0000 7fc6edd9b700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 0x7fc6dc0705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b3b0 tx=0x7fc6d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:53:34.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:34.787+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc6e8057610 con 0x7fc6f00fbd40 2024-04-24T22:53:35.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.078+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc6f004bed0 con 0x7fc6f00fbd40 2024-04-24T22:53:35.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.079+0000 7fc6e6ffd700 1 -- 172.21.15.63:0/2400454140 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc6e805acd0 con 0x7fc6f00fbd40 2024-04-24T22:53:35.080 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:53:35.080 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:53:35.080 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 msgr2=0x7fc6dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 0x7fc6dc0705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b3b0 tx=0x7fc6d8009e30 comp rx=0 tx=0).stop 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 msgr2=0x7fc6f010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cac0 tx=0x7fc6e8007ee0 comp rx=0 tx=0).stop 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 shutdown_connections 2024-04-24T22:53:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc6dc06e140 0x7fc6dc0705f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:35.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6f00fbd40 0x7fc6f010ecd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:35.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 --2- 172.21.15.63:0/2400454140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101d90 0x7fc6f010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:53:35.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 >> 172.21.15.63:0/2400454140 conn(0x7fc6f0071d00 msgr2=0x7fc6f0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:53:35.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 shutdown_connections 2024-04-24T22:53:35.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:53:35.081+0000 7fc6f4cb3700 1 -- 172.21.15.63:0/2400454140 wait complete. 2024-04-24T22:53:35.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:35 smithi192 ceph-mon[80463]: pgmap v130: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:35.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:35 smithi192 ceph-mon[80463]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:35.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:35 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2400454140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:53:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:35 smithi063 ceph-mon[111213]: pgmap v130: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:35 smithi063 ceph-mon[111213]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:53:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:35 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2400454140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:53:36.687 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:37 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:37 smithi063 ceph-mon[111213]: Upgrade: osd.4 is safe to restart 2024-04-24T22:53:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:37 smithi063 ceph-mon[111213]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:37 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T22:53:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:37 smithi192 ceph-mon[80463]: Upgrade: osd.4 is safe to restart 2024-04-24T22:53:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:37 smithi192 ceph-mon[80463]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:38 smithi192 ceph-mon[80463]: Upgrade: Updating osd.4 2024-04-24T22:53:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:53:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:38 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:38.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:38 smithi192 ceph-mon[80463]: Deploying daemon osd.4 on smithi192 2024-04-24T22:53:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:38 smithi063 ceph-mon[111213]: Upgrade: Updating osd.4 2024-04-24T22:53:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T22:53:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:38 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:38.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:38 smithi063 ceph-mon[111213]: Deploying daemon osd.4 on smithi192 2024-04-24T22:53:39.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:39 smithi192 ceph-mon[80463]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:39 smithi063 ceph-mon[111213]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:41.079 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:40 smithi192 systemd[1]: Stopping Ceph osd.4 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:53:41.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[45845]: 2024-04-24T22:53:41.077+0000 7f376a304700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:53:41.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[45845]: 2024-04-24T22:53:41.077+0000 7f376a304700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-24T22:53:41.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[45845]: 2024-04-24T22:53:41.077+0000 7f376a304700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:53:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:41 smithi063 ceph-mon[111213]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:41 smithi063 ceph-mon[111213]: osd.4 marked itself down and dead 2024-04-24T22:53:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:41 smithi192 ceph-mon[80463]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:41 smithi192 ceph-mon[80463]: osd.4 marked itself down and dead 2024-04-24T22:53:42.337 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:42 smithi192 podman[94658]: 2024-04-24 22:53:42.084036994 +0000 UTC m=+1.118181770 container died 18dcb9bd657367d9b01b66f4683284aee3695f75371c635f2c58d6567ef4a3be (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T22:53:42.655 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:42 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:53:42.655 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:42 smithi192 ceph-mon[80463]: osdmap e70: 6 total, 5 up, 6 in 2024-04-24T22:53:42.655 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:42 smithi192 podman[94658]: 2024-04-24 22:53:42.427267467 +0000 UTC m=+1.461412229 container cleanup 18dcb9bd657367d9b01b66f4683284aee3695f75371c635f2c58d6567ef4a3be (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T22:53:42.655 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:42 smithi192 bash[94658]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4 2024-04-24T22:53:42.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:42 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:53:42.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:42 smithi063 ceph-mon[111213]: osdmap e70: 6 total, 5 up, 6 in 2024-04-24T22:53:42.927 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:42 smithi192 podman[94688]: 2024-04-24 22:53:42.653744672 +0000 UTC m=+0.565665648 container remove 18dcb9bd657367d9b01b66f4683284aee3695f75371c635f2c58d6567ef4a3be (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T22:53:43.383 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:42.989013174 +0000 UTC m=+0.018194577 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:43.383 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:43.096438966 +0000 UTC m=+0.125620428 container create 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T22:53:43.383 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:43.382147541 +0000 UTC m=+0.411328938 container init 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T22:53:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:43 smithi192 ceph-mon[80463]: pgmap v135: 65 pgs: 6 stale+active+clean, 59 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:43.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:43 smithi192 ceph-mon[80463]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T22:53:43.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:43.385325998 +0000 UTC m=+0.414507397 container start 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T22:53:43.678 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:43.473823862 +0000 UTC m=+0.503005277 container attach 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T22:53:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:43 smithi063 ceph-mon[111213]: pgmap v135: 65 pgs: 6 stale+active+clean, 59 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:53:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:43 smithi063 ceph-mon[111213]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T22:53:43.977 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:43 smithi192 podman[94782]: 2024-04-24 22:53:43.689340742 +0000 UTC m=+0.718522158 container died 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T22:53:44.586 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 podman[94808]: 2024-04-24 22:53:44.243919085 +0000 UTC m=+0.548895360 container remove 4edc073c09bd811e5c3b5e6b0bdfebc8d95b1e02836f0f328a47accebb8d30ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T22:53:44.586 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-24T22:53:44.586 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 systemd[1]: Stopped Ceph osd.4 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:53:44.586 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.4.service: Consumed 51.547s CPU time. 2024-04-24T22:53:44.586 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 systemd[1]: Starting Ceph osd.4 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:53:44.843 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:44 smithi192 ceph-mon[80463]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T22:53:44.843 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:44 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:44.844 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 podman[94911]: 2024-04-24 22:53:44.591561359 +0000 UTC m=+0.018391436 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:44.844 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 podman[94911]: 2024-04-24 22:53:44.728755506 +0000 UTC m=+0.155585612 container create 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-24T22:53:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:44 smithi063 ceph-mon[111213]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T22:53:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:44 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:45.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 podman[94911]: 2024-04-24 22:53:44.989866688 +0000 UTC m=+0.416696756 container init 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:53:45.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:44 smithi192 podman[94911]: 2024-04-24 22:53:44.993175215 +0000 UTC m=+0.420005287 container start 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T22:53:45.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 podman[94911]: 2024-04-24 22:53:45.073099179 +0000 UTC m=+0.499929261 container attach 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T22:53:45.793 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-mon[80463]: pgmap v137: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22/264 objects degraded (8.333%) 2024-04-24T22:53:45.794 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:45.794 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T22:53:45.794 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 bash[94911]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate[94925]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T22:53:45.795 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:45 smithi192 podman[94911]: 2024-04-24 22:53:45.462697055 +0000 UTC m=+0.889527120 container died 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T22:53:45.954 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:45 smithi063 ceph-mon[111213]: pgmap v137: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22/264 objects degraded (8.333%) 2024-04-24T22:53:45.954 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:45.954 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:53:46.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 podman[95050]: 2024-04-24 22:53:46.085527125 +0000 UTC m=+0.614024404 container remove 632eabe069540334a52a6ce7c1560b9d21be742e806c73e469a50c96cf17b8d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T22:53:46.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 podman[95111]: 2024-04-24 22:53:46.203771165 +0000 UTC m=+0.018446722 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:53:46.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 podman[95111]: 2024-04-24 22:53:46.328118679 +0000 UTC m=+0.142794223 container create 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T22:53:46.927 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 podman[95111]: 2024-04-24 22:53:46.572415096 +0000 UTC m=+0.387090644 container init 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T22:53:46.927 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 podman[95111]: 2024-04-24 22:53:46.575466579 +0000 UTC m=+0.390142123 container start 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T22:53:46.927 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 ceph-osd[95130]: -- 172.21.15.192:0/3423925774 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55891afcf680 con 0x55891af99000 2024-04-24T22:53:46.927 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 bash[95111]: 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 2024-04-24T22:53:46.928 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:46 smithi192 systemd[1]: Started Ceph osd.4 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:53:47.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:47 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T22:53:47.114+0000 7f40af1d27c0 -1 Falling back to public interface 2024-04-24T22:53:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:47 smithi192 ceph-mon[80463]: pgmap v138: 65 pgs: 11 active+undersized+degraded, 10 peering, 13 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/264 objects degraded (14.773%) 2024-04-24T22:53:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:47 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:48.047 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:47 smithi063 ceph-mon[111213]: pgmap v138: 65 pgs: 11 active+undersized+degraded, 10 peering, 13 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/264 objects degraded (14.773%) 2024-04-24T22:53:48.047 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:48.047 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:48.048 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:47 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:49.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:48 smithi063 ceph-mon[111213]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T22:53:49.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:48 smithi192 ceph-mon[80463]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T22:53:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:49 smithi063 ceph-mon[111213]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:49 smithi192 ceph-mon[80463]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:51 smithi063 ceph-mon[111213]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:51 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:52.078 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:51 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T22:53:51.994+0000 7f40af1d27c0 -1 osd.4 69 log_to_monitors true 2024-04-24T22:53:52.078 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:51 smithi192 ceph-mon[80463]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:52.078 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:52.078 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:52.078 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:51 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:52 smithi192 ceph-mon[80463]: from='osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:53:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:52 smithi192 ceph-mon[80463]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:53:52.928 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 22:53:52 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T22:53:52.804+0000 7f40a062a700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:53:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:52 smithi063 ceph-mon[111213]: from='osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:53:53.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:52 smithi063 ceph-mon[111213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T22:53:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:53 smithi063 ceph-mon[111213]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:53 smithi063 ceph-mon[111213]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:53:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:53 smithi063 ceph-mon[111213]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T22:53:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:53 smithi063 ceph-mon[111213]: from='osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:53 smithi063 ceph-mon[111213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:54.091 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:53 smithi192 ceph-mon[80463]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:54.091 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:53 smithi192 ceph-mon[80463]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T22:53:54.091 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:53 smithi192 ceph-mon[80463]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T22:53:54.091 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:53 smithi192 ceph-mon[80463]: from='osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:54.091 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:53 smithi192 ceph-mon[80463]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:53:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:53:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084] boot 2024-04-24T22:53:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: osdmap e73: 6 total, 6 up, 6 in 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:55.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:53:55.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:54 smithi063 ceph-mon[111213]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: osd.4 [v2:172.21.15.192:6808/2986357084,v1:172.21.15.192:6809/2986357084] boot 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: osdmap e73: 6 total, 6 up, 6 in 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:53:55.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:54 smithi192 ceph-mon[80463]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T22:53:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:55 smithi063 ceph-mon[111213]: pgmap v144: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:55 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:53:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:55 smithi063 ceph-mon[111213]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-24T22:53:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:55 smithi192 ceph-mon[80463]: pgmap v144: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-24T22:53:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:55 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:53:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:55 smithi192 ceph-mon[80463]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-24T22:53:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:57 smithi192 ceph-mon[80463]: pgmap v146: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:53:57.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:57 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:57.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:57 smithi063 ceph-mon[111213]: pgmap v146: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-24T22:53:57.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:57 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T22:53:59.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:53:59 smithi192 ceph-mon[80463]: pgmap v147: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/264 objects degraded (8.333%) 2024-04-24T22:53:59.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:53:59 smithi063 ceph-mon[111213]: pgmap v147: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/264 objects degraded (8.333%) 2024-04-24T22:54:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:01 smithi192 ceph-mon[80463]: pgmap v148: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:01 smithi063 ceph-mon[111213]: pgmap v148: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:02 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded) 2024-04-24T22:54:02.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:02 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded) 2024-04-24T22:54:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:03 smithi192 ceph-mon[80463]: pgmap v149: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:03.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:03 smithi063 ceph-mon[111213]: pgmap v149: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.201+0000 7f40d1cf6700 1 -- 172.21.15.63:0/3315597969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 msgr2=0x7f40cc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.201+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/3315597969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0fce10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f40b4004940 tx=0x7f40b4009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 -- 172.21.15.63:0/3315597969 shutdown_connections 2024-04-24T22:54:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/3315597969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0fce10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/3315597969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 0x7f40cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 -- 172.21.15.63:0/3315597969 >> 172.21.15.63:0/3315597969 conn(0x7f40cc0f89d0 msgr2=0x7f40cc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 -- 172.21.15.63:0/3315597969 shutdown_connections 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 -- 172.21.15.63:0/3315597969 wait complete. 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 Processor -- start 2024-04-24T22:54:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.202+0000 7f40d1cf6700 1 -- start start 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40d1cf6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 0x7f40cc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40d1cf6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40d1cf6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0721b0 con 0x7f40cc1037a0 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40d1cf6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc070ce0 con 0x7f40cc102e00 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56220/0 (socket says 172.21.15.63:56220) 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 -- 172.21.15.63:0/1316453662 learned_addr learned my addr 172.21.15.63:0/1316453662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40cb7fe700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 0x7f40cc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 -- 172.21.15.63:0/1316453662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 msgr2=0x7f40cc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 0x7f40cc070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 -- 172.21.15.63:0/1316453662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4005040 con 0x7f40cc1037a0 2024-04-24T22:54:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c2ffd700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c00 tx=0x7f40b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b4021070 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc070f60 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.203+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b4013c00 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.204+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc119f70 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.204+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b400e070 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.205+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc1052d0 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.205+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40b401ca60 con 0x7f40cc1037a0 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.205+0000 7f40c8ff9700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 0x7f40b80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.206+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f40b408edd0 con 0x7f40cc1037a0 2024-04-24T22:54:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.206+0000 7f40cb7fe700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 0x7f40b80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.206+0000 7f40cb7fe700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 0x7f40b80705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009b80 tx=0x7f40bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.211+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f40b40582c0 con 0x7f40cc1037a0 2024-04-24T22:54:05.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:05 smithi192 ceph-mon[80463]: pgmap v150: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:05.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.438+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40cc0717b0 con 0x7f40b806e140 2024-04-24T22:54:05.441 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:05 smithi063 ceph-mon[111213]: pgmap v150: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:05.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.439+0000 7f40c8ff9700 1 -- 172.21.15.63:0/1316453662 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f40cc0717b0 con 0x7f40b806e140 2024-04-24T22:54:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.441+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 msgr2=0x7f40b80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.441+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 0x7f40b80705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009b80 tx=0x7f40bc00b040 comp rx=0 tx=0).stop 2024-04-24T22:54:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.441+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 msgr2=0x7f40cc0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.441+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c00 tx=0x7f40b4009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.441+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 shutdown_connections 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40cc102e00 0x7f40cc070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f40b806e140 0x7f40b80705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 --2- 172.21.15.63:0/1316453662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc1037a0 0x7f40cc0707a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 >> 172.21.15.63:0/1316453662 conn(0x7f40cc0f89d0 msgr2=0x7f40cc0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 shutdown_connections 2024-04-24T22:54:05.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.442+0000 7f40d1cf6700 1 -- 172.21.15.63:0/1316453662 wait complete. 2024-04-24T22:54:05.456 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:54:05.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 -- 172.21.15.63:0/3324832078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 msgr2=0x7fe5d00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/3324832078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00fcdf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0004940 tx=0x7fe5c0009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 -- 172.21.15.63:0/3324832078 shutdown_connections 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/3324832078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00fcdf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/3324832078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.546+0000 7fe5d72b8700 1 -- 172.21.15.63:0/3324832078 >> 172.21.15.63:0/3324832078 conn(0x7fe5d00f8990 msgr2=0x7fe5d00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 -- 172.21.15.63:0/3324832078 shutdown_connections 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 -- 172.21.15.63:0/3324832078 wait complete. 2024-04-24T22:54:05.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 Processor -- start 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 -- start start 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0072240 con 0x7fe5d0103780 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d72b8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0070ce0 con 0x7fe5d0102de0 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d4853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d4853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56242/0 (socket says 172.21.15.63:56242) 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.547+0000 7fe5d4853700 1 -- 172.21.15.63:0/2865620526 learned_addr learned my addr 172.21.15.63:0/2865620526 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d5054700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d4853700 1 -- 172.21.15.63:0/2865620526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 msgr2=0x7fe5d0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d4853700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d4853700 1 -- 172.21.15.63:0/2865620526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0005040 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d5054700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d4853700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009180 tx=0x7fe5c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c000e040 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5c000bbc0 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d0070f60 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c001c3f0 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.548+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d011a1b0 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.549+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d01000f0 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.550+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe5c000bd30 con 0x7fe5d0103780 2024-04-24T22:54:05.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.550+0000 7fe5c5ffb700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 0x7fe5bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.550+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe5c008ed10 con 0x7fe5d0103780 2024-04-24T22:54:05.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.550+0000 7fe5d5054700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 0x7fe5bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.551+0000 7fe5d5054700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 0x7fe5bc0705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009b10 tx=0x7fe5cc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.555+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe5c0058200 con 0x7fe5d0103780 2024-04-24T22:54:05.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.782+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5d00717b0 con 0x7fe5bc06e140 2024-04-24T22:54:05.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.783+0000 7fe5c5ffb700 1 -- 172.21.15.63:0/2865620526 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe5d00717b0 con 0x7fe5bc06e140 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 msgr2=0x7fe5bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 0x7fe5bc0705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009b10 tx=0x7fe5cc008040 comp rx=0 tx=0).stop 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 msgr2=0x7fe5d00707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009180 tx=0x7fe5c0009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 shutdown_connections 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5d0102de0 0x7fe5d0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe5bc06e140 0x7fe5bc0705f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 --2- 172.21.15.63:0/2865620526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d0103780 0x7fe5d00707a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 >> 172.21.15.63:0/2865620526 conn(0x7fe5d00f8990 msgr2=0x7fe5d00fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 shutdown_connections 2024-04-24T22:54:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.785+0000 7fe5d72b8700 1 -- 172.21.15.63:0/2865620526 wait complete. 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.890+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/3687292867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 msgr2=0x7f6f940fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.890+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/3687292867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f940fba10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004940 tx=0x7f6f80009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/3687292867 shutdown_connections 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/3687292867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f940fba10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/3687292867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f941008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/3687292867 >> 172.21.15.63:0/3687292867 conn(0x7f6f940714d0 msgr2=0x7f6f940718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/3687292867 shutdown_connections 2024-04-24T22:54:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/3687292867 wait complete. 2024-04-24T22:54:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 Processor -- start 2024-04-24T22:54:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.891+0000 7f6f9ac8a700 1 -- start start 2024-04-24T22:54:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f9ac8a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f94113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f9ac8a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f9ac8a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941144a0 con 0x7f6f941004d0 2024-04-24T22:54:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f9ac8a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9410daa0 con 0x7f6f940fb5a0 2024-04-24T22:54:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56266/0 (socket says 172.21.15.63:56266) 2024-04-24T22:54:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 -- 172.21.15.63:0/1042225921 learned_addr learned my addr 172.21.15.63:0/1042225921 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f98a26700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f94113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 -- 172.21.15.63:0/1042225921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 msgr2=0x7f6f94113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f94113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 -- 172.21.15.63:0/1042225921 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80005040 con 0x7f6f941004d0 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f98a26700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f94113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.892+0000 7f6f93fff700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b8b0 tx=0x7f6f80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.893+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8000e040 con 0x7f6f941004d0 2024-04-24T22:54:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.893+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9410dd20 con 0x7f6f941004d0 2024-04-24T22:54:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.893+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f80004290 con 0x7f6f941004d0 2024-04-24T22:54:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.893+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8001c3f0 con 0x7f6f941004d0 2024-04-24T22:54:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.893+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9410e2e0 con 0x7f6f941004d0 2024-04-24T22:54:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.894+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f94105a70 con 0x7f6f941004d0 2024-04-24T22:54:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.894+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6f800137b0 con 0x7f6f941004d0 2024-04-24T22:54:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.895+0000 7f6f917fa700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 0x7f6f840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.895+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6f8008ee00 con 0x7f6f941004d0 2024-04-24T22:54:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.895+0000 7f6f98a26700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 0x7f6f840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.895+0000 7f6f98a26700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 0x7f6f840705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009fd0 tx=0x7f6f88009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:05.899+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6f8005bd80 con 0x7f6f941004d0 2024-04-24T22:54:06.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.117+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f9410ef60 con 0x7f6f8406e140 2024-04-24T22:54:06.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.127+0000 7f6f917fa700 1 -- 172.21.15.63:0/1042225921 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6f9410ef60 con 0x7f6f8406e140 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 83s ago 17m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 83s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (3m) 16s ago 16m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 83s ago 16m 81.1M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (11m) 83s ago 11m 15.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (11m) 83s ago 11m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af a99f9b98ef82 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (11m) 16s ago 11m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (11m) 16s ago 11m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:54:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (7m) 83s ago 18m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (6m) 16s ago 16m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 83s ago 18m 47.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (3m) 16s ago 15m 38.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (6m) 83s ago 17m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 16s ago 16m 16.9M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 83s ago 15m 91.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 83s ago 14m 91.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (89s) 83s ago 14m 12.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (53s) 16s ago 13m 89.9M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (19s) 16s ago 13m 14.8M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (13m) 16s ago 13m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 99ca205d2f05 2024-04-24T22:54:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 83s ago 16m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 msgr2=0x7f6f840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 0x7f6f840705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009fd0 tx=0x7f6f88009ee0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 msgr2=0x7f6f94113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b8b0 tx=0x7f6f80009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 shutdown_connections 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f940fb5a0 0x7f6f94113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6f8406e140 0x7f6f840705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 --2- 172.21.15.63:0/1042225921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f941004d0 0x7f6f94113e50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.129+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 >> 172.21.15.63:0/1042225921 conn(0x7f6f940714d0 msgr2=0x7f6f940f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:06.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.130+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 shutdown_connections 2024-04-24T22:54:06.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.130+0000 7f6f9ac8a700 1 -- 172.21.15.63:0/1042225921 wait complete. 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 -- 172.21.15.63:0/780172077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80fb520 msgr2=0x7f75d80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/780172077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80fb520 0x7f75d80fb990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75c8004940 tx=0x7f75c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 -- 172.21.15.63:0/780172077 shutdown_connections 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/780172077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80fb520 0x7f75d80fb990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/780172077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d8102f60 0x7f75d8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 -- 172.21.15.63:0/780172077 >> 172.21.15.63:0/780172077 conn(0x7f75d80714d0 msgr2=0x7f75d80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 -- 172.21.15.63:0/780172077 shutdown_connections 2024-04-24T22:54:06.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.234+0000 7f75dd6b9700 1 -- 172.21.15.63:0/780172077 wait complete. 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 Processor -- start 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 -- start start 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 0x7f75d80fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80fe650 con 0x7f75d8102f60 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75dd6b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80fd180 con 0x7f75d80fb520 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75d67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75d67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56294/0 (socket says 172.21.15.63:56294) 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75d67fc700 1 -- 172.21.15.63:0/638865069 learned_addr learned my addr 172.21.15.63:0/638865069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.235+0000 7f75d6ffd700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 0x7f75d80fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75d67fc700 1 -- 172.21.15.63:0/638865069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 msgr2=0x7f75d80fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75d67fc700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 0x7f75d80fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75d67fc700 1 -- 172.21.15.63:0/638865069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8005040 con 0x7f75d8102f60 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75d6ffd700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 0x7f75d80fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75d67fc700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b8b0 tx=0x7f75c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75c800e040 con 0x7f75d8102f60 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75c8004290 con 0x7f75d8102f60 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d80fd400 con 0x7f75d8102f60 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75c801c3f0 con 0x7f75d8102f60 2024-04-24T22:54:06.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.236+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d811a090 con 0x7f75d8102f60 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.237+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d8100250 con 0x7f75d8102f60 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.238+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f75c80137b0 con 0x7f75d8102f60 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.238+0000 7f75bf7fe700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 0x7f75c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.238+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f75c808ed10 con 0x7f75d8102f60 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.238+0000 7f75d6ffd700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 0x7f75c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.239+0000 7f75d6ffd700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 0x7f75c00705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75cc007fd0 tx=0x7f75cc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.243+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f75c8058200 con 0x7f75d8102f60 2024-04-24T22:54:06.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:06 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:06.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:06 smithi192 ceph-mon[80463]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:06.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:06 smithi192 ceph-mon[80463]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:06.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.540+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f75d810a000 con 0x7f75d8102f60 2024-04-24T22:54:06.543 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:06 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:06.543 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:06 smithi063 ceph-mon[111213]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:06.543 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:06 smithi063 ceph-mon[111213]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:06.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.540+0000 7f75bf7fe700 1 -- 172.21.15.63:0/638865069 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f75c808e070 con 0x7f75d8102f60 2024-04-24T22:54:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:54:06.543 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:54:06.543 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 9 2024-04-24T22:54:06.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.542+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 msgr2=0x7f75c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.542+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 0x7f75c00705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75cc007fd0 tx=0x7f75cc009040 comp rx=0 tx=0).stop 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.542+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 msgr2=0x7f75d80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.542+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b8b0 tx=0x7f75c8009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 shutdown_connections 2024-04-24T22:54:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75d80fb520 0x7f75d80fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f75c006e140 0x7f75c00705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 --2- 172.21.15.63:0/638865069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d8102f60 0x7f75d80fcc40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 >> 172.21.15.63:0/638865069 conn(0x7f75d80714d0 msgr2=0x7f75d80fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:06.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 shutdown_connections 2024-04-24T22:54:06.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.543+0000 7f75dd6b9700 1 -- 172.21.15.63:0/638865069 wait complete. 2024-04-24T22:54:06.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 -- 172.21.15.63:0/3854784058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69201037c0 msgr2=0x7f69200fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 --2- 172.21.15.63:0/3854784058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69201037c0 0x7f69200fb3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6910004940 tx=0x7f6910009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 -- 172.21.15.63:0/3854784058 shutdown_connections 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 --2- 172.21.15.63:0/3854784058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69201037c0 0x7f69200fb3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 --2- 172.21.15.63:0/3854784058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6920102e20 0x7f69201031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 -- 172.21.15.63:0/3854784058 >> 172.21.15.63:0/3854784058 conn(0x7f6920072380 msgr2=0x7f6920072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 -- 172.21.15.63:0/3854784058 shutdown_connections 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.647+0000 7f6927d10700 1 -- 172.21.15.63:0/3854784058 wait complete. 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 Processor -- start 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 -- start start 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69201037c0 0x7f6920113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6920114660 con 0x7f6920102e20 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6927d10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692010dbb0 con 0x7f69201037c0 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6925aac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6925aac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56320/0 (socket says 172.21.15.63:56320) 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f6925aac700 1 -- 172.21.15.63:0/3872531115 learned_addr learned my addr 172.21.15.63:0/3872531115 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:06.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.648+0000 7f69252ab700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69201037c0 0x7f6920113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6925aac700 1 -- 172.21.15.63:0/3872531115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69201037c0 msgr2=0x7f6920113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6925aac700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69201037c0 0x7f6920113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6925aac700 1 -- 172.21.15.63:0/3872531115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910005040 con 0x7f6920102e20 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6925aac700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b790 tx=0x7f691c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c00f6f0 con 0x7f6920102e20 2024-04-24T22:54:06.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f691c00fd20 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692010de30 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c0103a0 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.649+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692010e370 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.650+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69200fe6e0 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.651+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f691c011400 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.651+0000 7f69167fc700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 0x7f690c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.651+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f691c08d780 con 0x7f6920102e20 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.651+0000 7f69252ab700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 0x7f690c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:06.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.651+0000 7f69252ab700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 0x7f690c0705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f692010f2e0 tx=0x7f6910009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:06.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.656+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f691c05a7a0 con 0x7f6920102e20 2024-04-24T22:54:06.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.907+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f692010f150 con 0x7f6920102e20 2024-04-24T22:54:06.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.908+0000 7f69167fc700 1 -- 172.21.15.63:0/3872531115 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f691c05a330 con 0x7f6920102e20 2024-04-24T22:54:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:e14 2024-04-24T22:54:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:54:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:54:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3569757524,v1:172.21.15.63:6827/3569757524] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 msgr2=0x7f690c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 0x7f690c0705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f692010f2e0 tx=0x7f6910009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 msgr2=0x7f6920113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b790 tx=0x7f691c009220 comp rx=0 tx=0).stop 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 shutdown_connections 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690c06e140 0x7f690c0705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6920102e20 0x7f6920113a40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 --2- 172.21.15.63:0/3872531115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69201037c0 0x7f6920113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:06.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 >> 172.21.15.63:0/3872531115 conn(0x7f6920072380 msgr2=0x7f69200f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:06.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 shutdown_connections 2024-04-24T22:54:06.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:06.910+0000 7f6927d10700 1 -- 172.21.15.63:0/3872531115 wait complete. 2024-04-24T22:54:06.915 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 14 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 -- 172.21.15.63:0/3250432238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34102de0 msgr2=0x7f7b341031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3250432238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34102de0 0x7f7b341031b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b24004940 tx=0x7f7b24009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 -- 172.21.15.63:0/3250432238 shutdown_connections 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3250432238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34103780 0x7f7b340fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3250432238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34102de0 0x7f7b341031b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 -- 172.21.15.63:0/3250432238 >> 172.21.15.63:0/3250432238 conn(0x7f7b340f89b0 msgr2=0x7f7b340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 -- 172.21.15.63:0/3250432238 shutdown_connections 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 -- 172.21.15.63:0/3250432238 wait complete. 2024-04-24T22:54:07.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.021+0000 7f7b3a903700 1 Processor -- start 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b3a903700 1 -- start start 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b3a903700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34102de0 0x7f7b34070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b3a903700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b3a903700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34070ce0 con 0x7f7b34103780 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b3a903700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34070e20 con 0x7f7b34102de0 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56332/0 (socket says 172.21.15.63:56332) 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 -- 172.21.15.63:0/3958254687 learned_addr learned my addr 172.21.15.63:0/3958254687 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b33fff700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34102de0 0x7f7b34070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 -- 172.21.15.63:0/3958254687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34102de0 msgr2=0x7f7b34070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34102de0 0x7f7b34070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.022+0000 7f7b337fe700 1 -- 172.21.15.63:0/3958254687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b24005040 con 0x7f7b34103780 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b337fe700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b28002a50 tx=0x7f7b28004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b28007cc0 con 0x7f7b34103780 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b340710d0 con 0x7f7b34103780 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b2800f040 con 0x7f7b34103780 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2800d9c0 con 0x7f7b34103780 2024-04-24T22:54:07.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.023+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3411a2c0 con 0x7f7b34103780 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.024+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b341000f0 con 0x7f7b34103780 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.024+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b2800db60 con 0x7f7b34103780 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.025+0000 7f7b30ff9700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 0x7f7b1c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.025+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7b2808d6c0 con 0x7f7b34103780 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.025+0000 7f7b33fff700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 0x7f7b1c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.025+0000 7f7b33fff700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 0x7f7b1c0705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24004f90 tx=0x7f7b24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.029+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7b28056a90 con 0x7f7b34103780 2024-04-24T22:54:07.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.258+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b34071970 con 0x7f7b1c06e140 2024-04-24T22:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.260+0000 7f7b30ff9700 1 -- 172.21.15.63:0/3958254687 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7b34071970 con 0x7f7b1c06e140 2024-04-24T22:54:07.262 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:54:07.262 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:54:07.262 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "11/21 daemons upgraded", 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:54:07.263 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:54:07.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 msgr2=0x7f7b1c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 0x7f7b1c0705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24004f90 tx=0x7f7b24009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 msgr2=0x7f7b340707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b28002a50 tx=0x7f7b28004600 comp rx=0 tx=0).stop 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 shutdown_connections 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b34102de0 0x7f7b34070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f7b1c06e140 0x7f7b1c0705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 --2- 172.21.15.63:0/3958254687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b34103780 0x7f7b340707a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 >> 172.21.15.63:0/3958254687 conn(0x7f7b340f89b0 msgr2=0x7f7b341073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 shutdown_connections 2024-04-24T22:54:07.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.263+0000 7f7b3a903700 1 -- 172.21.15.63:0/3958254687 wait complete. 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 -- 172.21.15.63:0/3550442670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744065640 msgr2=0x7fe744065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 --2- 172.21.15.63:0/3550442670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744065640 0x7fe744065ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe734004940 tx=0x7fe734009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 -- 172.21.15.63:0/3550442670 shutdown_connections 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 --2- 172.21.15.63:0/3550442670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744065640 0x7fe744065ab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 --2- 172.21.15.63:0/3550442670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744102e50 0x7fe744103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 -- 172.21.15.63:0/3550442670 >> 172.21.15.63:0/3550442670 conn(0x7fe744071d00 msgr2=0x7fe744072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 -- 172.21.15.63:0/3550442670 shutdown_connections 2024-04-24T22:54:07.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.369+0000 7fe74963b700 1 -- 172.21.15.63:0/3550442670 wait complete. 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 Processor -- start 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 -- start start 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744065640 0x7fe744113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe744114660 con 0x7fe744102e50 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe74963b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74410da50 con 0x7fe744065640 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe7427fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe7427fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56358/0 (socket says 172.21.15.63:56358) 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe7427fc700 1 -- 172.21.15.63:0/95113805 learned_addr learned my addr 172.21.15.63:0/95113805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.370+0000 7fe742ffd700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744065640 0x7fe744113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe7427fc700 1 -- 172.21.15.63:0/95113805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744065640 msgr2=0x7fe744113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe7427fc700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744065640 0x7fe744113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe7427fc700 1 -- 172.21.15.63:0/95113805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe734005040 con 0x7fe744102e50 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe7427fc700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe734009b30 tx=0x7fe734009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe734021070 con 0x7fe744102e50 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe734013c00 con 0x7fe744102e50 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74410dcd0 con 0x7fe744102e50 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73400e070 con 0x7fe744102e50 2024-04-24T22:54:07.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.371+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74410e1b0 con 0x7fe744102e50 2024-04-24T22:54:07.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.372+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe744100100 con 0x7fe744102e50 2024-04-24T22:54:07.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.373+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe73401cad0 con 0x7fe744102e50 2024-04-24T22:54:07.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.373+0000 7fe72b7fe700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 0x7fe72c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:07.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.373+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe73408ee00 con 0x7fe744102e50 2024-04-24T22:54:07.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.373+0000 7fe742ffd700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 0x7fe72c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:07.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.373+0000 7fe742ffd700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 0x7fe72c0705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe738009b80 tx=0x7fe73800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:07.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.378+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe73405be20 con 0x7fe744102e50 2024-04-24T22:54:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:07 smithi063 ceph-mon[111213]: pgmap v151: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:07 smithi063 ceph-mon[111213]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:07 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/638865069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:07 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3872531115' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:54:07.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:07 smithi192 ceph-mon[80463]: pgmap v151: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:07.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:07 smithi192 ceph-mon[80463]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:07.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:07 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/638865069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:07.679 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:07 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3872531115' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:54:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.675+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe74410f120 con 0x7fe744102e50 2024-04-24T22:54:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.676+0000 7fe72b7fe700 1 -- 172.21.15.63:0/95113805 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe734004490 con 0x7fe744102e50 2024-04-24T22:54:07.679 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:54:07.679 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:54:07.680 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:54:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 msgr2=0x7fe72c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 0x7fe72c0705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe738009b80 tx=0x7fe73800b040 comp rx=0 tx=0).stop 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 msgr2=0x7fe744114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe734009b30 tx=0x7fe734009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 shutdown_connections 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe744065640 0x7fe744113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe72c06e140 0x7fe72c0705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 --2- 172.21.15.63:0/95113805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe744102e50 0x7fe744114010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 >> 172.21.15.63:0/95113805 conn(0x7fe744071d00 msgr2=0x7fe7440fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 shutdown_connections 2024-04-24T22:54:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:07.678+0000 7fe74963b700 1 -- 172.21.15.63:0/95113805 wait complete. 2024-04-24T22:54:08.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:08 smithi063 ceph-mon[111213]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:08.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:08 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/95113805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:54:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:08 smithi192 ceph-mon[80463]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:08.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:08 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/95113805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:54:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:09 smithi063 ceph-mon[111213]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:09 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:54:09.915 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:09 smithi192 ceph-mon[80463]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:09.915 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:09 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:54:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:10 smithi192 ceph-mon[80463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:54:10.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:10 smithi192 ceph-mon[80463]: Upgrade: osd.5 is safe to restart 2024-04-24T22:54:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:10 smithi063 ceph-mon[111213]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T22:54:10.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:10 smithi063 ceph-mon[111213]: Upgrade: osd.5 is safe to restart 2024-04-24T22:54:12.095 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:11 smithi192 ceph-mon[80463]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:12.095 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:11 smithi192 ceph-mon[80463]: Upgrade: Updating osd.5 2024-04-24T22:54:12.095 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:12.095 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:54:12.095 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:12.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:11 smithi063 ceph-mon[111213]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:12.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:11 smithi063 ceph-mon[111213]: Upgrade: Updating osd.5 2024-04-24T22:54:12.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:12.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T22:54:12.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:12.923 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:12 smithi192 ceph-mon[80463]: Deploying daemon osd.5 on smithi192 2024-04-24T22:54:13.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:12 smithi063 ceph-mon[111213]: Deploying daemon osd.5 on smithi192 2024-04-24T22:54:14.116 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:13 smithi192 ceph-mon[80463]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:14.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:13 smithi063 ceph-mon[111213]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:14.367 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:14 smithi192 systemd[1]: Stopping Ceph osd.5 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:14.367 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:14 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[52335]: 2024-04-24T22:54:14.364+0000 7f4f2b4c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T22:54:14.367 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:14 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[52335]: 2024-04-24T22:54:14.364+0000 7f4f2b4c9700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-24T22:54:14.367 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:14 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[52335]: 2024-04-24T22:54:14.364+0000 7f4f2b4c9700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T22:54:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:14 smithi192 ceph-mon[80463]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:15.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:14 smithi192 ceph-mon[80463]: osd.5 marked itself down and dead 2024-04-24T22:54:15.204 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:14 smithi063 ceph-mon[111213]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:15.204 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:14 smithi063 ceph-mon[111213]: osd.5 marked itself down and dead 2024-04-24T22:54:15.700 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:15 smithi192 podman[99530]: 2024-04-24 22:54:15.449192285 +0000 UTC m=+1.192123560 container died 99ca205d2f05857bf45ec78b3abda5b1de69d40e913b6625f452b574b941719b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T22:54:16.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:15 smithi192 podman[99530]: 2024-04-24 22:54:15.789883498 +0000 UTC m=+1.532814783 container cleanup 99ca205d2f05857bf45ec78b3abda5b1de69d40e913b6625f452b574b941719b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:16.034 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:15 smithi192 bash[99530]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5 2024-04-24T22:54:16.034 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:15 smithi192 ceph-mon[80463]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:54:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:15 smithi192 ceph-mon[80463]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T22:54:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:15 smithi192 ceph-mon[80463]: osdmap e75: 6 total, 5 up, 6 in 2024-04-24T22:54:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:16.035 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:15 smithi063 ceph-mon[111213]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T22:54:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:15 smithi063 ceph-mon[111213]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T22:54:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:15 smithi063 ceph-mon[111213]: osdmap e75: 6 total, 5 up, 6 in 2024-04-24T22:54:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:16.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:16.427 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99546]: 2024-04-24 22:54:16.032982962 +0000 UTC m=+0.586971121 container remove 99ca205d2f05857bf45ec78b3abda5b1de69d40e913b6625f452b574b941719b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:16.721 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99638]: 2024-04-24 22:54:16.355158557 +0000 UTC m=+0.017866198 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:16.721 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99638]: 2024-04-24 22:54:16.50963347 +0000 UTC m=+0.172341131 container create b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T22:54:16.721 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99638]: 2024-04-24 22:54:16.720215537 +0000 UTC m=+0.382923155 container init b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T22:54:17.019 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:16 smithi192 ceph-mon[80463]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T22:54:17.019 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:16 smithi192 ceph-mon[80463]: pgmap v158: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:17.019 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99638]: 2024-04-24 22:54:16.723250841 +0000 UTC m=+0.385958464 container start b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T22:54:17.019 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:16 smithi192 podman[99638]: 2024-04-24 22:54:16.80359005 +0000 UTC m=+0.466297696 container attach b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T22:54:17.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:16 smithi063 ceph-mon[111213]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T22:54:17.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:16 smithi063 ceph-mon[111213]: pgmap v158: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:17.324 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 podman[99638]: 2024-04-24 22:54:17.018743858 +0000 UTC m=+0.681451488 container died b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T22:54:17.924 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 podman[99664]: 2024-04-24 22:54:17.632399397 +0000 UTC m=+0.604798612 container remove b3c2ab2e4d9607c2e9f29ff788ff3948dbb4fc678cd380cc05bd6b983a8e9d44 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T22:54:17.924 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-24T22:54:17.925 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 systemd[1]: Stopped Ceph osd.5 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:54:17.925 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 systemd[1]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.5.service: Consumed 50.674s CPU time. 2024-04-24T22:54:17.925 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:17 smithi192 systemd[1]: Starting Ceph osd.5 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T22:54:18.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:17 smithi192 ceph-mon[80463]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T22:54:18.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:18.0834807 +0000 UTC m=+0.113451992 container create fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T22:54:18.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:17.988086987 +0000 UTC m=+0.018058245 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:18.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:17 smithi063 ceph-mon[111213]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T22:54:18.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:18.319442752 +0000 UTC m=+0.349414002 container init fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T22:54:18.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:18.322843808 +0000 UTC m=+0.352815056 container start fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T22:54:18.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:18.411126439 +0000 UTC m=+0.441097697 container attach fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T22:54:19.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-mon[80463]: pgmap v159: 65 pgs: 7 active+undersized+degraded, 3 peering, 3 stale+active+clean, 8 active+undersized, 44 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/264 objects degraded (6.439%) 2024-04-24T22:54:19.131 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 bash[99764]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T22:54:19.132 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T22:54:19.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T22:54:19.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T22:54:19.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:54:19.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate[99779]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T22:54:19.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:18 smithi192 podman[99764]: 2024-04-24 22:54:18.789401991 +0000 UTC m=+0.819373229 container died fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T22:54:19.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:18 smithi063 ceph-mon[111213]: pgmap v159: 65 pgs: 7 active+undersized+degraded, 3 peering, 3 stale+active+clean, 8 active+undersized, 44 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/264 objects degraded (6.439%) 2024-04-24T22:54:19.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 podman[99899]: 2024-04-24 22:54:19.414822704 +0000 UTC m=+0.620270066 container remove fadf693b47f38f15a416e23efa8aed763a2b339672a3aa16cfd496b4aa7b7e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T22:54:19.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 podman[99964]: 2024-04-24 22:54:19.532004296 +0000 UTC m=+0.018338015 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T22:54:19.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 podman[99964]: 2024-04-24 22:54:19.640775606 +0000 UTC m=+0.127109305 container create b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T22:54:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:19 smithi192 ceph-mon[80463]: Health check failed: Degraded data redundancy: 17/264 objects degraded (6.439%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:20.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 podman[99964]: 2024-04-24 22:54:19.834824156 +0000 UTC m=+0.321157855 container init b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T22:54:20.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 podman[99964]: 2024-04-24 22:54:19.837898359 +0000 UTC m=+0.324232059 container start b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T22:54:20.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 ceph-osd[99982]: -- 172.21.15.192:0/4271974527 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0b2971680 con 0x55d0b293b000 2024-04-24T22:54:20.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 bash[99964]: b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 2024-04-24T22:54:20.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:19 smithi192 systemd[1]: Started Ceph osd.5 for c8a2676e-028a-11ef-bc93-c7b262605968. 2024-04-24T22:54:20.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:19 smithi063 ceph-mon[111213]: Health check failed: Degraded data redundancy: 17/264 objects degraded (6.439%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:20.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:20 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T22:54:20.384+0000 7f45259d57c0 -1 Falling back to public interface 2024-04-24T22:54:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:20 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:20 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:20 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:21.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:20 smithi192 ceph-mon[80463]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:20 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:20 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:20 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:21.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:20 smithi063 ceph-mon[111213]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 3 peering, 12 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:22.493 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:22 smithi192 ceph-mon[80463]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T22:54:22.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:22 smithi063 ceph-mon[111213]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T22:54:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:23 smithi063 ceph-mon[111213]: pgmap v161: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:23 smithi192 ceph-mon[80463]: pgmap v161: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:24.897 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:24 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T22:54:24.568+0000 7f45259d57c0 -1 osd.5 74 log_to_monitors true 2024-04-24T22:54:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:25 smithi063 ceph-mon[111213]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:25 smithi063 ceph-mon[111213]: from='osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:54:25.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:25 smithi063 ceph-mon[111213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:54:25.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:25 smithi192 ceph-mon[80463]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:25.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:25.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:25.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:25 smithi192 ceph-mon[80463]: from='osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:54:25.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:25 smithi192 ceph-mon[80463]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T22:54:26.428 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T22:54:26.070+0000 7f4512e25700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:54:26.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-mon[80463]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:54:26.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-mon[80463]: from='osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:54:26.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-mon[80463]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T22:54:26.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-mon[80463]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:54:26.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:26 smithi192 ceph-mon[80463]: from='osd.5 ' entity='osd.5' 2024-04-24T22:54:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:26 smithi063 ceph-mon[111213]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T22:54:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:26 smithi063 ceph-mon[111213]: from='osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:54:26.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:26 smithi063 ceph-mon[111213]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T22:54:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:26 smithi063 ceph-mon[111213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-24T22:54:26.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:26 smithi063 ceph-mon[111213]: from='osd.5 ' entity='osd.5' 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210] boot 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: osdmap e78: 6 total, 6 up, 6 in 2024-04-24T22:54:27.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: osd.5 [v2:172.21.15.192:6816/3828556210,v1:172.21.15.192:6817/3828556210] boot 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: osdmap e78: 6 total, 6 up, 6 in 2024-04-24T22:54:27.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T22:54:28.911 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.911 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.911 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:28.911 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:54:28.911 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all osd 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T22:54:28.912 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T22:54:28.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:28 smithi063 ceph-mon[111213]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T22:54:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all osd 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T22:54:28.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T22:54:28.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:28 smithi192 ceph-mon[80463]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T22:54:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:29 smithi192 ceph-mon[80463]: pgmap v167: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 32/264 objects degraded (12.121%) 2024-04-24T22:54:30.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:29 smithi063 ceph-mon[111213]: pgmap v167: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 32/264 objects degraded (12.121%) 2024-04-24T22:54:31.261 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: Upgrade: Updating mds.cephfs.smithi063.twfoxu 2024-04-24T22:54:31.261 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: Deploying daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:31.262 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:30 smithi063 ceph-mon[111213]: pgmap v168: 65 pgs: 7 peering, 4 active+undersized+degraded, 10 active+undersized, 44 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12/264 objects degraded (4.545%) 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: Upgrade: Updating mds.cephfs.smithi063.twfoxu 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.twfoxu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: Deploying daemon mds.cephfs.smithi063.twfoxu on smithi063 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:31.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:30 smithi192 ceph-mon[80463]: pgmap v168: 65 pgs: 7 peering, 4 active+undersized+degraded, 10 active+undersized, 44 active+clean; 258 MiB data, 989 MiB used, 535 GiB / 536 GiB avail; 12/264 objects degraded (4.545%) 2024-04-24T22:54:32.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:32 smithi063 ceph-mon[111213]: Health check update: Degraded data redundancy: 12/264 objects degraded (4.545%), 4 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:32.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:32 smithi192 ceph-mon[80463]: Health check update: Degraded data redundancy: 12/264 objects degraded (4.545%), 4 pgs degraded (PG_DEGRADED) 2024-04-24T22:54:33.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:33 smithi192 ceph-mon[80463]: pgmap v169: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 993 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:33.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:33 smithi063 ceph-mon[111213]: pgmap v169: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 993 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:34.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:34 smithi063 ceph-mon[111213]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/264 objects degraded (4.545%), 4 pgs degraded) 2024-04-24T22:54:34.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:34 smithi063 ceph-mon[111213]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T22:54:34.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:34 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 2 up:standby 2024-04-24T22:54:34.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:34 smithi192 ceph-mon[80463]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/264 objects degraded (4.545%), 4 pgs degraded) 2024-04-24T22:54:34.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:34 smithi192 ceph-mon[80463]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T22:54:34.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:34 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 2 up:standby 2024-04-24T22:54:35.346 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:35 smithi063 ceph-mon[111213]: pgmap v171: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:35.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:35 smithi192 ceph-mon[80463]: pgmap v171: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:36.586 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:36.586 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:36.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:36.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:boot 2024-04-24T22:54:36.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:54:36.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:36 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:boot 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:54:36.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:36 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:54:37.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:37 smithi063 ceph-mon[111213]: pgmap v172: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 -- 172.21.15.63:0/192126283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4102670 msgr2=0x7ff8f40fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/192126283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4102670 0x7ff8f40fb600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004940 tx=0x7ff8e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 -- 172.21.15.63:0/192126283 shutdown_connections 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/192126283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4102670 0x7ff8f40fb600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/192126283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4101cd0 0x7ff8f41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.795+0000 7ff8f9af6700 1 -- 172.21.15.63:0/192126283 >> 172.21.15.63:0/192126283 conn(0x7ff8f4071460 msgr2=0x7ff8f4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.796+0000 7ff8f9af6700 1 -- 172.21.15.63:0/192126283 shutdown_connections 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.796+0000 7ff8f9af6700 1 -- 172.21.15.63:0/192126283 wait complete. 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.796+0000 7ff8f9af6700 1 Processor -- start 2024-04-24T22:54:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.796+0000 7ff8f9af6700 1 -- start start 2024-04-24T22:54:37.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f9af6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:37.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f9af6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 0x7ff8f40fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f9af6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40fd2b0 con 0x7ff8f4101cd0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f9af6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40fd3f0 con 0x7ff8f4102670 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47876/0 (socket says 172.21.15.63:47876) 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 -- 172.21.15.63:0/456863868 learned_addr learned my addr 172.21.15.63:0/456863868 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f2ffd700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 0x7ff8f40fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 -- 172.21.15.63:0/456863868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 msgr2=0x7ff8f40fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 0x7ff8f40fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 -- 172.21.15.63:0/456863868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8005040 con 0x7ff8f4101cd0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f2ffd700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 0x7ff8f40fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.797+0000 7ff8f37fe700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c8c0 tx=0x7ff8e40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.798+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e400a980 con 0x7ff8f4101cd0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.798+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e400ce60 con 0x7ff8f4101cd0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.798+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e40115e0 con 0x7ff8f4101cd0 2024-04-24T22:54:37.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.798+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4119eb0 con 0x7ff8f4101cd0 2024-04-24T22:54:37.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.798+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f411a2d0 con 0x7ff8f4101cd0 2024-04-24T22:54:37.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.799+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f4065200 con 0x7ff8f4101cd0 2024-04-24T22:54:37.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.800+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8e4012400 con 0x7ff8f4101cd0 2024-04-24T22:54:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.800+0000 7ff8dbfff700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 0x7ff8dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.800+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff8e408da80 con 0x7ff8f4101cd0 2024-04-24T22:54:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.800+0000 7ff8f2ffd700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 0x7ff8dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.801+0000 7ff8f2ffd700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 0x7ff8dc0706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40fe120 tx=0x7ff8e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:37.809+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8e4056f70 con 0x7ff8f4101cd0 2024-04-24T22:54:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:37 smithi192 ceph-mon[80463]: pgmap v172: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:38.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.042+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f40fded0 con 0x7ff8dc06e210 2024-04-24T22:54:38.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.043+0000 7ff8dbfff700 1 -- 172.21.15.63:0/456863868 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff8f40fded0 con 0x7ff8dc06e210 2024-04-24T22:54:38.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 msgr2=0x7ff8dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 0x7ff8dc0706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40fe120 tx=0x7ff8e8009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 msgr2=0x7ff8f40fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c8c0 tx=0x7ff8e40044c0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 shutdown_connections 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff8dc06e210 0x7ff8dc0706c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8f4101cd0 0x7ff8f40fc7e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 --2- 172.21.15.63:0/456863868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f4102670 0x7ff8f40fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.045+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 >> 172.21.15.63:0/456863868 conn(0x7ff8f4071460 msgr2=0x7ff8f41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.046+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 shutdown_connections 2024-04-24T22:54:38.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.046+0000 7ff8f9af6700 1 -- 172.21.15.63:0/456863868 wait complete. 2024-04-24T22:54:38.057 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- 172.21.15.63:0/1648295994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc0fbd40 msgr2=0x7f36fc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 --2- 172.21.15.63:0/1648295994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc0fbd40 0x7f36fc0fc1b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36e4004940 tx=0x7f36e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- 172.21.15.63:0/1648295994 shutdown_connections 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 --2- 172.21.15.63:0/1648295994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc0fbd40 0x7f36fc0fc1b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 --2- 172.21.15.63:0/1648295994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc101d90 0x7f36fc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- 172.21.15.63:0/1648295994 >> 172.21.15.63:0/1648295994 conn(0x7f36fc071d00 msgr2=0x7f36fc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- 172.21.15.63:0/1648295994 shutdown_connections 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- 172.21.15.63:0/1648295994 wait complete. 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 Processor -- start 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.152+0000 7f3702b63700 1 -- start start 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f3702b63700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc0fbd40 0x7f36fc113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f3702b63700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f3702b63700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc114740 con 0x7f36fc101d90 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f3702b63700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc10db30 con 0x7f36fc0fbd40 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47886/0 (socket says 172.21.15.63:47886) 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 -- 172.21.15.63:0/2807822011 learned_addr learned my addr 172.21.15.63:0/2807822011 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f37008ff700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc0fbd40 0x7f36fc113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 -- 172.21.15.63:0/2807822011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc0fbd40 msgr2=0x7f36fc113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc0fbd40 0x7f36fc113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 -- 172.21.15.63:0/2807822011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4005040 con 0x7f36fc101d90 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36fbfff700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009180 tx=0x7f36e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.153+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e400e040 con 0x7f36fc101d90 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.154+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36e400bc90 con 0x7f36fc101d90 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.154+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc10ddb0 con 0x7f36fc101d90 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.154+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e401c630 con 0x7f36fc101d90 2024-04-24T22:54:38.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.154+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc10e370 con 0x7f36fc101d90 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.155+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc0fd430 con 0x7f36fc101d90 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.155+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36e4013980 con 0x7f36fc101d90 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.155+0000 7f36f97fa700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 0x7f36e8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.156+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f36e408ef40 con 0x7f36fc101d90 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.156+0000 7f37008ff700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 0x7f36e8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.156+0000 7f37008ff700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 0x7f36e8070600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004ec0 tx=0x7f36f4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.160+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f36e40c3740 con 0x7f36fc101d90 2024-04-24T22:54:38.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.404+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36fc108df0 con 0x7f36e806e150 2024-04-24T22:54:38.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.405+0000 7f36f97fa700 1 -- 172.21.15.63:0/2807822011 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f36fc108df0 con 0x7f36e806e150 2024-04-24T22:54:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 msgr2=0x7f36e8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 0x7f36e8070600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004ec0 tx=0x7f36f4009040 comp rx=0 tx=0).stop 2024-04-24T22:54:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 msgr2=0x7f36fc1140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009180 tx=0x7f36e4009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 shutdown_connections 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36fc0fbd40 0x7f36fc113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f36e806e150 0x7f36e8070600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 --2- 172.21.15.63:0/2807822011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36fc101d90 0x7f36fc1140f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 >> 172.21.15.63:0/2807822011 conn(0x7f36fc071d00 msgr2=0x7f36fc0655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.407+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 shutdown_connections 2024-04-24T22:54:38.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.408+0000 7f3702b63700 1 -- 172.21.15.63:0/2807822011 wait complete. 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- 172.21.15.63:0/14500846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fe7e0 msgr2=0x7f168c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 --2- 172.21.15.63:0/14500846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fe7e0 0x7f168c106e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1678004940 tx=0x7f1678009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- 172.21.15.63:0/14500846 shutdown_connections 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 --2- 172.21.15.63:0/14500846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fe7e0 0x7f168c106e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 --2- 172.21.15.63:0/14500846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fde40 0x7f168c0fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- 172.21.15.63:0/14500846 >> 172.21.15.63:0/14500846 conn(0x7f168c0f7850 msgr2=0x7f168c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- 172.21.15.63:0/14500846 shutdown_connections 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- 172.21.15.63:0/14500846 wait complete. 2024-04-24T22:54:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 Processor -- start 2024-04-24T22:54:38.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.513+0000 7f1692fe8700 1 -- start start 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1692fe8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1692fe8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fe7e0 0x7f168c113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1692fe8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c1145a0 con 0x7f168c0fde40 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1692fe8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c10d990 con 0x7f168c0fe7e0 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1690d84700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1690d84700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47902/0 (socket says 172.21.15.63:47902) 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f1690d84700 1 -- 172.21.15.63:0/1842277558 learned_addr learned my addr 172.21.15.63:0/1842277558 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:38.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.514+0000 7f168bfff700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fe7e0 0x7f168c113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1690d84700 1 -- 172.21.15.63:0/1842277558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fe7e0 msgr2=0x7f168c113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1690d84700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fe7e0 0x7f168c113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1690d84700 1 -- 172.21.15.63:0/1842277558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678005040 con 0x7f168c0fde40 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1690d84700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1680002a80 tx=0x7f16800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1680007cc0 con 0x7f168c0fde40 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f168000f040 con 0x7f168c0fde40 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c10dc70 con 0x7f168c0fde40 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168000dac0 con 0x7f168c0fde40 2024-04-24T22:54:38.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.515+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c10e1b0 con 0x7f168c0fde40 2024-04-24T22:54:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.516+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c0666f0 con 0x7f168c0fde40 2024-04-24T22:54:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.517+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f168000dc20 con 0x7f168c0fde40 2024-04-24T22:54:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.517+0000 7f16897fa700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 0x7f167c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.518+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f168008d870 con 0x7f168c0fde40 2024-04-24T22:54:38.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.518+0000 7f168bfff700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 0x7f167c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.518+0000 7f168bfff700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 0x7f167c070600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f168c10eb50 tx=0x7f1678009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.523+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f168005a770 con 0x7f168c0fde40 2024-04-24T22:54:38.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.754+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f168c109e00 con 0x7f167c06e150 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.764+0000 7f16897fa700 1 -- 172.21.15.63:0/1842277558 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f168c109e00 con 0x7f167c06e150 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 116s ago 17m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 116s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (3m) 15s ago 16m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 116s ago 17m 81.1M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (12m) 116s ago 12m 15.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1b90bc89046f 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 starting - - - - 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (12m) 15s ago 12m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:54:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (12m) 15s ago 12m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (7m) 116s ago 19m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (7m) 15s ago 16m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 116s ago 19m 47.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (4m) 15s ago 16m 39.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 116s ago 17m 15.9M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 15s ago 16m 17.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 116s ago 15m 91.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 116s ago 15m 91.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 116s ago 14m 12.0M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (86s) 15s ago 14m 90.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (52s) 15s ago 14m 86.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (19s) 15s ago 13m 12.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef b7243340f774 2024-04-24T22:54:38.767 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 116s ago 17m 61.2M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:54:38.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 msgr2=0x7f167c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 0x7f167c070600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f168c10eb50 tx=0x7f1678009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 msgr2=0x7f168c113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1680002a80 tx=0x7f16800046b0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 shutdown_connections 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f167c06e150 0x7f167c070600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c0fde40 0x7f168c113a10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 --2- 172.21.15.63:0/1842277558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f168c0fe7e0 0x7f168c113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 >> 172.21.15.63:0/1842277558 conn(0x7f168c0f7850 msgr2=0x7f168c0f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 shutdown_connections 2024-04-24T22:54:38.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.767+0000 7f1692fe8700 1 -- 172.21.15.63:0/1842277558 wait complete. 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 -- 172.21.15.63:0/2180391698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 msgr2=0x7f72480fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 --2- 172.21.15.63:0/2180391698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f72480fb970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f723c004940 tx=0x7f723c009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 -- 172.21.15.63:0/2180391698 shutdown_connections 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 --2- 172.21.15.63:0/2180391698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 0x7f7248101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 --2- 172.21.15.63:0/2180391698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f72480fb970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 -- 172.21.15.63:0/2180391698 >> 172.21.15.63:0/2180391698 conn(0x7f7248073350 msgr2=0x7f7248073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 -- 172.21.15.63:0/2180391698 shutdown_connections 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.870+0000 7f724f036700 1 -- 172.21.15.63:0/2180391698 wait complete. 2024-04-24T22:54:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.871+0000 7f724f036700 1 Processor -- start 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.871+0000 7f724f036700 1 -- start start 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.871+0000 7f724f036700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.871+0000 7f724f036700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 0x7f7248113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.871+0000 7f724f036700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248114510 con 0x7f72480fb5a0 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724f036700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810db30 con 0x7f72480fbf40 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34468/0 (socket says 172.21.15.63:34468) 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 -- 172.21.15.63:0/4080629275 learned_addr learned my addr 172.21.15.63:0/4080629275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f7247fff700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 0x7f7248113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 -- 172.21.15.63:0/4080629275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 msgr2=0x7f7248113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 0x7f7248113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 -- 172.21.15.63:0/4080629275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c005040 con 0x7f72480fb5a0 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724cdd2700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b3b0 tx=0x7f723c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f723c00e040 con 0x7f72480fb5a0 2024-04-24T22:54:38.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724810ddb0 con 0x7f72480fb5a0 2024-04-24T22:54:38.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f723c013420 con 0x7f72480fb5a0 2024-04-24T22:54:38.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f723c01c630 con 0x7f72480fb5a0 2024-04-24T22:54:38.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.872+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724810e2f0 con 0x7f72480fb5a0 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.873+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72480fe670 con 0x7f72480fb5a0 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.874+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f723c013590 con 0x7f72480fb5a0 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.874+0000 7f72457fa700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 0x7f72380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.874+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f723c08ecd0 con 0x7f72480fb5a0 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.874+0000 7f7247fff700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 0x7f72380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:38.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.875+0000 7f7247fff700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 0x7f72380705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724810f220 tx=0x7f723400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:38.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:38.879+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f723c058120 con 0x7f72480fb5a0 2024-04-24T22:54:39.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.195+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7248109f70 con 0x7f72480fb5a0 2024-04-24T22:54:39.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.196+0000 7f72457fa700 1 -- 172.21.15.63:0/4080629275 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f723c05b7e0 con 0x7f72480fb5a0 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:54:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T22:54:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 11 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:54:39.203 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 msgr2=0x7f72380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 0x7f72380705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724810f220 tx=0x7f723400b040 comp rx=0 tx=0).stop 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 msgr2=0x7f7248113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b3b0 tx=0x7f723c009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 shutdown_connections 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f723806e140 0x7f72380705f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72480fb5a0 0x7f7248113980 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 --2- 172.21.15.63:0/4080629275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72480fbf40 0x7f7248113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 >> 172.21.15.63:0/4080629275 conn(0x7f7248073350 msgr2=0x7f7248100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 shutdown_connections 2024-04-24T22:54:39.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.199+0000 7f724f036700 1 -- 172.21.15.63:0/4080629275 wait complete. 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- 172.21.15.63:0/2659839902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a8101d10 msgr2=0x7f58a81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 --2- 172.21.15.63:0/2659839902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a8101d10 0x7f58a81020e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5898004940 tx=0x7f5898009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- 172.21.15.63:0/2659839902 shutdown_connections 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 --2- 172.21.15.63:0/2659839902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a81026b0 0x7f58a80fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 --2- 172.21.15.63:0/2659839902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a8101d10 0x7f58a81020e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- 172.21.15.63:0/2659839902 >> 172.21.15.63:0/2659839902 conn(0x7f58a80f7860 msgr2=0x7f58a80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- 172.21.15.63:0/2659839902 shutdown_connections 2024-04-24T22:54:39.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- 172.21.15.63:0/2659839902 wait complete. 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 Processor -- start 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.311+0000 7f58ace79700 1 -- start start 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58ace79700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a8101d10 0x7f58a81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58ace79700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58ace79700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8114530 con 0x7f58a81026b0 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58ace79700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a810db50 con 0x7f58a8101d10 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34486/0 (socket says 172.21.15.63:34486) 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 -- 172.21.15.63:0/1822522878 learned_addr learned my addr 172.21.15.63:0/1822522878 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a659c700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a8101d10 0x7f58a81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 -- 172.21.15.63:0/1822522878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a8101d10 msgr2=0x7f58a81139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a8101d10 0x7f58a81139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 -- 172.21.15.63:0/1822522878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898005040 con 0x7f58a81026b0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58a5d9b700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f589c002a50 tx=0x7f589c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589c007cc0 con 0x7f58a81026b0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a810de30 con 0x7f58a81026b0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.312+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f589c00f040 con 0x7f58a81026b0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.313+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589c00d9c0 con 0x7f58a81026b0 2024-04-24T22:54:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.313+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a810e3f0 con 0x7f58a81026b0 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.313+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a80fd460 con 0x7f58a81026b0 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.314+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f589c009730 con 0x7f58a81026b0 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.314+0000 7f5896ffd700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 0x7f58900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.314+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f589c08d850 con 0x7f58a81026b0 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.314+0000 7f58a659c700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 0x7f58900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.315+0000 7f58a659c700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 0x7f58900705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f589800b3b0 tx=0x7f5898009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:39.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.319+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f589c056c20 con 0x7f58a81026b0 2024-04-24T22:54:39.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.576+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58a8108e10 con 0x7f58a81026b0 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.576+0000 7f5896ffd700 1 -- 172.21.15.63:0/1822522878 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f589c05a2e0 con 0x7f58a81026b0 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:e16 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 14 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:54:39.578 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:42:40.679489+0000 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 46 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:54:39.579 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24253} 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{0:24253} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.192:6824/780449385,v1:172.21.15.192:6825/780449385] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:39.580 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:54:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:54:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/3016441784,v1:172.21.15.63:6829/3016441784] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:54:39.581 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 msgr2=0x7f58900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 0x7f58900705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f589800b3b0 tx=0x7f5898009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 msgr2=0x7f58a8113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f589c002a50 tx=0x7f589c004600 comp rx=0 tx=0).stop 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 shutdown_connections 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58a8101d10 0x7f58a81139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f589006e140 0x7f58900705f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 --2- 172.21.15.63:0/1822522878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a81026b0 0x7f58a8113ee0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 >> 172.21.15.63:0/1822522878 conn(0x7f58a80f7860 msgr2=0x7f58a80662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 shutdown_connections 2024-04-24T22:54:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.578+0000 7f58ace79700 1 -- 172.21.15.63:0/1822522878 wait complete. 2024-04-24T22:54:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 16 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 -- 172.21.15.63:0/1613865933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 msgr2=0x7f1c94065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 --2- 172.21.15.63:0/1613865933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c94065ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c88004940 tx=0x7f1c88009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 -- 172.21.15.63:0/1613865933 shutdown_connections 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 --2- 172.21.15.63:0/1613865933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c94065ab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 --2- 172.21.15.63:0/1613865933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 0x7f1c94103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 -- 172.21.15.63:0/1613865933 >> 172.21.15.63:0/1613865933 conn(0x7f1c94071d00 msgr2=0x7f1c94072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:39.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 -- 172.21.15.63:0/1613865933 shutdown_connections 2024-04-24T22:54:39.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.684+0000 7f1c99f32700 1 -- 172.21.15.63:0/1613865933 wait complete. 2024-04-24T22:54:39.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 Processor -- start 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 -- start start 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 0x7f1c940fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940fc880 con 0x7f1c94065640 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c99f32700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940fc9c0 con 0x7f1c94102e50 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34512/0 (socket says 172.21.15.63:34512) 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c937fe700 1 -- 172.21.15.63:0/2184499709 learned_addr learned my addr 172.21.15.63:0/2184499709 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.685+0000 7f1c92ffd700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 0x7f1c940fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c937fe700 1 -- 172.21.15.63:0/2184499709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 msgr2=0x7f1c940fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c937fe700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 0x7f1c940fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c937fe700 1 -- 172.21.15.63:0/2184499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c88005040 con 0x7f1c94065640 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c937fe700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c84002a50 tx=0x7f1c840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c84007cc0 con 0x7f1c94065640 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c8400f040 con 0x7f1c94065640 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94119e90 con 0x7f1c94065640 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8400da70 con 0x7f1c94065640 2024-04-24T22:54:39.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.686+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9411a310 con 0x7f1c94065640 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.687+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c94100100 con 0x7f1c94065640 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.688+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1c8400dc10 con 0x7f1c94065640 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.688+0000 7f1c7bfff700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 0x7f1c7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.688+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1c8408d120 con 0x7f1c94065640 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.689+0000 7f1c92ffd700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 0x7f1c7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:39.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.689+0000 7f1c92ffd700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 0x7f1c7c0705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c940fd6f0 tx=0x7f1c88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.694+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1c840564f0 con 0x7f1c94065640 2024-04-24T22:54:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:39 smithi063 ceph-mon[111213]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:39 smithi063 ceph-mon[111213]: pgmap v173: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:39 smithi063 ceph-mon[111213]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:39 smithi063 ceph-mon[111213]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:39 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/4080629275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:39 smithi192 ceph-mon[80463]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:39 smithi192 ceph-mon[80463]: pgmap v173: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:39 smithi192 ceph-mon[80463]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:39 smithi192 ceph-mon[80463]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:39 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/4080629275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.967+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c9404bed0 con 0x7f1c7c06e140 2024-04-24T22:54:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.968+0000 7f1c7bfff700 1 -- 172.21.15.63:0/2184499709 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1c9404bed0 con 0x7f1c7c06e140 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "12/21 daemons upgraded", 2024-04-24T22:54:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T22:54:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:54:39.971 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.970+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 msgr2=0x7f1c7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.970+0000 7f1c99f32700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 0x7f1c7c0705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c940fd6f0 tx=0x7f1c88009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.970+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 msgr2=0x7f1c940fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.970+0000 7f1c99f32700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c84002a50 tx=0x7f1c840046b0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 shutdown_connections 2024-04-24T22:54:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1c7c06e140 0x7f1c7c0705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c94065640 0x7f1c940fbdb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 --2- 172.21.15.63:0/2184499709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c94102e50 0x7f1c940fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:39.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 >> 172.21.15.63:0/2184499709 conn(0x7f1c94071d00 msgr2=0x7f1c940faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:39.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 shutdown_connections 2024-04-24T22:54:39.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:39.971+0000 7f1c99f32700 1 -- 172.21.15.63:0/2184499709 wait complete. 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 -- 172.21.15.63:0/1778493535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c8101d90 msgr2=0x7fd4c8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/1778493535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c8101d90 0x7fd4c8102160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc004940 tx=0x7fd4bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 -- 172.21.15.63:0/1778493535 shutdown_connections 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/1778493535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c80fbd40 0x7fd4c80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/1778493535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c8101d90 0x7fd4c8102160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 -- 172.21.15.63:0/1778493535 >> 172.21.15.63:0/1778493535 conn(0x7fd4c8071d00 msgr2=0x7fd4c8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 -- 172.21.15.63:0/1778493535 shutdown_connections 2024-04-24T22:54:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.077+0000 7fd4cef0c700 1 -- 172.21.15.63:0/1778493535 wait complete. 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 Processor -- start 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 -- start start 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c8101d90 0x7fd4c810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c810f720 con 0x7fd4c80fbd40 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4cef0c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c810f860 con 0x7fd4c8101d90 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4ccca8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4ccca8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34526/0 (socket says 172.21.15.63:34526) 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4ccca8700 1 -- 172.21.15.63:0/52072101 learned_addr learned my addr 172.21.15.63:0/52072101 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.078+0000 7fd4c7fff700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c8101d90 0x7fd4c810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4ccca8700 1 -- 172.21.15.63:0/52072101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c8101d90 msgr2=0x7fd4c810f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4ccca8700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c8101d90 0x7fd4c810f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4ccca8700 1 -- 172.21.15.63:0/52072101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc005040 con 0x7fd4c80fbd40 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4ccca8700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009150 tx=0x7fd4bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc00e040 con 0x7fd4c80fbd40 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4bc00bbc0 con 0x7fd4c80fbd40 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c8117b50 con 0x7fd4c80fbd40 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc01c630 con 0x7fd4c80fbd40 2024-04-24T22:54:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.079+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c81180b0 con 0x7fd4c80fbd40 2024-04-24T22:54:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.080+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c80fd430 con 0x7fd4c80fbd40 2024-04-24T22:54:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.081+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd4bc00bd30 con 0x7fd4c80fbd40 2024-04-24T22:54:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.081+0000 7fd4c57fa700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 0x7fd4b8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:54:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.081+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd4bc08ec30 con 0x7fd4c80fbd40 2024-04-24T22:54:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.081+0000 7fd4c7fff700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 0x7fd4b8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:54:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.082+0000 7fd4c7fff700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 0x7fd4b8070600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4007fd0 tx=0x7fd4b4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:54:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.086+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd4bc058120 con 0x7fd4c80fbd40 2024-04-24T22:54:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.382+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4c804bed0 con 0x7fd4c80fbd40 2024-04-24T22:54:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.383+0000 7fd4c57fa700 1 -- 172.21.15.63:0/52072101 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd4bc05b7e0 con 0x7fd4c80fbd40 2024-04-24T22:54:40.384 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:54:40.384 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:54:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 msgr2=0x7fd4b8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 0x7fd4b8070600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4007fd0 tx=0x7fd4b4009040 comp rx=0 tx=0).stop 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 msgr2=0x7fd4c810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009150 tx=0x7fd4bc009e30 comp rx=0 tx=0).stop 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 shutdown_connections 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd4b806e150 0x7fd4b8070600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4c80fbd40 0x7fd4c810eca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 --2- 172.21.15.63:0/52072101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c8101d90 0x7fd4c810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 >> 172.21.15.63:0/52072101 conn(0x7fd4c8071d00 msgr2=0x7fd4c80fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 shutdown_connections 2024-04-24T22:54:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:54:40.385+0000 7fd4cef0c700 1 -- 172.21.15.63:0/52072101 wait complete. 2024-04-24T22:54:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:40 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1822522878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:54:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:40 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:40.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:40 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/52072101' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:54:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:40 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1822522878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:54:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:40 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:40.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:40 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/52072101' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:54:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:41 smithi063 ceph-mon[111213]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:41 smithi063 ceph-mon[111213]: pgmap v174: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:41 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:41 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:42.096 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:41 smithi192 ceph-mon[80463]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:54:42.096 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:41 smithi192 ceph-mon[80463]: pgmap v174: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:42.096 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:41 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:42.096 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:41 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:42 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:43.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:42 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:43.218 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:42 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:43.219 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:42 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:44.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:43 smithi192 ceph-mon[80463]: pgmap v175: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:44.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:43 smithi063 ceph-mon[111213]: pgmap v175: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:45.049 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:44 smithi063 ceph-mon[111213]: pgmap v176: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:44 smithi192 ceph-mon[80463]: pgmap v176: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:54:46.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:46.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:46 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:46.437 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:46 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:54:47.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:47 smithi192 ceph-mon[80463]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:47.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:47 smithi063 ceph-mon[111213]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:48 smithi063 ceph-mon[111213]: Upgrade: Updating mds.cephfs.smithi063.dnliiv 2024-04-24T22:54:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:54:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:48 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:48.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:48 smithi063 ceph-mon[111213]: Deploying daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:54:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:48 smithi192 ceph-mon[80463]: Upgrade: Updating mds.cephfs.smithi063.dnliiv 2024-04-24T22:54:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.dnliiv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:54:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:48 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:54:48.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:48 smithi192 ceph-mon[80463]: Deploying daemon mds.cephfs.smithi063.dnliiv on smithi063 2024-04-24T22:54:49.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:49 smithi063 ceph-mon[111213]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:49.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:49 smithi192 ceph-mon[80463]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:51.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:51 smithi192 ceph-mon[80463]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T22:54:51.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:51 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 2 up:standby 2024-04-24T22:54:51.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:51 smithi192 ceph-mon[80463]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:51.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:54:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:51 smithi063 ceph-mon[111213]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T22:54:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:51 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 2 up:standby 2024-04-24T22:54:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:51 smithi063 ceph-mon[111213]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:54:53.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:53 smithi192 ceph-mon[80463]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:53 smithi063 ceph-mon[111213]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:55 smithi063 ceph-mon[111213]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:55 smithi192 ceph-mon[80463]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:54:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:54:56.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:56 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.63:6828/3118675914,v1:172.21.15.63:6829/3118675914] up:boot 2024-04-24T22:54:56.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:56 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:54:56.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:56 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:54:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:56 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.63:6828/3118675914,v1:172.21.15.63:6829/3118675914] up:boot 2024-04-24T22:54:56.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:56 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.mnfuxk=up:active} 3 up:standby 2024-04-24T22:54:56.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:56 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:54:57.689 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:57 smithi063 ceph-mon[111213]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:57.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:57 smithi192 ceph-mon[80463]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:54:59 smithi063 ceph-mon[111213]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:54:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:54:59 smithi192 ceph-mon[80463]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:01 smithi192 ceph-mon[80463]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:01 smithi063 ceph-mon[111213]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:03 smithi063 ceph-mon[111213]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:03 smithi192 ceph-mon[80463]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:05.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:05 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:05.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:05 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:07.297 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: Upgrade: Updating mds.cephfs.smithi192.mnfuxk 2024-04-24T22:55:07.297 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:07.298 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:55:07.298 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:07.298 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: Deploying daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:55:07.298 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:07 smithi192 ceph-mon[80463]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: Upgrade: Updating mds.cephfs.smithi192.mnfuxk 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.mnfuxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: Deploying daemon mds.cephfs.smithi192.mnfuxk on smithi192 2024-04-24T22:55:07.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:07 smithi063 ceph-mon[111213]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:55:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T22:55:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: Standby daemon mds.cephfs.smithi192.ozemqi assigned to filesystem cephfs as rank 0 2024-04-24T22:55:08.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:55:08.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T22:55:08.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:08 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:replay} 2 up:standby 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[111209]: 2024-04-24T22:55:08.099+0000 7fcca103f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: Standby daemon mds.cephfs.smithi192.ozemqi assigned to filesystem cephfs as rank 0 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T22:55:08.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:08 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:replay} 2 up:standby 2024-04-24T22:55:09.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:09 smithi063 ceph-mon[111213]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:09.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:09 smithi192 ceph-mon[80463]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- 172.21.15.63:0/539574005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 msgr2=0x7fb284103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 --2- 172.21.15.63:0/539574005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284103190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb278004940 tx=0x7fb278009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- 172.21.15.63:0/539574005 shutdown_connections 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 --2- 172.21.15.63:0/539574005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 0x7fb2840fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 --2- 172.21.15.63:0/539574005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284103190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- 172.21.15.63:0/539574005 >> 172.21.15.63:0/539574005 conn(0x7fb2840f8950 msgr2=0x7fb2840fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- 172.21.15.63:0/539574005 shutdown_connections 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- 172.21.15.63:0/539574005 wait complete. 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 Processor -- start 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.508+0000 7fb28ae59700 1 -- start start 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb28ae59700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb28ae59700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 0x7fb284114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb28ae59700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284114670 con 0x7fb284102dc0 2024-04-24T22:55:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb28ae59700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28410da60 con 0x7fb284103760 2024-04-24T22:55:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36784/0 (socket says 172.21.15.63:36784) 2024-04-24T22:55:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 -- 172.21.15.63:0/1912224650 learned_addr learned my addr 172.21.15.63:0/1912224650 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb283fff700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 0x7fb284114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 -- 172.21.15.63:0/1912224650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 msgr2=0x7fb284114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 0x7fb284114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.509+0000 7fb288bf5700 1 -- 172.21.15.63:0/1912224650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278005040 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb288bf5700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb278009b30 tx=0x7fb278009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb278021070 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb278013c00 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28410dce0 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb27800e070 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.510+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28410e1c0 con 0x7fb284102dc0 2024-04-24T22:55:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.511+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2840fe510 con 0x7fb284102dc0 2024-04-24T22:55:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.512+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb27801cad0 con 0x7fb284102dc0 2024-04-24T22:55:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.512+0000 7fb2817fa700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 0x7fb2740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.512+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb27808ef50 con 0x7fb284102dc0 2024-04-24T22:55:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.512+0000 7fb283fff700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 0x7fb2740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.513+0000 7fb283fff700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 0x7fb2740705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb28410ec20 tx=0x7fb27000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.518+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb2780c3740 con 0x7fb284102dc0 2024-04-24T22:55:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.744+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb284109f30 con 0x7fb27406e140 2024-04-24T22:55:10.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.746+0000 7fb2817fa700 1 -- 172.21.15.63:0/1912224650 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb284109f30 con 0x7fb27406e140 2024-04-24T22:55:10.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 msgr2=0x7fb2740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 0x7fb2740705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb28410ec20 tx=0x7fb27000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 msgr2=0x7fb284113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb278009b30 tx=0x7fb278009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 shutdown_connections 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb27406e140 0x7fb2740705f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb284102dc0 0x7fb284113ae0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 --2- 172.21.15.63:0/1912224650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284103760 0x7fb284114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 >> 172.21.15.63:0/1912224650 conn(0x7fb2840f8950 msgr2=0x7fb2840f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 shutdown_connections 2024-04-24T22:55:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.748+0000 7fb28ae59700 1 -- 172.21.15.63:0/1912224650 wait complete. 2024-04-24T22:55:10.760 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 -- 172.21.15.63:0/1513669508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c102c90 msgr2=0x7f500c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 --2- 172.21.15.63:0/1513669508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c102c90 0x7f500c103060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5004004940 tx=0x7f5004009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 -- 172.21.15.63:0/1513669508 shutdown_connections 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 --2- 172.21.15.63:0/1513669508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c065640 0x7f500c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 --2- 172.21.15.63:0/1513669508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c102c90 0x7f500c103060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 -- 172.21.15.63:0/1513669508 >> 172.21.15.63:0/1513669508 conn(0x7f500c071b40 msgr2=0x7f500c071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 -- 172.21.15.63:0/1513669508 shutdown_connections 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.855+0000 7f5012f9d700 1 -- 172.21.15.63:0/1513669508 wait complete. 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 Processor -- start 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 -- start start 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c102c90 0x7f500c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c114580 con 0x7f500c065640 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5012f9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c10d970 con 0x7f500c102c90 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36796/0 (socket says 172.21.15.63:36796) 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 -- 172.21.15.63:0/2099119927 learned_addr learned my addr 172.21.15.63:0/2099119927 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f501179a700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c102c90 0x7f500c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 -- 172.21.15.63:0/2099119927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c102c90 msgr2=0x7f500c113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:10.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c102c90 0x7f500c113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.856+0000 7f5011f9b700 1 -- 172.21.15.63:0/2099119927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004005040 con 0x7f500c065640 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f5011f9b700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f500400b3b0 tx=0x7f5004009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500400e040 con 0x7f500c065640 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c10dbf0 con 0x7f500c065640 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5004013420 con 0x7f500c065640 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500401c630 con 0x7f500c065640 2024-04-24T22:55:10.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.857+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c10e1b0 con 0x7f500c065640 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.858+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c0fe380 con 0x7f500c065640 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.858+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5004013590 con 0x7f500c065640 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.859+0000 7f500a7fc700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 0x7f4ff40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.859+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f500408ed20 con 0x7f500c065640 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.859+0000 7f501179a700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 0x7f4ff40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:10.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.859+0000 7f501179a700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 0x7f4ff40705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f500c10eb30 tx=0x7f500000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:10.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:10.864+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f500405bc50 con 0x7f500c065640 2024-04-24T22:55:11.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.089+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f500c109d30 con 0x7f4ff406e140 2024-04-24T22:55:11.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.091+0000 7f500a7fc700 1 -- 172.21.15.63:0/2099119927 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f500c109d30 con 0x7f4ff406e140 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 msgr2=0x7f4ff40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 0x7f4ff40705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f500c10eb30 tx=0x7f500000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 msgr2=0x7f500c1139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f500400b3b0 tx=0x7f5004009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 shutdown_connections 2024-04-24T22:55:11.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f4ff406e140 0x7f4ff40705f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c065640 0x7f500c1139f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 --2- 172.21.15.63:0/2099119927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f500c102c90 0x7f500c113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 >> 172.21.15.63:0/2099119927 conn(0x7f500c071b40 msgr2=0x7f500c0f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 shutdown_connections 2024-04-24T22:55:11.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.093+0000 7f5012f9d700 1 -- 172.21.15.63:0/2099119927 wait complete. 2024-04-24T22:55:11.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1427900218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a00fc4d0 msgr2=0x7f79a00fc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1427900218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a00fc4d0 0x7f79a00fc940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7990004940 tx=0x7f7990009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1427900218 shutdown_connections 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1427900218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a00fc4d0 0x7f79a00fc940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1427900218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a0101d30 0x7f79a0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1427900218 >> 172.21.15.63:0/1427900218 conn(0x7f79a0072380 msgr2=0x7f79a0072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1427900218 shutdown_connections 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1427900218 wait complete. 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 Processor -- start 2024-04-24T22:55:11.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.198+0000 7f79a5cd7700 1 -- start start 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f79a5cd7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a00fc4d0 0x7f79a01138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f79a5cd7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f79a5cd7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0114460 con 0x7f79a0101d30 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f79a5cd7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a010dab0 con 0x7f79a00fc4d0 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36822/0 (socket says 172.21.15.63:36822) 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 -- 172.21.15.63:0/1038953200 learned_addr learned my addr 172.21.15.63:0/1038953200 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799f7fe700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a00fc4d0 0x7f79a01138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 -- 172.21.15.63:0/1038953200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a00fc4d0 msgr2=0x7f79a01138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a00fc4d0 0x7f79a01138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 -- 172.21.15.63:0/1038953200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7990005040 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f799effd700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7990009b30 tx=0x7f7990009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7990021070 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a010dd30 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7990013c00 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.199+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799000e070 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.200+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a010e210 con 0x7f79a0101d30 2024-04-24T22:55:11.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.200+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a0100050 con 0x7f79a0101d30 2024-04-24T22:55:11.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.201+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f799001cad0 con 0x7f79a0101d30 2024-04-24T22:55:11.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.201+0000 7f7987fff700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 0x7f79880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.202+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f799008ef40 con 0x7f79a0101d30 2024-04-24T22:55:11.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.202+0000 7f799f7fe700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 0x7f79880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.202+0000 7f799f7fe700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 0x7f79880705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7994009b80 tx=0x7f799400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.206+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f79900c3740 con 0x7f79a0101d30 2024-04-24T22:55:11.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.424+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79a010ef90 con 0x7f798806e140 2024-04-24T22:55:11.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.435+0000 7f7987fff700 1 -- 172.21.15.63:0/1038953200 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f79a010ef90 con 0x7f798806e140 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 11s ago 18m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 11s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (4m) 48s ago 17m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 11s ago 17m 79.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (16s) 11s ago 12m 11.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 091ce07d1d7d 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (36s) 11s ago 12m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea388604c60e 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (12m) 48s ago 12m 150M - 17.2.7-902-gc61472c8 0de407c1b3af bda827e19883 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (12m) 48s ago 12m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af c7490acb630c 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (8m) 11s ago 19m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:55:11.437 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (7m) 48s ago 17m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 11s ago 19m 54.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (4m) 48s ago 17m 39.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 11s ago 18m 15.8M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 48s ago 17m 17.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 11s ago 16m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 11s ago 15m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 11s ago 15m 76.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (118s) 48s ago 15m 90.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (85s) 48s ago 14m 86.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (51s) 48s ago 14m 12.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef b7243340f774 2024-04-24T22:55:11.438 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 11s ago 17m 61.3M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 msgr2=0x7f79880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 0x7f79880705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7994009b80 tx=0x7f799400b040 comp rx=0 tx=0).stop 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 msgr2=0x7f79a0113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7990009b30 tx=0x7f7990009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 shutdown_connections 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a00fc4d0 0x7f79a01138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f798806e140 0x7f79880705f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 --2- 172.21.15.63:0/1038953200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79a0101d30 0x7f79a0113e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 >> 172.21.15.63:0/1038953200 conn(0x7f79a0072380 msgr2=0x7f79a01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 shutdown_connections 2024-04-24T22:55:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.437+0000 7f79a5cd7700 1 -- 172.21.15.63:0/1038953200 wait complete. 2024-04-24T22:55:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.543+0000 7f31b4d92700 1 -- 172.21.15.63:0/91019374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0065640 msgr2=0x7f31b0065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.543+0000 7f31b4d92700 1 --2- 172.21.15.63:0/91019374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0065640 0x7f31b0065ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31a4004940 tx=0x7f31a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- 172.21.15.63:0/91019374 shutdown_connections 2024-04-24T22:55:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 --2- 172.21.15.63:0/91019374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0065640 0x7f31b0065ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 --2- 172.21.15.63:0/91019374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0102e50 0x7f31b0103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- 172.21.15.63:0/91019374 >> 172.21.15.63:0/91019374 conn(0x7f31b0071d00 msgr2=0x7f31b0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- 172.21.15.63:0/91019374 shutdown_connections 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- 172.21.15.63:0/91019374 wait complete. 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 Processor -- start 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- start start 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0065640 0x7f31b0113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0114660 con 0x7f31b0102e50 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.544+0000 7f31b4d92700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b010da50 con 0x7f31b0065640 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31ae59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0065640 0x7f31b0113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36828/0 (socket says 172.21.15.63:36828) 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 -- 172.21.15.63:0/1897577320 learned_addr learned my addr 172.21.15.63:0/1897577320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:11.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 -- 172.21.15.63:0/1897577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0065640 msgr2=0x7f31b0113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0065640 0x7f31b0113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 -- 172.21.15.63:0/1897577320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4005040 con 0x7f31b0102e50 2024-04-24T22:55:11.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31add9b700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009180 tx=0x7f31a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a400e040 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31a4013420 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b010dcd0 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a401c630 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.545+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b010e290 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.546+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b0100100 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.547+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f31a4013590 con 0x7f31b0102e50 2024-04-24T22:55:11.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.547+0000 7f319effd700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 0x7f3198070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.547+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f31a408eca0 con 0x7f31b0102e50 2024-04-24T22:55:11.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.547+0000 7f31ae59c700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 0x7f3198070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.548+0000 7f31ae59c700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 0x7f3198070600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009fd0 tx=0x7f31a000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.552+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f31a4093070 con 0x7f31b0102e50 2024-04-24T22:55:11.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:11 smithi063 ceph-mon[111213]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-04-24T22:55:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:11 smithi063 ceph-mon[111213]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:11 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:11.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.851+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f31b010f120 con 0x7f31b0102e50 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.851+0000 7f319effd700 1 -- 172.21.15.63:0/1897577320 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f31a405bbd0 con 0x7f31b0102e50 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:11.853 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 12 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:55:11.854 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 msgr2=0x7f3198070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 0x7f3198070600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009fd0 tx=0x7f31a000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 msgr2=0x7f31b0114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009180 tx=0x7f31a4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 shutdown_connections 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b0065640 0x7f31b0113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f319806e150 0x7f3198070600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 --2- 172.21.15.63:0/1897577320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b0102e50 0x7f31b0114010 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 >> 172.21.15.63:0/1897577320 conn(0x7f31b0071d00 msgr2=0x7f31b00fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 shutdown_connections 2024-04-24T22:55:11.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.853+0000 7f31b4d92700 1 -- 172.21.15.63:0/1897577320 wait complete. 2024-04-24T22:55:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:11 smithi192 ceph-mon[80463]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-04-24T22:55:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:11 smithi192 ceph-mon[80463]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:11 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:11.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 -- 172.21.15.63:0/2517723348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e900fb520 msgr2=0x7f3e900fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 --2- 172.21.15.63:0/2517723348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e900fb520 0x7f3e900fb990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e84004940 tx=0x7f3e84009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 -- 172.21.15.63:0/2517723348 shutdown_connections 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 --2- 172.21.15.63:0/2517723348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e900fb520 0x7f3e900fb990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 --2- 172.21.15.63:0/2517723348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e90102f60 0x7f3e90103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 -- 172.21.15.63:0/2517723348 >> 172.21.15.63:0/2517723348 conn(0x7f3e900714d0 msgr2=0x7f3e900718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.959+0000 7f3e94b36700 1 -- 172.21.15.63:0/2517723348 shutdown_connections 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 -- 172.21.15.63:0/2517723348 wait complete. 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 Processor -- start 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 -- start start 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e900fb520 0x7f3e900fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900fe650 con 0x7f3e90102f60 2024-04-24T22:55:11.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e94b36700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900fd180 con 0x7f3e900fb520 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e8dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e8dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36848/0 (socket says 172.21.15.63:36848) 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e8dd9b700 1 -- 172.21.15.63:0/112285744 learned_addr learned my addr 172.21.15.63:0/112285744 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.960+0000 7f3e8e59c700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e900fb520 0x7f3e900fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e8dd9b700 1 -- 172.21.15.63:0/112285744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e900fb520 msgr2=0x7f3e900fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e8dd9b700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e900fb520 0x7f3e900fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e8dd9b700 1 -- 172.21.15.63:0/112285744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84005040 con 0x7f3e90102f60 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e8dd9b700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009150 tx=0x7f3e84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8400e040 con 0x7f3e90102f60 2024-04-24T22:55:11.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e84013420 con 0x7f3e90102f60 2024-04-24T22:55:11.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e900fd400 con 0x7f3e90102f60 2024-04-24T22:55:11.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8401c610 con 0x7f3e90102f60 2024-04-24T22:55:11.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.961+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9011a090 con 0x7f3e90102f60 2024-04-24T22:55:11.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.962+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90100250 con 0x7f3e90102f60 2024-04-24T22:55:11.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.962+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e8400baf0 con 0x7f3e90102f60 2024-04-24T22:55:11.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.963+0000 7f3e7effd700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 0x7f3e780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:11.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.963+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3e8408ed10 con 0x7f3e90102f60 2024-04-24T22:55:11.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.963+0000 7f3e8e59c700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 0x7f3e780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:11.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.963+0000 7f3e8e59c700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 0x7f3e780705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e900678d0 tx=0x7f3e8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:11.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:11.968+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3e84058110 con 0x7f3e90102f60 2024-04-24T22:55:12.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.223+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e9010a000 con 0x7f3e90102f60 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.223+0000 7f3e7effd700 1 -- 172.21.15.63:0/112285744 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1703 (secure 0 0 0) 0x7f3e8405b7d0 con 0x7f3e90102f60 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:e20 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:55:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 20 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:55:08.105802+0000 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 82 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:55:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24261} 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{0:24261} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:55:12.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:12.228 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:12.228 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/3118675914,v1:172.21.15.63:6829/3118675914] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:12.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 msgr2=0x7f3e780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 0x7f3e780705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e900678d0 tx=0x7f3e8000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:12.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 msgr2=0x7f3e900fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009150 tx=0x7f3e84009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 shutdown_connections 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e900fb520 0x7f3e900fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.225+0000 7f3e94b36700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7806e140 0x7f3e780705f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.226+0000 7f3e94b36700 1 --2- 172.21.15.63:0/112285744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90102f60 0x7f3e900fcc40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.226+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 >> 172.21.15.63:0/112285744 conn(0x7f3e900714d0 msgr2=0x7f3e900fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.226+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 shutdown_connections 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.226+0000 7f3e94b36700 1 -- 172.21.15.63:0/112285744 wait complete. 2024-04-24T22:55:12.229 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 20 2024-04-24T22:55:12.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 -- 172.21.15.63:0/2130188518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c103790 msgr2=0x7fdf0c0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2130188518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c103790 0x7fdf0c0fce00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdef4004940 tx=0x7fdef4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:12.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 -- 172.21.15.63:0/2130188518 shutdown_connections 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2130188518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c103790 0x7fdf0c0fce00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2130188518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c102df0 0x7fdf0c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 -- 172.21.15.63:0/2130188518 >> 172.21.15.63:0/2130188518 conn(0x7fdf0c0f8980 msgr2=0x7fdf0c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 -- 172.21.15.63:0/2130188518 shutdown_connections 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 -- 172.21.15.63:0/2130188518 wait complete. 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.331+0000 7fdf11d72700 1 Processor -- start 2024-04-24T22:55:12.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf11d72700 1 -- start start 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf11d72700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf11d72700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c103790 0x7fdf0c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf11d72700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c070d30 con 0x7fdf0c102df0 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf11d72700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c070e70 con 0x7fdf0c103790 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36876/0 (socket says 172.21.15.63:36876) 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 -- 172.21.15.63:0/2079493661 learned_addr learned my addr 172.21.15.63:0/2079493661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0affd700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c103790 0x7fdf0c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 -- 172.21.15.63:0/2079493661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c103790 msgr2=0x7fdf0c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c103790 0x7fdf0c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.332+0000 7fdf0b7fe700 1 -- 172.21.15.63:0/2079493661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4005040 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf0b7fe700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf04001d80 tx=0x7fdf04009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf04007d10 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c119e90 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf04001e80 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf04014640 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.333+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c11a310 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.334+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c0fe540 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.334+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf0400e960 con 0x7fdf0c102df0 2024-04-24T22:55:12.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.335+0000 7fdf03fff700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 0x7fdef80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.335+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf0408d870 con 0x7fdf0c102df0 2024-04-24T22:55:12.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.335+0000 7fdf0affd700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 0x7fdef80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.335+0000 7fdf0affd700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 0x7fdef80705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c071ba0 tx=0x7fdef4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:12.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.339+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf04056ca0 con 0x7fdf0c102df0 2024-04-24T22:55:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:12 smithi063 ceph-mon[111213]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:12 smithi063 ceph-mon[111213]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:12 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1897577320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:12.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:12 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/112285744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:55:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.571+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf0c04bf70 con 0x7fdef806e140 2024-04-24T22:55:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.572+0000 7fdf03fff700 1 -- 172.21.15.63:0/2079493661 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdef400b670 con 0x7fdef806e140 2024-04-24T22:55:12.573 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:55:12.573 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "14/21 daemons upgraded", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:55:12.574 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 msgr2=0x7fdef80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 0x7fdef80705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c071ba0 tx=0x7fdef4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 msgr2=0x7fdf0c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf04001d80 tx=0x7fdf04009ec0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 shutdown_connections 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdef806e140 0x7fdef80705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf0c102df0 0x7fdf0c070260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 --2- 172.21.15.63:0/2079493661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf0c103790 0x7fdf0c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 >> 172.21.15.63:0/2079493661 conn(0x7fdf0c0f8980 msgr2=0x7fdf0c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 shutdown_connections 2024-04-24T22:55:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.574+0000 7fdf11d72700 1 -- 172.21.15.63:0/2079493661 wait complete. 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 -- 172.21.15.63:0/1267667650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 msgr2=0x7f0dac0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 --2- 172.21.15.63:0/1267667650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac0fcdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c004940 tx=0x7f0d9c009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 -- 172.21.15.63:0/1267667650 shutdown_connections 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 --2- 172.21.15.63:0/1267667650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac0fcdf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 --2- 172.21.15.63:0/1267667650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 0x7f0dac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.678+0000 7f0db395c700 1 -- 172.21.15.63:0/1267667650 >> 172.21.15.63:0/1267667650 conn(0x7f0dac0f8990 msgr2=0x7f0dac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:12.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 -- 172.21.15.63:0/1267667650 shutdown_connections 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 -- 172.21.15.63:0/1267667650 wait complete. 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 Processor -- start 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 -- start start 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 0x7f0dac113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac114520 con 0x7f0dac103780 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.679+0000 7f0db395c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac10db20 con 0x7f0dac102de0 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36886/0 (socket says 172.21.15.63:36886) 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 -- 172.21.15.63:0/3871509440 learned_addr learned my addr 172.21.15.63:0/3871509440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db16f8700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 0x7f0dac113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 -- 172.21.15.63:0/3871509440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 msgr2=0x7f0dac113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 0x7f0dac113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 -- 172.21.15.63:0/3871509440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c005040 con 0x7f0dac103780 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db0ef7700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009100 tx=0x7f0d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c00e040 con 0x7f0dac103780 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac10dda0 con 0x7f0dac103780 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9c009af0 con 0x7f0dac103780 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c01c3f0 con 0x7f0dac103780 2024-04-24T22:55:12.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.680+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac10e360 con 0x7f0dac103780 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.682+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac0fe530 con 0x7f0dac103780 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.682+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0d9c00baf0 con 0x7f0dac103780 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.682+0000 7f0da1ffb700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 0x7f0d980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.682+0000 7f0db16f8700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 0x7f0d980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.683+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0d9c08ed60 con 0x7f0dac103780 2024-04-24T22:55:12.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.683+0000 7f0db16f8700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 0x7f0d980705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004eb0 tx=0x7f0da800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.687+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0d9c05bc90 con 0x7f0dac103780 2024-04-24T22:55:12.740 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:12 smithi192 ceph-mon[80463]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:12.741 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:12 smithi192 ceph-mon[80463]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:12.741 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:12 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1897577320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:12.741 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:12 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/112285744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:55:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.979+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0dac109ec0 con 0x7f0dac103780 2024-04-24T22:55:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.979+0000 7f0da1ffb700 1 -- 172.21.15.63:0/3871509440 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f0d9c05b820 con 0x7f0dac103780 2024-04-24T22:55:12.981 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-24T22:55:12.981 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-24T22:55:12.981 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is degraded 2024-04-24T22:55:12.981 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:55:12.981 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:55:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 msgr2=0x7f0d980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 0x7f0d980705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004eb0 tx=0x7f0da800b040 comp rx=0 tx=0).stop 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 msgr2=0x7f0dac113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009100 tx=0x7f0d9c009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 shutdown_connections 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac102de0 0x7f0dac113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f0d9806e140 0x7f0d980705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 --2- 172.21.15.63:0/3871509440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dac103780 0x7f0dac113ed0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 >> 172.21.15.63:0/3871509440 conn(0x7f0dac0f8990 msgr2=0x7f0dac0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 shutdown_connections 2024-04-24T22:55:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:12.981+0000 7f0db395c700 1 -- 172.21.15.63:0/3871509440 wait complete. 2024-04-24T22:55:13.724 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:13 smithi192 ceph-mon[80463]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-04-24T22:55:13.724 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:13 smithi192 ceph-mon[80463]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:13.724 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:13 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3871509440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:55:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:13 smithi063 ceph-mon[111213]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-04-24T22:55:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:13 smithi063 ceph-mon[111213]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:13 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3871509440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:55:15.771 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:15 smithi192 ceph-mon[80463]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T22:55:15.772 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:15.772 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:15.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:15 smithi063 ceph-mon[111213]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T22:55:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:17.806 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:17 smithi063 ceph-mon[111213]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T22:55:17.934 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:17 smithi192 ceph-mon[80463]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-24T22:55:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:19 smithi063 ceph-mon[111213]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-24T22:55:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:19 smithi063 ceph-mon[111213]: reconnect by client.24305 192.168.0.1:0/857152497 after 0.000999982 2024-04-24T22:55:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:19 smithi063 ceph-mon[111213]: reconnect by client.14522 192.168.0.1:0/3741521470 after 0.00199996 2024-04-24T22:55:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:19 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:reconnect 2024-04-24T22:55:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:19 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:reconnect} 2 up:standby 2024-04-24T22:55:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:19 smithi192 ceph-mon[80463]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-24T22:55:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:19 smithi192 ceph-mon[80463]: reconnect by client.24305 192.168.0.1:0/857152497 after 0.000999982 2024-04-24T22:55:19.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:19 smithi192 ceph-mon[80463]: reconnect by client.14522 192.168.0.1:0/3741521470 after 0.00199996 2024-04-24T22:55:19.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:19 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:reconnect 2024-04-24T22:55:19.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:19 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:reconnect} 2 up:standby 2024-04-24T22:55:20.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:20 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:rejoin 2024-04-24T22:55:20.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:20 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:rejoin} 2 up:standby 2024-04-24T22:55:20.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:20 smithi192 ceph-mon[80463]: daemon mds.cephfs.smithi192.ozemqi is now active in filesystem cephfs as rank 0 2024-04-24T22:55:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:20 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:rejoin 2024-04-24T22:55:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:20 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi192.ozemqi=up:rejoin} 2 up:standby 2024-04-24T22:55:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:20 smithi063 ceph-mon[111213]: daemon mds.cephfs.smithi192.ozemqi is now active in filesystem cephfs as rank 0 2024-04-24T22:55:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:21 smithi063 ceph-mon[111213]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T22:55:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:21 smithi063 ceph-mon[111213]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:55:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:21 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:active 2024-04-24T22:55:21.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:21 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.ozemqi=up:active} 2 up:standby 2024-04-24T22:55:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:21 smithi192 ceph-mon[80463]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T22:55:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:21 smithi192 ceph-mon[80463]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:55:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:21 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.192:6826/2625956780,v1:172.21.15.192:6827/2625956780] up:active 2024-04-24T22:55:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:21 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.ozemqi=up:active} 2 up:standby 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.192:6824/3626023513,v1:172.21.15.192:6825/3626023513] up:boot 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi192.ozemqi=up:active} 3 up:standby 2024-04-24T22:55:23.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:23 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.192:6824/3626023513,v1:172.21.15.192:6825/3626023513] up:boot 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi192.ozemqi=up:active} 3 up:standby 2024-04-24T22:55:23.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:23 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:25 smithi063 ceph-mon[111213]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-24T22:55:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:25 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:25 smithi192 ceph-mon[80463]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-24T22:55:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:25 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:27.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:27.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:27 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:27.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:27 smithi063 ceph-mon[111213]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:27 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:27.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:27 smithi192 ceph-mon[80463]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:29.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:29 smithi192 ceph-mon[80463]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:29 smithi063 ceph-mon[111213]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:30.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:31.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:31 smithi192 ceph-mon[80463]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:55:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:55:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:31 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:31 smithi063 ceph-mon[111213]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:55:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.ozemqi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T22:55:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:31 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:32.898 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:32 smithi192 ceph-mon[80463]: Upgrade: Updating mds.cephfs.smithi192.ozemqi 2024-04-24T22:55:32.899 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:32 smithi192 ceph-mon[80463]: Deploying daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:55:33.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:32 smithi063 ceph-mon[111213]: Upgrade: Updating mds.cephfs.smithi192.ozemqi 2024-04-24T22:55:33.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:32 smithi063 ceph-mon[111213]: Deploying daemon mds.cephfs.smithi192.ozemqi on smithi192 2024-04-24T22:55:33.682 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[111209]: 2024-04-24T22:55:33.427+0000 7fcca103f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: Standby daemon mds.cephfs.smithi063.twfoxu assigned to filesystem cephfs as rank 0 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T22:55:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:33 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:replay} 2 up:standby 2024-04-24T22:55:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:55:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T22:55:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T22:55:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T22:55:34.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: Standby daemon mds.cephfs.smithi063.twfoxu assigned to filesystem cephfs as rank 0 2024-04-24T22:55:34.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T22:55:34.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T22:55:34.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:33 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:replay} 2 up:standby 2024-04-24T22:55:36.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:35 smithi063 ceph-mon[111213]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T22:55:36.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:35 smithi192 ceph-mon[80463]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T22:55:38.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:37 smithi063 ceph-mon[111213]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-04-24T22:55:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:37 smithi192 ceph-mon[80463]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-04-24T22:55:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:39 smithi063 ceph-mon[111213]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-04-24T22:55:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:39 smithi192 ceph-mon[80463]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-04-24T22:55:41.870 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:41 smithi192 ceph-mon[80463]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:41 smithi063 ceph-mon[111213]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 -- 172.21.15.63:0/4124993997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 msgr2=0x7f2c34109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 --2- 172.21.15.63:0/4124993997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c34109160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c28004970 tx=0x7f2c28009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 -- 172.21.15.63:0/4124993997 shutdown_connections 2024-04-24T22:55:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 --2- 172.21.15.63:0/4124993997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c34109160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 --2- 172.21.15.63:0/4124993997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 -- 172.21.15.63:0/4124993997 >> 172.21.15.63:0/4124993997 conn(0x7f2c340f8980 msgr2=0x7f2c340fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 -- 172.21.15.63:0/4124993997 shutdown_connections 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.092+0000 7f2c395ae700 1 -- 172.21.15.63:0/4124993997 wait complete. 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 Processor -- start 2024-04-24T22:55:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 -- start start 2024-04-24T22:55:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341169b0 con 0x7f2c34070bb0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c395ae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3410fcf0 con 0x7f2c340702a0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c327fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c327fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32934/0 (socket says 172.21.15.63:32934) 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c327fc700 1 -- 172.21.15.63:0/3435350443 learned_addr learned my addr 172.21.15.63:0/3435350443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.093+0000 7f2c32ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c32ffd700 1 -- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 msgr2=0x7f2c341162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c32ffd700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c32ffd700 1 -- 172.21.15.63:0/3435350443 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c28005040 con 0x7f2c340702a0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c327fc700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c32ffd700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34115d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400b580 tx=0x7f2c240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2400f6f0 con 0x7f2c340702a0 2024-04-24T22:55:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2400fd20 con 0x7f2c340702a0 2024-04-24T22:55:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3410ff70 con 0x7f2c340702a0 2024-04-24T22:55:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c240103a0 con 0x7f2c340702a0 2024-04-24T22:55:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.094+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c341104b0 con 0x7f2c340702a0 2024-04-24T22:55:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.095+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c340719e0 con 0x7f2c340702a0 2024-04-24T22:55:43.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.096+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2c2400f850 con 0x7f2c340702a0 2024-04-24T22:55:43.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.096+0000 7f2c1b7fe700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 0x7f2c1c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.096+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2c2408d800 con 0x7f2c340702a0 2024-04-24T22:55:43.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.096+0000 7f2c327fc700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 0x7f2c1c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.097+0000 7f2c327fc700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 0x7f2c1c0705f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b020 tx=0x7f2c28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.102+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2c2405a720 con 0x7f2c340702a0 2024-04-24T22:55:43.331 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:43.331 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:43.331 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:43.332 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:43.332 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.192:6826/3112021694,v1:172.21.15.192:6827/3112021694] up:boot 2024-04-24T22:55:43.332 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:replay} 3 up:standby 2024-04-24T22:55:43.332 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:43 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:55:43.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.335+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c341113e0 con 0x7f2c1c06e140 2024-04-24T22:55:43.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.336+0000 7f2c1b7fe700 1 -- 172.21.15.63:0/3435350443 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2c341113e0 con 0x7f2c1c06e140 2024-04-24T22:55:43.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 msgr2=0x7f2c1c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 0x7f2c1c0705f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b020 tx=0x7f2c28009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:43.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 msgr2=0x7f2c34115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34115d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400b580 tx=0x7f2c240091a0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 shutdown_connections 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c340702a0 0x7f2c34115d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2c1c06e140 0x7f2c1c0705f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 --2- 172.21.15.63:0/3435350443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34070bb0 0x7f2c341162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 >> 172.21.15.63:0/3435350443 conn(0x7f2c340f8980 msgr2=0x7f2c340f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 shutdown_connections 2024-04-24T22:55:43.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.338+0000 7f2c395ae700 1 -- 172.21.15.63:0/3435350443 wait complete. 2024-04-24T22:55:43.350 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.192:6826/3112021694,v1:172.21.15.192:6827/3112021694] up:boot 2024-04-24T22:55:43.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:replay} 3 up:standby 2024-04-24T22:55:43.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:43 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:55:43.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- 172.21.15.63:0/708758528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c102f60 msgr2=0x7f1d3c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 --2- 172.21.15.63:0/708758528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c102f60 0x7f1d3c103330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c004940 tx=0x7f1d2c009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- 172.21.15.63:0/708758528 shutdown_connections 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 --2- 172.21.15.63:0/708758528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 --2- 172.21.15.63:0/708758528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c102f60 0x7f1d3c103330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- 172.21.15.63:0/708758528 >> 172.21.15.63:0/708758528 conn(0x7f1d3c0714d0 msgr2=0x7f1d3c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- 172.21.15.63:0/708758528 shutdown_connections 2024-04-24T22:55:43.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- 172.21.15.63:0/708758528 wait complete. 2024-04-24T22:55:43.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 Processor -- start 2024-04-24T22:55:43.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.444+0000 7f1d4279d700 1 -- start start 2024-04-24T22:55:43.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d4279d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d4279d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c102f60 0x7f1d3c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d4279d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c116720 con 0x7f1d3c0fb520 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d4279d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c10fd00 con 0x7f1d3c102f60 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32952/0 (socket says 172.21.15.63:32952) 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 -- 172.21.15.63:0/278938529 learned_addr learned my addr 172.21.15.63:0/278938529 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3b7fe700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c102f60 0x7f1d3c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 -- 172.21.15.63:0/278938529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c102f60 msgr2=0x7f1d3c1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c102f60 0x7f1d3c1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 -- 172.21.15.63:0/278938529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c005040 con 0x7f1d3c0fb520 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d3bfff700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00b3b0 tx=0x7f1d2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.445+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2c00e040 con 0x7f1d3c0fb520 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.446+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d2c013420 con 0x7f1d3c0fb520 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.446+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2c01c630 con 0x7f1d3c0fb520 2024-04-24T22:55:43.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.446+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c10ff80 con 0x7f1d3c0fb520 2024-04-24T22:55:43.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.446+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1104c0 con 0x7f1d3c0fb520 2024-04-24T22:55:43.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.447+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c107410 con 0x7f1d3c0fb520 2024-04-24T22:55:43.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.447+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1d2c013590 con 0x7f1d3c0fb520 2024-04-24T22:55:43.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.448+0000 7f1d38ff9700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 0x7f1d240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.448+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1d2c08ec60 con 0x7f1d3c0fb520 2024-04-24T22:55:43.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.448+0000 7f1d3b7fe700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 0x7f1d240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.448+0000 7f1d3b7fe700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 0x7f1d240705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c111430 tx=0x7f1d3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.453+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1d2c05bc00 con 0x7f1d3c0fb520 2024-04-24T22:55:43.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.683+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d3c10a080 con 0x7f1d2406e140 2024-04-24T22:55:43.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.684+0000 7f1d38ff9700 1 -- 172.21.15.63:0/278938529 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1d3000f040 con 0x7f1d2406e140 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 msgr2=0x7f1d240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 0x7f1d240705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c111430 tx=0x7f1d3000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 msgr2=0x7f1d3c115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00b3b0 tx=0x7f1d2c009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 shutdown_connections 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1d2406e140 0x7f1d240705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d3c0fb520 0x7f1d3c115b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 --2- 172.21.15.63:0/278938529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c102f60 0x7f1d3c1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.686+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 >> 172.21.15.63:0/278938529 conn(0x7f1d3c0714d0 msgr2=0x7f1d3c0fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.687+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 shutdown_connections 2024-04-24T22:55:43.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.687+0000 7f1d4279d700 1 -- 172.21.15.63:0/278938529 wait complete. 2024-04-24T22:55:43.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 -- 172.21.15.63:0/2609166026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c101d00 msgr2=0x7f702c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 --2- 172.21.15.63:0/2609166026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c101d00 0x7f702c1020d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f701c009ce0 tx=0x7f701c004d10 comp rx=0 tx=0).stop 2024-04-24T22:55:43.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 -- 172.21.15.63:0/2609166026 shutdown_connections 2024-04-24T22:55:43.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 --2- 172.21.15.63:0/2609166026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c1026a0 0x7f702c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 --2- 172.21.15.63:0/2609166026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c101d00 0x7f702c1020d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 -- 172.21.15.63:0/2609166026 >> 172.21.15.63:0/2609166026 conn(0x7f702c0f7870 msgr2=0x7f702c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 -- 172.21.15.63:0/2609166026 shutdown_connections 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 -- 172.21.15.63:0/2609166026 wait complete. 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.792+0000 7f7030c10700 1 Processor -- start 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7030c10700 1 -- start start 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7030c10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c101d00 0x7f702c102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7030c10700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7030c10700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c104ab0 con 0x7f702c1026a0 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7030c10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c1035e0 con 0x7f702c101d00 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f702a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c101d00 0x7f702c102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32958/0 (socket says 172.21.15.63:32958) 2024-04-24T22:55:43.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 -- 172.21.15.63:0/2582204157 learned_addr learned my addr 172.21.15.63:0/2582204157 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:43.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 -- 172.21.15.63:0/2582204157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c101d00 msgr2=0x7f702c102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:43.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c101d00 0x7f702c102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 -- 172.21.15.63:0/2582204157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c009940 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f7029d9b700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f702000eb40 tx=0x7f702000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.793+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7020007cc0 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.794+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f702000f040 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.794+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c103860 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.794+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f702000c9d0 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.794+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c11a150 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.795+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c0fd450 con 0x7f702c1026a0 2024-04-24T22:55:43.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.795+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f702000cb30 con 0x7f702c1026a0 2024-04-24T22:55:43.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.796+0000 7f701affd700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 0x7f70140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:43.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.796+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f702008d4c0 con 0x7f702c1026a0 2024-04-24T22:55:43.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.796+0000 7f702a59c700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 0x7f70140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:43.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.796+0000 7f702a59c700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 0x7f70140705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f701c0048e0 tx=0x7f701c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:43.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:43.801+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7020056920 con 0x7f702c1026a0 2024-04-24T22:55:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.016+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f702c108ea0 con 0x7f701406e140 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.027+0000 7f701affd700 1 -- 172.21.15.63:0/2582204157 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f701c009500 con 0x7f701406e140 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (7m) 44s ago 18m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 44s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (4m) 18s ago 17m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 44s ago 18m 79.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (49s) 44s ago 13m 11.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 091ce07d1d7d 2024-04-24T22:55:44.029 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (69s) 44s ago 13m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea388604c60e 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (24s) 18s ago 13m 14.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef e6cfd9a816b1 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 starting - - - - 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (9m) 44s ago 20m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (8m) 18s ago 17m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 44s ago 20m 54.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (5m) 18s ago 17m 44.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 44s ago 18m 15.8M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:55:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 18s ago 17m 17.0M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 44s ago 16m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 44s ago 16m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (3m) 44s ago 15m 76.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (2m) 18s ago 15m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (117s) 18s ago 15m 91.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (84s) 18s ago 14m 90.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef b7243340f774 2024-04-24T22:55:44.031 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 44s ago 18m 61.3M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 msgr2=0x7f70140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 0x7f70140705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f701c0048e0 tx=0x7f701c004d10 comp rx=0 tx=0).stop 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 msgr2=0x7f702c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f702000eb40 tx=0x7f702000b040 comp rx=0 tx=0).stop 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 shutdown_connections 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f702c101d00 0x7f702c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f701406e140 0x7f70140705f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 --2- 172.21.15.63:0/2582204157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c1026a0 0x7f702c1030a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 >> 172.21.15.63:0/2582204157 conn(0x7f702c0f7870 msgr2=0x7f702c066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 shutdown_connections 2024-04-24T22:55:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.029+0000 7f7030c10700 1 -- 172.21.15.63:0/2582204157 wait complete. 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 -- 172.21.15.63:0/2634422128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc102de0 msgr2=0x7f89bc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 --2- 172.21.15.63:0/2634422128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc102de0 0x7f89bc1031b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004940 tx=0x7f89ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 -- 172.21.15.63:0/2634422128 shutdown_connections 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 --2- 172.21.15.63:0/2634422128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc103780 0x7f89bc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 --2- 172.21.15.63:0/2634422128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc102de0 0x7f89bc1031b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 -- 172.21.15.63:0/2634422128 >> 172.21.15.63:0/2634422128 conn(0x7f89bc0f8950 msgr2=0x7f89bc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 -- 172.21.15.63:0/2634422128 shutdown_connections 2024-04-24T22:55:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 -- 172.21.15.63:0/2634422128 wait complete. 2024-04-24T22:55:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.132+0000 7f89c166f700 1 Processor -- start 2024-04-24T22:55:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89c166f700 1 -- start start 2024-04-24T22:55:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89c166f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc102de0 0x7f89bc113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89c166f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89c166f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc114700 con 0x7f89bc103780 2024-04-24T22:55:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89c166f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc10daf0 con 0x7f89bc102de0 2024-04-24T22:55:44.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32984/0 (socket says 172.21.15.63:32984) 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 -- 172.21.15.63:0/3823213436 learned_addr learned my addr 172.21.15.63:0/3823213436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89baffd700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc102de0 0x7f89bc113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 -- 172.21.15.63:0/3823213436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc102de0 msgr2=0x7f89bc113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc102de0 0x7f89bc113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 -- 172.21.15.63:0/3823213436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac005040 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89ba7fc700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89b0002a50 tx=0x7f89b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.133+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b0007cc0 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.134+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc10ddd0 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.134+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89b000f040 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.134+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b000d9c0 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.134+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc10e390 con 0x7f89bc103780 2024-04-24T22:55:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.135+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89bc1000f0 con 0x7f89bc103780 2024-04-24T22:55:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.135+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f89b0009730 con 0x7f89bc103780 2024-04-24T22:55:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.136+0000 7f89a37fe700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 0x7f89a4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.136+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f89b008d8e0 con 0x7f89bc103780 2024-04-24T22:55:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.136+0000 7f89baffd700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 0x7f89a4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.136+0000 7f89baffd700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 0x7f89a4070600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b3b0 tx=0x7f89ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.140+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f89b005a740 con 0x7f89bc103780 2024-04-24T22:55:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.436+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f89bc10f1c0 con 0x7f89bc103780 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.437+0000 7f89a37fe700 1 -- 172.21.15.63:0/3823213436 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f89b005a2d0 con 0x7f89bc103780 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:44.439 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:55:44.440 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:55:44.441 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-24T22:55:44.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:55:44.441 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:55:44.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 msgr2=0x7f89a4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 0x7f89a4070600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00b3b0 tx=0x7f89ac009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 msgr2=0x7f89bc1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89b0002a50 tx=0x7f89b0004600 comp rx=0 tx=0).stop 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 shutdown_connections 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89bc102de0 0x7f89bc113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f89a406e150 0x7f89a4070600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 --2- 172.21.15.63:0/3823213436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89bc103780 0x7f89bc1140b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 >> 172.21.15.63:0/3823213436 conn(0x7f89bc0f8950 msgr2=0x7f89bc1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 shutdown_connections 2024-04-24T22:55:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.439+0000 7f89c166f700 1 -- 172.21.15.63:0/3823213436 wait complete. 2024-04-24T22:55:44.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 -- 172.21.15.63:0/2457369233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8103780 msgr2=0x7f44d80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 --2- 172.21.15.63:0/2457369233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8103780 0x7f44d80fcdf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44d4004940 tx=0x7f44d4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:44.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 -- 172.21.15.63:0/2457369233 shutdown_connections 2024-04-24T22:55:44.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 --2- 172.21.15.63:0/2457369233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8103780 0x7f44d80fcdf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 --2- 172.21.15.63:0/2457369233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8102de0 0x7f44d81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.545+0000 7f44e05e1700 1 -- 172.21.15.63:0/2457369233 >> 172.21.15.63:0/2457369233 conn(0x7f44d80f8990 msgr2=0x7f44d80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 -- 172.21.15.63:0/2457369233 shutdown_connections 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 -- 172.21.15.63:0/2457369233 wait complete. 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 Processor -- start 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 -- start start 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8103780 0x7f44d8113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8114490 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.546+0000 7f44e05e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d810da90 con 0x7f44d8103780 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33004/0 (socket says 172.21.15.63:33004) 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 -- 172.21.15.63:0/1798948179 learned_addr learned my addr 172.21.15.63:0/1798948179 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44ddb7c700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8103780 0x7f44d8113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 -- 172.21.15.63:0/1798948179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8103780 msgr2=0x7f44d8113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8103780 0x7f44d8113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 -- 172.21.15.63:0/1798948179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4005040 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44de37d700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44c800eb40 tx=0x7f44c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c8007cc0 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44c800f040 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c800ca10 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d810dd10 con 0x7f44d8102de0 2024-04-24T22:55:44.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.547+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d810e220 con 0x7f44d8102de0 2024-04-24T22:55:44.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.548+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d80fe530 con 0x7f44d8102de0 2024-04-24T22:55:44.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.549+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44c800cb70 con 0x7f44d8102de0 2024-04-24T22:55:44.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.549+0000 7f44ceffd700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 0x7f44c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.550+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f44c808d670 con 0x7f44d8102de0 2024-04-24T22:55:44.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.550+0000 7f44ddb7c700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 0x7f44c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.550+0000 7f44ddb7c700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 0x7f44c40705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44d4004890 tx=0x7f44d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.554+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f44c805a600 con 0x7f44d8102de0 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.812+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f44d8109f10 con 0x7f44d8102de0 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.812+0000 7f44ceffd700 1 -- 172.21.15.63:0/1798948179 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1928 (secure 0 0 0) 0x7f44c805a190 con 0x7f44d8102de0 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stdout:e29 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:55:44.814 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 29 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:55:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:55:44.817 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:55:43.696911+0000 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 83 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34374} 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:55:44.818 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{0:34374} state up:rejoin seq 19 join_fscid=1 addr [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:44.819 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/3118675914,v1:172.21.15.63:6829/3118675914] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{-1:44309} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3626023513,v1:172.21.15.192:6825/3626023513] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:44.820 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3112021694,v1:172.21.15.192:6827/3112021694] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:55:44.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 msgr2=0x7f44c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 0x7f44c40705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44d4004890 tx=0x7f44d4009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 msgr2=0x7f44d8113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44c800eb40 tx=0x7f44c800b040 comp rx=0 tx=0).stop 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 shutdown_connections 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f44c406e140 0x7f44c40705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d8102de0 0x7f44d8113900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 --2- 172.21.15.63:0/1798948179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d8103780 0x7f44d8113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 >> 172.21.15.63:0/1798948179 conn(0x7f44d80f8990 msgr2=0x7f44d80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 shutdown_connections 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.814+0000 7f44e05e1700 1 -- 172.21.15.63:0/1798948179 wait complete. 2024-04-24T22:55:44.822 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3893081003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd50101d10 msgr2=0x7fdd501020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3893081003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd50101d10 0x7fdd501020e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009ce0 tx=0x7fdd48004d10 comp rx=0 tx=0).stop 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3893081003 shutdown_connections 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3893081003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd501026b0 0x7fdd500fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3893081003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd50101d10 0x7fdd501020e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3893081003 >> 172.21.15.63:0/3893081003 conn(0x7fdd500f7860 msgr2=0x7fdd500f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3893081003 shutdown_connections 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.919+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3893081003 wait complete. 2024-04-24T22:55:44.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 Processor -- start 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 -- start start 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd50101d10 0x7fdd5010c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50067100 con 0x7fdd501026b0 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd54fbe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5010d3e0 con 0x7fdd50101d10 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33032/0 (socket says 172.21.15.63:33032) 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd50101d10 0x7fdd5010c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 -- 172.21.15.63:0/3992253553 learned_addr learned my addr 172.21.15.63:0/3992253553 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 -- 172.21.15.63:0/3992253553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd50101d10 msgr2=0x7fdd5010c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:44.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd50101d10 0x7fdd5010c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.920+0000 7fdd4dd9b700 1 -- 172.21.15.63:0/3992253553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009940 con 0x7fdd501026b0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd4dd9b700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800eb40 tx=0x7fdd3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd38007cc0 con 0x7fdd501026b0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd3800f040 con 0x7fdd501026b0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5010d660 con 0x7fdd501026b0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3800c9d0 con 0x7fdd501026b0 2024-04-24T22:55:44.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.921+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5011a190 con 0x7fdd501026b0 2024-04-24T22:55:44.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.922+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd500fd460 con 0x7fdd501026b0 2024-04-24T22:55:44.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.922+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdd3800cb30 con 0x7fdd501026b0 2024-04-24T22:55:44.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.923+0000 7fdd46ffd700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 0x7fdd3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:44.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.923+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdd3808d4c0 con 0x7fdd501026b0 2024-04-24T22:55:44.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.923+0000 7fdd4e59c700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 0x7fdd3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:44.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.923+0000 7fdd4e59c700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 0x7fdd3c0705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd48004f80 tx=0x7fdd48004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:44.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:44.928+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdd3805a450 con 0x7fdd501026b0 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: from='client.44315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: reconnect by client.14522 192.168.0.1:0/3741521470 after 0.000999992 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: reconnect by client.24305 192.168.0.1:0/857152497 after 0.000999992 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:reconnect 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:reconnect} 3 up:standby 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3823213436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:45.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:44 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:45.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.162+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd50108e80 con 0x7fdd3c06e140 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.163+0000 7fdd46ffd700 1 -- 172.21.15.63:0/3992253553 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd50108e80 con 0x7fdd3c06e140 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "mds", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-04-24T22:55:45.165 INFO:teuthology.orchestra.run.smithi063.stdout: "osd" 2024-04-24T22:55:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-04-24T22:55:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "16/21 daemons upgraded", 2024-04-24T22:55:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T22:55:45.166 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:55:45.166 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:55:45.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 msgr2=0x7fdd3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 0x7fdd3c0705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd48004f80 tx=0x7fdd48004d10 comp rx=0 tx=0).stop 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 msgr2=0x7fdd5010cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800eb40 tx=0x7fdd3800b040 comp rx=0 tx=0).stop 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 shutdown_connections 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd50101d10 0x7fdd5010c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fdd3c06e140 0x7fdd3c0705f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 --2- 172.21.15.63:0/3992253553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd501026b0 0x7fdd5010cea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 >> 172.21.15.63:0/3992253553 conn(0x7fdd500f7860 msgr2=0x7fdd500f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 shutdown_connections 2024-04-24T22:55:45.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.166+0000 7fdd54fbe700 1 -- 172.21.15.63:0/3992253553 wait complete. 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: from='client.44315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: reconnect by client.14522 192.168.0.1:0/3741521470 after 0.000999992 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: reconnect by client.24305 192.168.0.1:0/857152497 after 0.000999992 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:reconnect 2024-04-24T22:55:45.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:reconnect} 3 up:standby 2024-04-24T22:55:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3823213436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:45.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:44 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:45.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 -- 172.21.15.63:0/2315480334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40102e50 msgr2=0x7f2e40103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 --2- 172.21.15.63:0/2315480334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40102e50 0x7f2e40103220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004940 tx=0x7f2e30009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 -- 172.21.15.63:0/2315480334 shutdown_connections 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 --2- 172.21.15.63:0/2315480334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40065640 0x7f2e40065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 --2- 172.21.15.63:0/2315480334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40102e50 0x7f2e40103220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 -- 172.21.15.63:0/2315480334 >> 172.21.15.63:0/2315480334 conn(0x7f2e40071d00 msgr2=0x7f2e40072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 -- 172.21.15.63:0/2315480334 shutdown_connections 2024-04-24T22:55:45.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.271+0000 7f2e4511b700 1 -- 172.21.15.63:0/2315480334 wait complete. 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 Processor -- start 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 -- start start 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40102e50 0x7f2e40113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40114610 con 0x7f2e40065640 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e4511b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4010da40 con 0x7f2e40102e50 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33050/0 (socket says 172.21.15.63:33050) 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 -- 172.21.15.63:0/3310382954 learned_addr learned my addr 172.21.15.63:0/3310382954 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3e59c700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40102e50 0x7f2e40113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 -- 172.21.15.63:0/3310382954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40102e50 msgr2=0x7f2e40113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40102e50 0x7f2e40113fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.272+0000 7f2e3ed9d700 1 -- 172.21.15.63:0/3310382954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30005040 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e3ed9d700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b30 tx=0x7f2e30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e30021070 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4010dcc0 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e30013c00 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3000e070 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.273+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4010e1a0 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.274+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e40100100 con 0x7f2e40065640 2024-04-24T22:55:45.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.274+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2e3001cad0 con 0x7f2e40065640 2024-04-24T22:55:45.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.275+0000 7f2e277fe700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 0x7f2e280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:55:45.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.275+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2e3008efa0 con 0x7f2e40065640 2024-04-24T22:55:45.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.275+0000 7f2e3e59c700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 0x7f2e280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:55:45.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.276+0000 7f2e3e59c700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 0x7f2e280705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e400fa7c0 tx=0x7f2e3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:55:45.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.280+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2e300c3740 con 0x7f2e40065640 2024-04-24T22:55:45.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.572+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e4004bed0 con 0x7f2e40065640 2024-04-24T22:55:45.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.573+0000 7f2e277fe700 1 -- 172.21.15.63:0/3310382954 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f2e3005be80 con 0x7f2e40065640 2024-04-24T22:55:45.574 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-24T22:55:45.574 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-24T22:55:45.574 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is degraded 2024-04-24T22:55:45.575 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:55:45.575 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:55:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 msgr2=0x7f2e280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 0x7f2e280705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e400fa7c0 tx=0x7f2e3400b040 comp rx=0 tx=0).stop 2024-04-24T22:55:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 msgr2=0x7f2e40113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:55:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b30 tx=0x7f2e30009e30 comp rx=0 tx=0).stop 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 shutdown_connections 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2e2806e140 0x7f2e280705f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e40065640 0x7f2e40113a80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 --2- 172.21.15.63:0/3310382954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e40102e50 0x7f2e40113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 >> 172.21.15.63:0/3310382954 conn(0x7f2e40071d00 msgr2=0x7f2e400faf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 shutdown_connections 2024-04-24T22:55:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:55:45.575+0000 7f2e4511b700 1 -- 172.21.15.63:0/3310382954 wait complete. 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:rejoin 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:rejoin} 3 up:standby 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: daemon mds.cephfs.smithi063.twfoxu is now active in filesystem cephfs as rank 0 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1798948179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3310382954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:55:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:rejoin 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: fsmap cephfs:1/1 {0=cephfs.smithi063.twfoxu=up:rejoin} 3 up:standby 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: daemon mds.cephfs.smithi063.twfoxu is now active in filesystem cephfs as rank 0 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1798948179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:55:46.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3310382954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:55:46.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:46 smithi192 ceph-mon[80463]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:46 smithi192 ceph-mon[80463]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:55:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:46 smithi192 ceph-mon[80463]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:active 2024-04-24T22:55:46.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:46 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:55:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:46 smithi063 ceph-mon[111213]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:55:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:46 smithi063 ceph-mon[111213]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T22:55:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:46 smithi063 ceph-mon[111213]: mds.? [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] up:active 2024-04-24T22:55:47.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:46 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:55:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:47 smithi192 ceph-mon[80463]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T22:55:48.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:47 smithi063 ceph-mon[111213]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-24T22:55:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 0 B/s wr, 7 op/s 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:50.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.dnliiv"}]': finished 2024-04-24T22:55:50.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.twfoxu"}]': finished 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.mnfuxk"}]': finished 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.ozemqi"}]': finished 2024-04-24T22:55:50.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:49 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 0 B/s wr, 7 op/s 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.dnliiv"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.dnliiv"}]': finished 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.twfoxu"}]: dispatch 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.twfoxu"}]': finished 2024-04-24T22:55:50.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.mnfuxk"}]: dispatch 2024-04-24T22:55:50.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.mnfuxk"}]': finished 2024-04-24T22:55:50.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.ozemqi"}]: dispatch 2024-04-24T22:55:50.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.ozemqi"}]': finished 2024-04-24T22:55:50.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:49 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T22:55:51.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all mds 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T22:55:51.055 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T22:55:51.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:51.057 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.058 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:51.058 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:51.058 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:51.058 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:50 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all mds 2024-04-24T22:55:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T22:55:51.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: fsmap cephfs:1 {0=cephfs.smithi063.twfoxu=up:active} 3 up:standby 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.178 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T22:55:51.179 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T22:55:51.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:55:51.181 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:55:51.182 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:50 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all rgw 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all iscsi 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all nfs 2024-04-24T22:55:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Setting container_image for all nvmeof 2024-04-24T22:55:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Finalizing container_image settings 2024-04-24T22:55:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T22:55:52.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:51 smithi063 ceph-mon[111213]: Upgrade: Complete! 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all rgw 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all iscsi 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all nfs 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Setting container_image for all nvmeof 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Finalizing container_image settings 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T22:55:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:51 smithi192 ceph-mon[80463]: Upgrade: Complete! 2024-04-24T22:55:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:53 smithi063 ceph-mon[111213]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T22:55:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:53 smithi192 ceph-mon[80463]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T22:55:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:55 smithi063 ceph-mon[111213]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:55 smithi192 ceph-mon[80463]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-24T22:55:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:55:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:57 smithi063 ceph-mon[111213]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:55:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:57 smithi192 ceph-mon[80463]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-24T22:56:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:55:59 smithi063 ceph-mon[111213]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-24T22:56:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:55:59 smithi192 ceph-mon[80463]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-24T22:56:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:01 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:01 smithi192 ceph-mon[80463]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 4.1 KiB/s wr, 2 op/s 2024-04-24T22:56:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:01 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:01 smithi063 ceph-mon[111213]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 4.1 KiB/s wr, 2 op/s 2024-04-24T22:56:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:03 smithi063 ceph-mon[111213]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-24T22:56:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:03 smithi192 ceph-mon[80463]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-24T22:56:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:05 smithi063 ceph-mon[111213]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-24T22:56:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:05 smithi192 ceph-mon[80463]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-24T22:56:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:07 smithi063 ceph-mon[111213]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:07.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:07 smithi192 ceph-mon[80463]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:09 smithi063 ceph-mon[111213]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:09 smithi192 ceph-mon[80463]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:11.729 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:11 smithi192 ceph-mon[80463]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:11 smithi063 ceph-mon[111213]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:13 smithi063 ceph-mon[111213]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:13.919 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:13 smithi192 ceph-mon[80463]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 -- 172.21.15.63:0/973386212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838101d10 msgr2=0x7fe8381020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 --2- 172.21.15.63:0/973386212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838101d10 0x7fe8381020e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe834004940 tx=0x7fe834009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 -- 172.21.15.63:0/973386212 shutdown_connections 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 --2- 172.21.15.63:0/973386212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8381026b0 0x7fe8380fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 --2- 172.21.15.63:0/973386212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838101d10 0x7fe8381020e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 -- 172.21.15.63:0/973386212 >> 172.21.15.63:0/973386212 conn(0x7fe8380f78e0 msgr2=0x7fe8380f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 -- 172.21.15.63:0/973386212 shutdown_connections 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.694+0000 7fe83fb38700 1 -- 172.21.15.63:0/973386212 wait complete. 2024-04-24T22:56:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 Processor -- start 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 -- start start 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe838101d10 0x7fe838113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8381144f0 con 0x7fe8381026b0 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83fb38700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83810daf0 con 0x7fe838101d10 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83d0d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83d0d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57704/0 (socket says 172.21.15.63:57704) 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83d0d3700 1 -- 172.21.15.63:0/4174540288 learned_addr learned my addr 172.21.15.63:0/4174540288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.695+0000 7fe83d8d4700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe838101d10 0x7fe838113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83d0d3700 1 -- 172.21.15.63:0/4174540288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe838101d10 msgr2=0x7fe838113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83d0d3700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe838101d10 0x7fe838113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83d0d3700 1 -- 172.21.15.63:0/4174540288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834005040 con 0x7fe8381026b0 2024-04-24T22:56:15.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83d0d3700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe828002a50 tx=0x7fe828004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe828007cc0 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82800f040 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83810ddd0 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82800d9c0 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.696+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83810e390 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.697+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8380fd460 con 0x7fe8381026b0 2024-04-24T22:56:15.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.698+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe828009730 con 0x7fe8381026b0 2024-04-24T22:56:15.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.698+0000 7fe82e7fc700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 0x7fe8240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:15.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.698+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe82808d840 con 0x7fe8381026b0 2024-04-24T22:56:15.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.698+0000 7fe83d8d4700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 0x7fe8240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:15.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.699+0000 7fe83d8d4700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 0x7fe8240705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe834004f90 tx=0x7fe834009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:15.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.703+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe82805a6a0 con 0x7fe8381026b0 2024-04-24T22:56:15.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:15 smithi063 ceph-mon[111213]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:15 smithi192 ceph-mon[80463]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.931+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe838108e10 con 0x7fe82406e140 2024-04-24T22:56:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.932+0000 7fe82e7fc700 1 -- 172.21.15.63:0/4174540288 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe838108e10 con 0x7fe82406e140 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 msgr2=0x7fe8240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 0x7fe8240705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe834004f90 tx=0x7fe834009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 msgr2=0x7fe838113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe828002a50 tx=0x7fe828004600 comp rx=0 tx=0).stop 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 shutdown_connections 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe838101d10 0x7fe838113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe82406e140 0x7fe8240705f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 --2- 172.21.15.63:0/4174540288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8381026b0 0x7fe838113ea0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 >> 172.21.15.63:0/4174540288 conn(0x7fe8380f78e0 msgr2=0x7fe8380f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 shutdown_connections 2024-04-24T22:56:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:15.934+0000 7fe83fb38700 1 -- 172.21.15.63:0/4174540288 wait complete. 2024-04-24T22:56:16.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T22:56:17.100 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:17.544 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:17 smithi063 ceph-mon[111213]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:17.545 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:17 smithi063 ceph-mon[111213]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:17 smithi192 ceph-mon[80463]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:17 smithi192 ceph-mon[80463]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 -- 172.21.15.63:0/3749409962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 msgr2=0x7fae4c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 --2- 172.21.15.63:0/3749409962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0fcdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fae40004940 tx=0x7fae40009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 -- 172.21.15.63:0/3749409962 shutdown_connections 2024-04-24T22:56:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 --2- 172.21.15.63:0/3749409962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0fcdf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 --2- 172.21.15.63:0/3749409962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 0x7fae4c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 -- 172.21.15.63:0/3749409962 >> 172.21.15.63:0/3749409962 conn(0x7fae4c0f8990 msgr2=0x7fae4c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 -- 172.21.15.63:0/3749409962 shutdown_connections 2024-04-24T22:56:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.855+0000 7fae51e54700 1 -- 172.21.15.63:0/3749409962 wait complete. 2024-04-24T22:56:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 Processor -- start 2024-04-24T22:56:18.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 -- start start 2024-04-24T22:56:18.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 0x7fae4c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:18.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c0721b0 con 0x7fae4c103780 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae51e54700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c070ce0 con 0x7fae4c102de0 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45848/0 (socket says 172.21.15.63:45848) 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 -- 172.21.15.63:0/3961345740 learned_addr learned my addr 172.21.15.63:0/3961345740 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4b7fe700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 0x7fae4c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 -- 172.21.15.63:0/3961345740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 msgr2=0x7fae4c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:18.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 0x7fae4c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.856+0000 7fae4affd700 1 -- 172.21.15.63:0/3961345740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40005040 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae4affd700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae40009100 tx=0x7fae40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae4000e040 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c070f60 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae40009af0 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae4001c3f0 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.857+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c11a0e0 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.858+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c0fe530 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.859+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fae4000baf0 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.859+0000 7fae33fff700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 0x7fae340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.859+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fae4008edb0 con 0x7fae4c103780 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.859+0000 7fae4b7fe700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 0x7fae340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:18.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.860+0000 7fae4b7fe700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 0x7fae340705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004eb0 tx=0x7fae3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:18.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:18.864+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fae40058160 con 0x7fae4c103780 2024-04-24T22:56:19.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.079+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae4c0fa080 con 0x7fae3406e140 2024-04-24T22:56:19.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.090+0000 7fae33fff700 1 -- 172.21.15.63:0/3961345740 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fae4c0fa080 con 0x7fae3406e140 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (8m) 79s ago 19m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 79s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (5m) 34s ago 18m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 79s ago 19m 79.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (84s) 79s ago 13m 11.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 091ce07d1d7d 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (104s) 79s ago 13m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea388604c60e 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (60s) 34s ago 13m 14.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef e6cfd9a816b1 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (37s) 34s ago 13m 11.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef a318f20d3c35 2024-04-24T22:56:19.092 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (9m) 79s ago 20m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (8m) 34s ago 18m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 79s ago 20m 54.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (6m) 34s ago 18m 44.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 79s ago 19m 15.8M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 34s ago 18m 16.9M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 79s ago 17m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:56:19.093 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 79s ago 17m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:56:19.094 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (3m) 79s ago 16m 76.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:56:19.094 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (3m) 34s ago 16m 103M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:56:19.094 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (2m) 34s ago 15m 91.9M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:56:19.094 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (119s) 34s ago 15m 90.5M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef b7243340f774 2024-04-24T22:56:19.094 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (8m) 79s ago 18m 61.3M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 msgr2=0x7fae340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 0x7fae340705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004eb0 tx=0x7fae3c00b040 comp rx=0 tx=0).stop 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 msgr2=0x7fae4c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae40009100 tx=0x7fae40009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 shutdown_connections 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae4c102de0 0x7fae4c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fae3406e140 0x7fae340705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 --2- 172.21.15.63:0/3961345740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c103780 0x7fae4c0707a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 >> 172.21.15.63:0/3961345740 conn(0x7fae4c0f8990 msgr2=0x7fae4c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 shutdown_connections 2024-04-24T22:56:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:19.092+0000 7fae51e54700 1 -- 172.21.15.63:0/3961345740 wait complete. 2024-04-24T22:56:19.689 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:19 smithi063 ceph-mon[111213]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:19.714 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade status' 2024-04-24T22:56:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:19 smithi192 ceph-mon[80463]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:19.992 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:20.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:20 smithi063 ceph-mon[111213]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:20.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:20 smithi192 ceph-mon[80463]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:21.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:21 smithi063 ceph-mon[111213]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 -- 172.21.15.63:0/1152036686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 msgr2=0x7fc140102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 --2- 172.21.15.63:0/1152036686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc140102160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc130009ce0 tx=0x7fc130004d10 comp rx=0 tx=0).stop 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 -- 172.21.15.63:0/1152036686 shutdown_connections 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 --2- 172.21.15.63:0/1152036686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 0x7fc1400fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 --2- 172.21.15.63:0/1152036686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc140102160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:21.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 -- 172.21.15.63:0/1152036686 >> 172.21.15.63:0/1152036686 conn(0x7fc140071d00 msgr2=0x7fc140072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 -- 172.21.15.63:0/1152036686 shutdown_connections 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.830+0000 7fc147fcb700 1 -- 172.21.15.63:0/1152036686 wait complete. 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 Processor -- start 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 -- start start 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 0x7fc140112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140113720 con 0x7fc140101d90 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc147fcb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14010cb80 con 0x7fc1400fbd40 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc145566700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc145566700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45862/0 (socket says 172.21.15.63:45862) 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc145566700 1 -- 172.21.15.63:0/3232135616 learned_addr learned my addr 172.21.15.63:0/3232135616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:56:21.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.831+0000 7fc145d67700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 0x7fc140112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc145566700 1 -- 172.21.15.63:0/3232135616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 msgr2=0x7fc140112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc145566700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 0x7fc140112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc145566700 1 -- 172.21.15.63:0/3232135616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130009940 con 0x7fc140101d90 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc145566700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc13400fce0 tx=0x7fc1340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc134007ea0 con 0x7fc140101d90 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc13400dda0 con 0x7fc140101d90 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14010ce00 con 0x7fc140101d90 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1340099b0 con 0x7fc140101d90 2024-04-24T22:56:21.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.832+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14010d3c0 con 0x7fc140101d90 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.833+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1400fd430 con 0x7fc140101d90 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.834+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc13400d8d0 con 0x7fc140101d90 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.834+0000 7fc13e7fc700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 0x7fc1280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.834+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc134090220 con 0x7fc140101d90 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.834+0000 7fc145d67700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 0x7fc1280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:21.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.834+0000 7fc145d67700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 0x7fc1280705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b740 tx=0x7fc130004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:21.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:21.839+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc1340595d0 con 0x7fc140101d90 2024-04-24T22:56:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:21 smithi192 ceph-mon[80463]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:22.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.071+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc140108e60 con 0x7fc12806e140 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.072+0000 7fc13e7fc700 1 -- 172.21.15.63:0/3232135616 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc140108e60 con 0x7fc12806e140 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": null, 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": false, 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "", 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": null, 2024-04-24T22:56:22.073 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-04-24T22:56:22.074 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-04-24T22:56:22.074 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:56:22.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 msgr2=0x7fc1280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:22.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 0x7fc1280705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b740 tx=0x7fc130004d10 comp rx=0 tx=0).stop 2024-04-24T22:56:22.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 msgr2=0x7fc1401130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:22.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc13400fce0 tx=0x7fc1340046b0 comp rx=0 tx=0).stop 2024-04-24T22:56:22.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 shutdown_connections 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1400fbd40 0x7fc140112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc12806e140 0x7fc1280705f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 --2- 172.21.15.63:0/3232135616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc140101d90 0x7fc1401130d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 >> 172.21.15.63:0/3232135616 conn(0x7fc140071d00 msgr2=0x7fc140104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 shutdown_connections 2024-04-24T22:56:22.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:22.074+0000 7fc147fcb700 1 -- 172.21.15.63:0/3232135616 wait complete. 2024-04-24T22:56:22.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph health detail' 2024-04-24T22:56:23.017 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:23 smithi192 ceph-mon[80463]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:23.677 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:23 smithi192 ceph-mon[80463]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:23 smithi063 ceph-mon[111213]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:56:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:23 smithi063 ceph-mon[111213]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:24.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/2860401151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c102e50 msgr2=0x7fc39c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/2860401151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c102e50 0x7fc39c103220 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009ce0 tx=0x7fc38c004d10 comp rx=0 tx=0).stop 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/2860401151 shutdown_connections 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/2860401151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c065640 0x7fc39c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/2860401151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c102e50 0x7fc39c103220 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/2860401151 >> 172.21.15.63:0/2860401151 conn(0x7fc39c071d00 msgr2=0x7fc39c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/2860401151 shutdown_connections 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.720+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/2860401151 wait complete. 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 Processor -- start 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 -- start start 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c102e50 0x7fc39c0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0fc910 con 0x7fc39c065640 2024-04-24T22:56:24.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc3a1a2b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0fca50 con 0x7fc39c102e50 2024-04-24T22:56:24.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc39affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:24.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc39affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45892/0 (socket says 172.21.15.63:45892) 2024-04-24T22:56:24.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc39affd700 1 -- 172.21.15.63:0/3763133673 learned_addr learned my addr 172.21.15.63:0/3763133673 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:56:24.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.721+0000 7fc39a7fc700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c102e50 0x7fc39c0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc39affd700 1 -- 172.21.15.63:0/3763133673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c102e50 msgr2=0x7fc39c0fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc39affd700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c102e50 0x7fc39c0fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc39affd700 1 -- 172.21.15.63:0/3763133673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009940 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc39affd700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005010 tx=0x7fc38c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38c00e040 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc38c008da0 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38c01c620 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c11a010 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.722+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c11a490 con 0x7fc39c065640 2024-04-24T22:56:24.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.723+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39c0fe540 con 0x7fc39c065640 2024-04-24T22:56:24.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.724+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc38c012a70 con 0x7fc39c065640 2024-04-24T22:56:24.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.724+0000 7fc3937fe700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 0x7fc388070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:24.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.725+0000 7fc39a7fc700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 0x7fc388070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:24.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.725+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc38c08f6a0 con 0x7fc39c065640 2024-04-24T22:56:24.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.725+0000 7fc39a7fc700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 0x7fc388070600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0fd6f0 tx=0x7fc3840091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:24.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:24.729+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc38c0589d0 con 0x7fc39c065640 2024-04-24T22:56:25.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.021+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc39c0fd4a0 con 0x7fc39c065640 2024-04-24T22:56:25.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.021+0000 7fc3937fe700 1 -- 172.21.15.63:0/3763133673 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc38c05c090 con 0x7fc39c065640 2024-04-24T22:56:25.023 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T22:56:25.023 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-24T22:56:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-24T22:56:25.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 msgr2=0x7fc388070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 0x7fc388070600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0fd6f0 tx=0x7fc3840091a0 comp rx=0 tx=0).stop 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 msgr2=0x7fc39c0fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc38c005010 tx=0x7fc38c004d10 comp rx=0 tx=0).stop 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 shutdown_connections 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc38806e150 0x7fc388070600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc39c065640 0x7fc39c0fbdb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 --2- 172.21.15.63:0/3763133673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc39c102e50 0x7fc39c0fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 >> 172.21.15.63:0/3763133673 conn(0x7fc39c071d00 msgr2=0x7fc39c0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 shutdown_connections 2024-04-24T22:56:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:25.023+0000 7fc3a1a2b700 1 -- 172.21.15.63:0/3763133673 wait complete. 2024-04-24T22:56:25.670 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:25 smithi063 ceph-mon[111213]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:25.670 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:25 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3763133673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:56:25.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-24T22:56:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:25 smithi192 ceph-mon[80463]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:25 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3763133673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T22:56:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:27.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:27 smithi063 ceph-mon[111213]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 -- 172.21.15.63:0/933460905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5824101d10 msgr2=0x7f58241020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 --2- 172.21.15.63:0/933460905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5824101d10 0x7f58241020e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5814004940 tx=0x7f5814009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 -- 172.21.15.63:0/933460905 shutdown_connections 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 --2- 172.21.15.63:0/933460905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58241026b0 0x7f58240fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 --2- 172.21.15.63:0/933460905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5824101d10 0x7f58241020e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 -- 172.21.15.63:0/933460905 >> 172.21.15.63:0/933460905 conn(0x7f58240f7860 msgr2=0x7f58240f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 -- 172.21.15.63:0/933460905 shutdown_connections 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.759+0000 7f582bed7700 1 -- 172.21.15.63:0/933460905 wait complete. 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 Processor -- start 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 -- start start 2024-04-24T22:56:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5824101d10 0x7f5824113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5824114590 con 0x7f58241026b0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f582bed7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582410db90 con 0x7f5824101d10 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f5829472700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f5829472700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45922/0 (socket says 172.21.15.63:45922) 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f5829472700 1 -- 172.21.15.63:0/741655747 learned_addr learned my addr 172.21.15.63:0/741655747 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.760+0000 7f5829c73700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5824101d10 0x7f5824113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f5829472700 1 -- 172.21.15.63:0/741655747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5824101d10 msgr2=0x7f5824113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f5829472700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5824101d10 0x7f5824113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f5829472700 1 -- 172.21.15.63:0/741655747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814005040 con 0x7f58241026b0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f5829472700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f582000c800 tx=0x7f5820004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582000a760 con 0x7f58241026b0 2024-04-24T22:56:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f582000ad90 con 0x7f58241026b0 2024-04-24T22:56:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582410de70 con 0x7f58241026b0 2024-04-24T22:56:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58200113a0 con 0x7f58241026b0 2024-04-24T22:56:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.761+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582410e430 con 0x7f58241026b0 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.762+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58240fd460 con 0x7f58241026b0 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.763+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5820012400 con 0x7f58241026b0 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.763+0000 7f581a7fc700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 0x7f58100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.763+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f582008dba0 con 0x7f58241026b0 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.763+0000 7f5829c73700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 0x7f58100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:56:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.764+0000 7f5829c73700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 0x7f58100705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f581400b3b0 tx=0x7f5814009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:56:27.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:27.768+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f582005aa80 con 0x7f58241026b0 2024-04-24T22:56:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:27 smithi192 ceph-mon[80463]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.065+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5824108e40 con 0x7f58241026b0 2024-04-24T22:56:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.065+0000 7f581a7fc700 1 -- 172.21.15.63:0/741655747 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f582005a610 con 0x7f58241026b0 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:56:28.068 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:56:28.069 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.067+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 msgr2=0x7f58100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.067+0000 7f582bed7700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 0x7f58100705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f581400b3b0 tx=0x7f5814009e30 comp rx=0 tx=0).stop 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.067+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 msgr2=0x7f5824113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.067+0000 7f582bed7700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f582000c800 tx=0x7f5820004470 comp rx=0 tx=0).stop 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 shutdown_connections 2024-04-24T22:56:28.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5824101d10 0x7f5824113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:28.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f581006e140 0x7f58100705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:28.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 --2- 172.21.15.63:0/741655747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58241026b0 0x7f5824113f40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:56:28.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 >> 172.21.15.63:0/741655747 conn(0x7f58240f7860 msgr2=0x7f58240f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:56:28.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 shutdown_connections 2024-04-24T22:56:28.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:56:28.068+0000 7f582bed7700 1 -- 172.21.15.63:0/741655747 wait complete. 2024-04-24T22:56:28.659 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:28 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/741655747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:56:28.684 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-24T22:56:28.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:28 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/741655747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:56:28.952 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:29.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:29 smithi063 ceph-mon[111213]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:29 smithi192 ceph-mon[80463]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:30.586 INFO:teuthology.orchestra.run.smithi063.stdout:wait for servicemap items w/ changing names to refresh 2024-04-24T22:56:30.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:31.260 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'sleep 60' 2024-04-24T22:56:31.531 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:56:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:31 smithi063 ceph-mon[111213]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:31 smithi192 ceph-mon[80463]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:33.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:33 smithi063 ceph-mon[111213]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:33 smithi192 ceph-mon[80463]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:35 smithi063 ceph-mon[111213]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:35 smithi192 ceph-mon[80463]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:37 smithi063 ceph-mon[111213]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:37 smithi192 ceph-mon[80463]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:39 smithi063 ceph-mon[111213]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:39 smithi192 ceph-mon[80463]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:41.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:41 smithi063 ceph-mon[111213]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:41.926 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:41 smithi192 ceph-mon[80463]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:43 smithi063 ceph-mon[111213]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:43 smithi192 ceph-mon[80463]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:45 smithi063 ceph-mon[111213]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:45.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:45 smithi192 ceph-mon[80463]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:56:47.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:47 smithi063 ceph-mon[111213]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:47 smithi192 ceph-mon[80463]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:49.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:49 smithi063 ceph-mon[111213]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:49 smithi192 ceph-mon[80463]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:51 smithi063 ceph-mon[111213]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:56:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:51.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:51 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:51 smithi192 ceph-mon[80463]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:56:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:51 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:52.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:56:52.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:56:52.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:56:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:56:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:56:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:53 smithi192 ceph-mon[80463]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:54.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:53 smithi063 ceph-mon[111213]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:55 smithi192 ceph-mon[80463]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:56.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:55 smithi063 ceph-mon[111213]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:58.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:57 smithi063 ceph-mon[111213]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:58.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:57 smithi192 ceph-mon[80463]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:59.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:56:58 smithi063 ceph-mon[111213]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:56:59.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:56:58 smithi192 ceph-mon[80463]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:01 smithi192 ceph-mon[80463]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:01.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:01 smithi063 ceph-mon[111213]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:03.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:03 smithi063 ceph-mon[111213]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:03 smithi192 ceph-mon[80463]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:05 smithi063 ceph-mon[111213]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:05 smithi192 ceph-mon[80463]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:07 smithi063 ceph-mon[111213]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:07.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:07 smithi192 ceph-mon[80463]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:09.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:09 smithi063 ceph-mon[111213]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:09 smithi192 ceph-mon[80463]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:11.729 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:11 smithi192 ceph-mon[80463]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:11 smithi063 ceph-mon[111213]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:13 smithi063 ceph-mon[111213]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:13 smithi192 ceph-mon[80463]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:15.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:15 smithi063 ceph-mon[111213]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:15.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:15 smithi192 ceph-mon[80463]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:17.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:17 smithi063 ceph-mon[111213]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:17.926 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:17 smithi192 ceph-mon[80463]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:19 smithi063 ceph-mon[111213]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:19.926 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:19 smithi192 ceph-mon[80463]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:21 smithi063 ceph-mon[111213]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:21 smithi192 ceph-mon[80463]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:23.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:23 smithi063 ceph-mon[111213]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:23 smithi192 ceph-mon[80463]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:25 smithi063 ceph-mon[111213]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:25 smithi192 ceph-mon[80463]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:27 smithi063 ceph-mon[111213]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:27 smithi192 ceph-mon[80463]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:29 smithi063 ceph-mon[111213]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:29 smithi192 ceph-mon[80463]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:30.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:31.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:31 smithi063 ceph-mon[111213]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:31 smithi192 ceph-mon[80463]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:33.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:33 smithi063 ceph-mon[111213]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:33 smithi192 ceph-mon[80463]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:34.022 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T22:57:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:57:35.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:35 smithi063 ceph-mon[111213]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:35 smithi192 ceph-mon[80463]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.116+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3670418190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 msgr2=0x7f2cf80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.116+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3670418190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf80fcdf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cec004940 tx=0x7f2cec009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3670418190 shutdown_connections 2024-04-24T22:57:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3670418190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf80fcdf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3670418190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 0x7f2cf81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3670418190 >> 172.21.15.63:0/3670418190 conn(0x7f2cf80f89b0 msgr2=0x7f2cf80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3670418190 shutdown_connections 2024-04-24T22:57:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3670418190 wait complete. 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 Processor -- start 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.117+0000 7f2cfce9d700 1 -- start start 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cfce9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 0x7f2cf8113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cfce9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cfce9d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8114640 con 0x7f2cf8103780 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cfce9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf810da30 con 0x7f2cf8102de0 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45554/0 (socket says 172.21.15.63:45554) 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 -- 172.21.15.63:0/3294776887 learned_addr learned my addr 172.21.15.63:0/3294776887 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf659c700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 0x7f2cf8113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 -- 172.21.15.63:0/3294776887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 msgr2=0x7f2cf8113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 0x7f2cf8113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 -- 172.21.15.63:0/3294776887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec005040 con 0x7f2cf8103780 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.118+0000 7f2cf5d9b700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cec000c00 tx=0x7f2cec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.119+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cec021070 con 0x7f2cf8103780 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.119+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cec013cd0 con 0x7f2cf8103780 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.119+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf810dcb0 con 0x7f2cf8103780 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.119+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cec00e070 con 0x7f2cf8103780 2024-04-24T22:57:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.119+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf810e190 con 0x7f2cf8103780 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.120+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf80fe530 con 0x7f2cf8103780 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.120+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2cec01caf0 con 0x7f2cf8103780 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.121+0000 7f2ce6ffd700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 0x7f2ce00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.121+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2cec08ef70 con 0x7f2cf8103780 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.121+0000 7f2cf659c700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 0x7f2ce00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.121+0000 7f2cf659c700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 0x7f2ce00705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8004600 tx=0x7f2ce800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.126+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2cec0c3740 con 0x7f2cf8103780 2024-04-24T22:57:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.343+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 --> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cf8109ec0 con 0x7f2ce006e140 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.353+0000 7f2ce6ffd700 1 -- 172.21.15.63:0/3294776887 <== mgr.34104 v2:172.21.15.63:6800/3731828681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2cf8109ec0 con 0x7f2ce006e140 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (9m) 2m ago 20m 26.3M - 0.25.0 c8568f914cd2 e532a397c3f7 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 2m ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ff33ce6712a 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi192 smithi192 running (6m) 111s ago 19m 6665k - 19.0.0-2445-gd2c45d97 f68e37b807ef 42e8f1155fd4 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (9m) 2m ago 20m 79.8M - 9.4.12 2bacad6d85d8 ed7e87fa8599 2024-04-24T22:57:36.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.dnliiv smithi063 running (2m) 2m ago 15m 11.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 091ce07d1d7d 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.twfoxu smithi063 running (3m) 2m ago 15m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea388604c60e 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.mnfuxk smithi192 running (2m) 111s ago 15m 14.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef e6cfd9a816b1 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi192.ozemqi smithi192 running (114s) 111s ago 14m 11.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef a318f20d3c35 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.upcnvd smithi063 *:8443,9283,8765 running (10m) 2m ago 22m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3195f20e5b42 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi192.ctmwwc smithi192 *:8443,9283,8765 running (10m) 111s ago 19m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 73795f1725b4 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (7m) 2m ago 22m 54.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 780ec798c65b 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi192 smithi192 running (7m) 111s ago 19m 44.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1bfbbd31af16 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (10m) 2m ago 20m 15.8M - 1.5.0 0da6a335fe13 72bbf8ce7f83 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi192 smithi192 *:9100 running (9m) 111s ago 19m 16.9M - 1.5.0 0da6a335fe13 4d23a28b008f 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 18m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 4e96a1c73c3f 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 2m ago 18m 94.9M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 40c2a390f714 2024-04-24T22:57:36.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 2m ago 17m 76.5M 1557M 19.0.0-2445-gd2c45d97 f68e37b807ef 53024f97d574 2024-04-24T22:57:36.357 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi192 running (4m) 111s ago 17m 103M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 8d7ee6c98238 2024-04-24T22:57:36.357 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi192 running (3m) 111s ago 17m 91.9M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 713be80c3f24 2024-04-24T22:57:36.357 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi192 running (3m) 111s ago 16m 90.5M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef b7243340f774 2024-04-24T22:57:36.357 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (9m) 2m ago 20m 61.3M - 2.43.0 a07b618ecd1d 6563a9de44ff 2024-04-24T22:57:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 msgr2=0x7f2ce00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 0x7f2ce00705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8004600 tx=0x7f2ce800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 msgr2=0x7f2cf8113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cec000c00 tx=0x7f2cec009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 shutdown_connections 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf8102de0 0x7f2cf8113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ce006e140 0x7f2ce00705f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 --2- 172.21.15.63:0/3294776887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf8103780 0x7f2cf8113ff0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.355+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 >> 172.21.15.63:0/3294776887 conn(0x7f2cf80f89b0 msgr2=0x7f2cf80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.356+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 shutdown_connections 2024-04-24T22:57:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:36.356+0000 7f2cfce9d700 1 -- 172.21.15.63:0/3294776887 wait complete. 2024-04-24T22:57:37.051 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-24T22:57:37.324 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:57:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:37 smithi063 ceph-mon[111213]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:57:37.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:37 smithi063 ceph-mon[111213]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:37 smithi192 ceph-mon[80463]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T22:57:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:37 smithi192 ceph-mon[80463]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- 172.21.15.63:0/3394198882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 msgr2=0x7eff7c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 --2- 172.21.15.63:0/3394198882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c1031c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff78004940 tx=0x7eff78009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- 172.21.15.63:0/3394198882 shutdown_connections 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 --2- 172.21.15.63:0/3394198882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 0x7eff7c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 --2- 172.21.15.63:0/3394198882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c1031c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- 172.21.15.63:0/3394198882 >> 172.21.15.63:0/3394198882 conn(0x7eff7c0f8980 msgr2=0x7eff7c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:38.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- 172.21.15.63:0/3394198882 shutdown_connections 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- 172.21.15.63:0/3394198882 wait complete. 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 Processor -- start 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.993+0000 7eff83c82700 1 -- start start 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff83c82700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff83c82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 0x7eff7c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:38.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff83c82700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c070d30 con 0x7eff7c102df0 2024-04-24T22:57:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff83c82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c070e70 con 0x7eff7c103790 2024-04-24T22:57:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58060/0 (socket says 172.21.15.63:58060) 2024-04-24T22:57:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 -- 172.21.15.63:0/3255527276 learned_addr learned my addr 172.21.15.63:0/3255527276 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:38.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff8121d700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 0x7eff7c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 -- 172.21.15.63:0/3255527276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 msgr2=0x7eff7c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 0x7eff7c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 -- 172.21.15.63:0/3255527276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78005040 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff81a1e700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7eff7800b3b0 tx=0x7eff78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.994+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7800e040 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.995+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff7800bbc0 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.995+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0710c0 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.995+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7801c3f0 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.995+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c11a2b0 con 0x7eff7c102df0 2024-04-24T22:57:38.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.996+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c100100 con 0x7eff7c102df0 2024-04-24T22:57:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.997+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff7800bd30 con 0x7eff7c102df0 2024-04-24T22:57:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.997+0000 7eff727fc700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 0x7eff680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.997+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7eff7808edf0 con 0x7eff7c102df0 2024-04-24T22:57:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.997+0000 7eff8121d700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 0x7eff680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:38.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:38.997+0000 7eff8121d700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 0x7eff680705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7eff6c007fd0 tx=0x7eff6c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:39.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.002+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7eff78058120 con 0x7eff7c102df0 2024-04-24T22:57:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.299+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7eff7c04bed0 con 0x7eff7c102df0 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.299+0000 7eff727fc700 1 -- 172.21.15.63:0/3255527276 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7eff7805b7e0 con 0x7eff7c102df0 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-04-24T22:57:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-04-24T22:57:39.302 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 msgr2=0x7eff680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 0x7eff680705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7eff6c007fd0 tx=0x7eff6c009040 comp rx=0 tx=0).stop 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 msgr2=0x7eff7c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7eff7800b3b0 tx=0x7eff78009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 shutdown_connections 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7eff6806e140 0x7eff680705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff7c102df0 0x7eff7c070260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 --2- 172.21.15.63:0/3255527276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff7c103790 0x7eff7c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.301+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 >> 172.21.15.63:0/3255527276 conn(0x7eff7c0f8980 msgr2=0x7eff7c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.302+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 shutdown_connections 2024-04-24T22:57:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:39.302+0000 7eff83c82700 1 -- 172.21.15.63:0/3255527276 wait complete. 2024-04-24T22:57:39.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:39 smithi063 ceph-mon[111213]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:39.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:39 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3255527276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:39 smithi192 ceph-mon[80463]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:39 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3255527276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:39.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-24T22:57:40.208 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:57:41.573 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:41 smithi063 ceph-mon[111213]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 -- 172.21.15.63:0/3192908475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 msgr2=0x7f1588070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 --2- 172.21.15.63:0/3192908475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588070670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f157c004970 tx=0x7f157c009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 -- 172.21.15.63:0/3192908475 shutdown_connections 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 --2- 172.21.15.63:0/3192908475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 0x7f1588109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 --2- 172.21.15.63:0/3192908475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588070670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 -- 172.21.15.63:0/3192908475 >> 172.21.15.63:0/3192908475 conn(0x7f15880f8990 msgr2=0x7f15880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 -- 172.21.15.63:0/3192908475 shutdown_connections 2024-04-24T22:57:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.798+0000 7f158ee4e700 1 -- 172.21.15.63:0/3192908475 wait complete. 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 Processor -- start 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 -- start start 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 0x7f1588115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881165f0 con 0x7f15880702a0 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158ee4e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810fc80 con 0x7f1588070bb0 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58068/0 (socket says 172.21.15.63:58068) 2024-04-24T22:57:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 -- 172.21.15.63:0/2171914226 learned_addr learned my addr 172.21.15.63:0/2171914226 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f1587fff700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 0x7f1588115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 -- 172.21.15.63:0/2171914226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 msgr2=0x7f1588115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 0x7f1588115fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.799+0000 7f158cbea700 1 -- 172.21.15.63:0/2171914226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c005040 con 0x7f15880702a0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f158cbea700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f157c005010 tx=0x7f157c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f157c00e040 con 0x7f15880702a0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f157c009950 con 0x7f15880702a0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158810ff00 con 0x7f15880702a0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f157c01c3f0 con 0x7f15880702a0 2024-04-24T22:57:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.800+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1588110440 con 0x7f15880702a0 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.801+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880719e0 con 0x7f15880702a0 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.801+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f157c009480 con 0x7f15880702a0 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.802+0000 7f15857fa700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 0x7f15780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.802+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f157c08ec80 con 0x7f15880702a0 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.802+0000 7f1587fff700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 0x7f15780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.802+0000 7f1587fff700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 0x7f15780705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1574004600 tx=0x7f157400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:41.807+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f157c0580f0 con 0x7f15880702a0 2024-04-24T22:57:41.926 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:41 smithi192 ceph-mon[80463]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:42.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.103+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f158810c0d0 con 0x7f15880702a0 2024-04-24T22:57:42.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.103+0000 7f15857fa700 1 -- 172.21.15.63:0/2171914226 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f157c05b7b0 con 0x7f15880702a0 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.105+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 msgr2=0x7f15780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.105+0000 7f158ee4e700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 0x7f15780705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1574004600 tx=0x7f157400b040 comp rx=0 tx=0).stop 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.105+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 msgr2=0x7f1588115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.105+0000 7f158ee4e700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f157c005010 tx=0x7f157c009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 shutdown_connections 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f157806e140 0x7f15780705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15880702a0 0x7f1588115a60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:42.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 --2- 172.21.15.63:0/2171914226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1588070bb0 0x7f1588115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:42.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 >> 172.21.15.63:0/2171914226 conn(0x7f15880f8990 msgr2=0x7f15880f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:42.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 shutdown_connections 2024-04-24T22:57:42.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:42.106+0000 7f158ee4e700 1 -- 172.21.15.63:0/2171914226 wait complete. 2024-04-24T22:57:42.117 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-04-24T22:57:42.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:42 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2171914226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:42.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-24T22:57:42.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:42 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2171914226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:43.131 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:57:43.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:43 smithi063 ceph-mon[111213]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:43 smithi192 ceph-mon[80463]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 -- 172.21.15.63:0/4181842892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 msgr2=0x7fc0940fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 --2- 172.21.15.63:0/4181842892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc0940fe1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc088004940 tx=0x7fc088009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 -- 172.21.15.63:0/4181842892 shutdown_connections 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 --2- 172.21.15.63:0/4181842892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 0x7fc094106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 --2- 172.21.15.63:0/4181842892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc0940fe1e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 -- 172.21.15.63:0/4181842892 >> 172.21.15.63:0/4181842892 conn(0x7fc0940f7860 msgr2=0x7fc0940f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 -- 172.21.15.63:0/4181842892 shutdown_connections 2024-04-24T22:57:44.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.838+0000 7fc09988d700 1 -- 172.21.15.63:0/4181842892 wait complete. 2024-04-24T22:57:44.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 Processor -- start 2024-04-24T22:57:44.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 -- start start 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 0x7fc094113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0941145a0 con 0x7fc0940fde10 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc09988d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410dba0 con 0x7fc0940fe7b0 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc092ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc092ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58074/0 (socket says 172.21.15.63:58074) 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.839+0000 7fc092ffd700 1 -- 172.21.15.63:0/3246680826 learned_addr learned my addr 172.21.15.63:0/3246680826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc0927fc700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 0x7fc094113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc092ffd700 1 -- 172.21.15.63:0/3246680826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 msgr2=0x7fc094113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc092ffd700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 0x7fc094113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc092ffd700 1 -- 172.21.15.63:0/3246680826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc088005040 con 0x7fc0940fde10 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc092ffd700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc088009380 tx=0x7fc088009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc08800e040 con 0x7fc0940fde10 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc088009af0 con 0x7fc0940fde10 2024-04-24T22:57:44.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09410de20 con 0x7fc0940fde10 2024-04-24T22:57:44.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc08801c630 con 0x7fc0940fde10 2024-04-24T22:57:44.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.840+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09410e360 con 0x7fc0940fde10 2024-04-24T22:57:44.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.841+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094066b50 con 0x7fc0940fde10 2024-04-24T22:57:44.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.842+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc08800bbc0 con 0x7fc0940fde10 2024-04-24T22:57:44.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.842+0000 7fc07b7fe700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 0x7fc07c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:44.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.842+0000 7fc0927fc700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 0x7fc07c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:44.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.842+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc08808eeb0 con 0x7fc0940fde10 2024-04-24T22:57:44.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.843+0000 7fc0927fc700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 0x7fc07c0705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc084007fd0 tx=0x7fc084009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:44.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:44.847+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc08805bd90 con 0x7fc0940fde10 2024-04-24T22:57:45.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.140+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc09410eff0 con 0x7fc0940fde10 2024-04-24T22:57:45.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.141+0000 7fc07b7fe700 1 -- 172.21.15.63:0/3246680826 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc08805b920 con 0x7fc0940fde10 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.143+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 msgr2=0x7fc07c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.143+0000 7fc09988d700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 0x7fc07c0705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc084007fd0 tx=0x7fc084009040 comp rx=0 tx=0).stop 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.143+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 msgr2=0x7fc094113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.143+0000 7fc09988d700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc088009380 tx=0x7fc088009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 shutdown_connections 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc07c06e140 0x7fc07c0705f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:45.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0940fde10 0x7fc094113a10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:45.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 --2- 172.21.15.63:0/3246680826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0940fe7b0 0x7fc094113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:45.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 >> 172.21.15.63:0/3246680826 conn(0x7fc0940f7860 msgr2=0x7fc0940f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:45.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 shutdown_connections 2024-04-24T22:57:45.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:45.144+0000 7fc09988d700 1 -- 172.21.15.63:0/3246680826 wait complete. 2024-04-24T22:57:45.157 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-24T22:57:45.783 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:45 smithi063 ceph-mon[111213]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:45.783 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:45.783 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:45 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3246680826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:45.808 DEBUG:teuthology.parallel:result is None 2024-04-24T22:57:45.808 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T22:57:45.821 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T22:57:45.821 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8a2676e-028a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T22:57:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:45 smithi192 ceph-mon[80463]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:57:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:45 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3246680826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T22:57:46.091 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/mon.smithi063/config 2024-04-24T22:57:47.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:47 smithi063 ceph-mon[111213]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 -- 172.21.15.63:0/3027582749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab80702a0 msgr2=0x7fcab8070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/3027582749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab80702a0 0x7fcab8070670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcab4004970 tx=0x7fcab4009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 -- 172.21.15.63:0/3027582749 shutdown_connections 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/3027582749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab8070bb0 0x7fcab8109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/3027582749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab80702a0 0x7fcab8070670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:47.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 -- 172.21.15.63:0/3027582749 >> 172.21.15.63:0/3027582749 conn(0x7fcab80f8990 msgr2=0x7fcab80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 -- 172.21.15.63:0/3027582749 shutdown_connections 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 -- 172.21.15.63:0/3027582749 wait complete. 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.831+0000 7fcabfcc6700 1 Processor -- start 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabfcc6700 1 -- start start 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabfcc6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 0x7fcab8115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabfcc6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabfcc6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab81166a0 con 0x7fcab8070bb0 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabfcc6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab810fc80 con 0x7fcab80702a0 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58092/0 (socket says 172.21.15.63:58092) 2024-04-24T22:57:47.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 -- 172.21.15.63:0/2140262703 learned_addr learned my addr 172.21.15.63:0/2140262703 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabda62700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 0x7fcab8115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 -- 172.21.15.63:0/2140262703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 msgr2=0x7fcab8115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 0x7fcab8115b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 -- 172.21.15.63:0/2140262703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4005040 con 0x7fcab8070bb0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabda62700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 0x7fcab8115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.832+0000 7fcabd261700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8002a50 tx=0x7fcaa80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.833+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa8007cc0 con 0x7fcab8070bb0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.833+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaa800f040 con 0x7fcab8070bb0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.833+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab810ff60 con 0x7fcab8070bb0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.833+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa8009640 con 0x7fcab8070bb0 2024-04-24T22:57:47.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.833+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab81104a0 con 0x7fcab8070bb0 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.834+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab80719e0 con 0x7fcab8070bb0 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.834+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcaa800d690 con 0x7fcab8070bb0 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.835+0000 7fcaae7fc700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 0x7fcaa40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.835+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcaa808d890 con 0x7fcab8070bb0 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.835+0000 7fcabda62700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 0x7fcaa40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:47.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.835+0000 7fcabda62700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 0x7fcaa40705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:47.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:47.840+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcaa805a770 con 0x7fcab8070bb0 2024-04-24T22:57:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:47 smithi192 ceph-mon[80463]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:48.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.092+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcab804c710 con 0x7fcab8070bb0 2024-04-24T22:57:48.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.093+0000 7fcaae7fc700 1 -- 172.21.15.63:0/2140262703 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1937 (secure 0 0 0) 0x7fcaa805a300 con 0x7fcab8070bb0 2024-04-24T22:57:48.094 INFO:teuthology.orchestra.run.smithi063.stdout:e31 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 31 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-04-24T22:42:22.073567+0000 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-04-24T22:55:49.318934+0000 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-04-24T22:57:48.095 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 83 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34374} 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-04-24T22:57:48.096 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data enabled 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34374 members: 34374 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.twfoxu{0:34374} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.63:6826/995129537,v1:172.21.15.63:6827/995129537] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.dnliiv{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/3118675914,v1:172.21.15.63:6829/3118675914] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.mnfuxk{-1:44309} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3626023513,v1:172.21.15.192:6825/3626023513] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:57:48.097 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi192.ozemqi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3112021694,v1:172.21.15.192:6827/3112021694] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 msgr2=0x7fcaa40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 0x7fcaa40705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab4009e30 comp rx=0 tx=0).stop 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 msgr2=0x7fcab8116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8002a50 tx=0x7fcaa80046b0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 shutdown_connections 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab80702a0 0x7fcab8115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fcaa406e140 0x7fcaa40705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 --2- 172.21.15.63:0/2140262703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcab8070bb0 0x7fcab8116050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 >> 172.21.15.63:0/2140262703 conn(0x7fcab80f8990 msgr2=0x7fcab80f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 shutdown_connections 2024-04-24T22:57:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.095+0000 7fcabfcc6700 1 -- 172.21.15.63:0/2140262703 wait complete. 2024-04-24T22:57:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 31 2024-04-24T22:57:48.747 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:48 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2140262703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:57:48.773 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-24T22:57:48.785 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.884+0000 7fbc3902e640 1 -- 172.21.15.63:0/4125880321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102b20 msgr2=0x7fbc340fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.884+0000 7fbc3902e640 1 --2- 172.21.15.63:0/4125880321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102b20 0x7fbc340fa7f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0048d0 tx=0x7fbc1c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- 172.21.15.63:0/4125880321 shutdown_connections 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 --2- 172.21.15.63:0/4125880321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102b20 0x7fbc340fa7f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 --2- 172.21.15.63:0/4125880321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102210 0x7fbc341025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- 172.21.15.63:0/4125880321 >> 172.21.15.63:0/4125880321 conn(0x7fbc34071770 msgr2=0x7fbc34071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:48.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- 172.21.15.63:0/4125880321 shutdown_connections 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- 172.21.15.63:0/4125880321 wait complete. 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 Processor -- start 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- start start 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 0x7fbc341132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc341138c0 con 0x7fbc34102210 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.885+0000 7fbc3902e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3410cf10 con 0x7fbc34102b20 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33080/0 (socket says 172.21.15.63:33080) 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 -- 172.21.15.63:0/520785278 learned_addr learned my addr 172.21.15.63:0/520785278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc3259c640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 0x7fbc341132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 -- 172.21.15.63:0/520785278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 msgr2=0x7fbc341132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 0x7fbc341132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 -- 172.21.15.63:0/520785278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c005040 con 0x7fbc34102210 2024-04-24T22:57:48.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc3259c640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 0x7fbc341132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc32d9d640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c790 tx=0x7fbc240043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2400a820 con 0x7fbc34102210 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc2400ae50 con 0x7fbc34102210 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3410d1f0 con 0x7fbc34102210 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc240115e0 con 0x7fbc34102210 2024-04-24T22:57:48.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.886+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3410d6e0 con 0x7fbc34102210 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.887+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc340fdb10 con 0x7fbc34102210 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.888+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbc24012400 con 0x7fbc34102210 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.888+0000 7fbc2b7fe640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 0x7fbc20070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.888+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbc2408dbe0 con 0x7fbc34102210 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.888+0000 7fbc3259c640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 0x7fbc20070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:48.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.889+0000 7fbc3259c640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 0x7fbc20070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc34102ff0 tx=0x7fbc1c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:48.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:48.893+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbc24057050 con 0x7fbc34102210 2024-04-24T22:57:48.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:48 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2140262703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T22:57:49.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.079+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc3410e3f0 con 0x7fbc34102210 2024-04-24T22:57:49.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.081+0000 7fbc2b7fe640 1 -- 172.21.15.63:0/520785278 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16661 (secure 0 0 0) 0x7fbc2405a710 con 0x7fbc34102210 2024-04-24T22:57:49.082 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:49.083 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":83,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:55:33.427955+0000","last_up_change":"2024-04-24T22:54:27.067935+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T22:42:21.011439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T22:42:21.501399+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6803","nonce":3067429146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6805","nonce":3067429146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6809","nonce":3067429146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6807","nonce":3067429146}]},"public_addr":"172.21.15.63:6803/3067429146","cluster_addr":"172.21.15.63:6805/3067429146","heartbeat_back_addr":"172.21.15.63:6809/3067429146","heartbeat_front_addr":"172.21.15.63:6807/3067429146","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6811","nonce":2869549179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6813","nonce":2869549179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6817","nonce":2869549179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6815","nonce":2869549179}]},"public_addr":"172.21.15.63:6811/2869549179","cluster_addr":"172.21.15.63:6813/2869549179","heartbeat_back_addr":"172.21.15.63:6817/2869549179","heartbeat_front_addr":"172.21.15.63:6815/2869549179","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6819","nonce":2703892347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6821","nonce":2703892347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6825","nonce":2703892347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6823","nonce":2703892347}]},"public_addr":"172.21.15.63:6819/2703892347","cluster_addr":"172.21.15.63:6821/2703892347","heartbeat_back_addr":"172.21.15.63:6825/2703892347","heartbeat_front_addr":"172.21.15.63:6823/2703892347","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6801","nonce":534042226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6803","nonce":534042226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6807","nonce":534042226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6805","nonce":534042226}]},"public_addr":"172.21.15.192:6801/534042226","cluster_addr":"172.21.15.192:6803/534042226","heartbeat_back_addr":"172.21.15.192:6807/534042226","heartbeat_front_addr":"172.21.15.192:6805/534042226","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6809","nonce":2986357084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6811","nonce":2986357084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6815","nonce":2986357084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6813","nonce":2986357084}]},"public_addr":"172.21.15.192:6809/2986357084","cluster_addr":"172.21.15.192:6811/2986357084","heartbeat_back_addr":"172.21.15.192:6815/2986357084","heartbeat_front_addr":"172.21.15.192:6813/2986357084","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6817","nonce":3828556210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6819","nonce":3828556210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6823","nonce":3828556210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6821","nonce":3828556210}]},"public_addr":"172.21.15.192:6817/3828556210","cluster_addr":"172.21.15.192:6819/3828556210","heartbeat_back_addr":"172.21.15.192:6823/3828556210","heartbeat_front_addr":"172.21.15.192:6821/3828556210","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6827/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6826/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6825/780449385":"2024-04-25T22:55:08.099739+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/4263661376":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2702451571":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2342719260":"2024-04-25T22:45:09.797346+0000","172.21.15.192:0/2604587148":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.192:0/370487052":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/2143811635":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/3666997364":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1700011071":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/3056400683":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6826/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:6801/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6827/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:0/3239723549":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/3265850425":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.192:6828/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.192:6829/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:6801/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/1398662955":"2024-04-25T22:50:00.020010+0000","172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.192:6824/780449385":"2024-04-25T22:55:08.099739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:57:49.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 msgr2=0x7fbc20070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 0x7fbc20070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc34102ff0 tx=0x7fbc1c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 msgr2=0x7fbc34112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c790 tx=0x7fbc240043c0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 shutdown_connections 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fbc2006e0b0 0x7fbc20070560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc34102b20 0x7fbc341132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 --2- 172.21.15.63:0/520785278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc34102210 0x7fbc34112d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 >> 172.21.15.63:0/520785278 conn(0x7fbc34071770 msgr2=0x7fbc340f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 shutdown_connections 2024-04-24T22:57:49.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.083+0000 7fbc3902e640 1 -- 172.21.15.63:0/520785278 wait complete. 2024-04-24T22:57:49.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:54.076844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T22:42:21.011439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T22:42:21.501399+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:57:49.096 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- 172.21.15.63:0/664459450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fd1a0 msgr2=0x7fe0b00fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 --2- 172.21.15.63:0/664459450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fd1a0 0x7fe0b00fd570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4004900 tx=0x7fe0a4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- 172.21.15.63:0/664459450 shutdown_connections 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 --2- 172.21.15.63:0/664459450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0b00fdb40 0x7fe0b0106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 --2- 172.21.15.63:0/664459450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fd1a0 0x7fe0b00fd570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- 172.21.15.63:0/664459450 >> 172.21.15.63:0/664459450 conn(0x7fe0b00f6c60 msgr2=0x7fe0b00f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- 172.21.15.63:0/664459450 shutdown_connections 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- 172.21.15.63:0/664459450 wait complete. 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 Processor -- start 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.195+0000 7fe0b499d640 1 -- start start 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0b00fd1a0 0x7fe0b0113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0113bd0 con 0x7fe0b00fdb40 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b010d220 con 0x7fe0b00fd1a0 2024-04-24T22:57:49.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33104/0 (socket says 172.21.15.63:33104) 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 -- 172.21.15.63:0/3202168773 learned_addr learned my addr 172.21.15.63:0/3202168773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 -- 172.21.15.63:0/3202168773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0b00fd1a0 msgr2=0x7fe0b0113090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0b00fd1a0 0x7fe0b0113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 -- 172.21.15.63:0/3202168773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4005040 con 0x7fe0b00fdb40 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0add9b640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0002a50 tx=0x7fe0a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a0007c30 con 0x7fe0b00fdb40 2024-04-24T22:57:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b010d500 con 0x7fe0b00fdb40 2024-04-24T22:57:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0a000f040 con 0x7fe0b00fdb40 2024-04-24T22:57:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a000dbe0 con 0x7fe0b00fdb40 2024-04-24T22:57:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.196+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b010da20 con 0x7fe0b00fdb40 2024-04-24T22:57:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.197+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b0065d90 con 0x7fe0b00fdb40 2024-04-24T22:57:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.198+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe0a000dd40 con 0x7fe0b00fdb40 2024-04-24T22:57:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.198+0000 7fe09effd640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 0x7fe098070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.198+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe0a008db70 con 0x7fe0b00fdb40 2024-04-24T22:57:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.198+0000 7fe0ae59c640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 0x7fe098070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.199+0000 7fe0ae59c640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 0x7fe098070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009a10 tx=0x7fe0a4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.203+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe0a0056ea0 con 0x7fe0b00fdb40 2024-04-24T22:57:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.389+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0b004bcc0 con 0x7fe0b00fdb40 2024-04-24T22:57:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.389+0000 7fe09effd640 1 -- 172.21.15.63:0/3202168773 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0a005a560 con 0x7fe0b00fdb40 2024-04-24T22:57:49.391 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 msgr2=0x7fe098070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 0x7fe098070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009a10 tx=0x7fe0a4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 msgr2=0x7fe0b01135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0002a50 tx=0x7fe0a0004600 comp rx=0 tx=0).stop 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 shutdown_connections 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fe09806e0b0 0x7fe098070560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b00fdb40 0x7fe0b01135d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 --2- 172.21.15.63:0/3202168773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0b00fd1a0 0x7fe0b0113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 >> 172.21.15.63:0/3202168773 conn(0x7fe0b00f6c60 msgr2=0x7fe0b00f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 shutdown_connections 2024-04-24T22:57:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.391+0000 7fe0b499d640 1 -- 172.21.15.63:0/3202168773 wait complete. 2024-04-24T22:57:49.404 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T22:57:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 -- 172.21.15.63:0/4243600824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fdb50 msgr2=0x7f311c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 --2- 172.21.15.63:0/4243600824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fdb50 0x7f311c106220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f310c004900 tx=0x7f310c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 -- 172.21.15.63:0/4243600824 shutdown_connections 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 --2- 172.21.15.63:0/4243600824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fdb50 0x7f311c106220 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 --2- 172.21.15.63:0/4243600824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fd1b0 0x7f311c0fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 -- 172.21.15.63:0/4243600824 >> 172.21.15.63:0/4243600824 conn(0x7f311c0f6cb0 msgr2=0x7f311c0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 -- 172.21.15.63:0/4243600824 shutdown_connections 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.509+0000 7f3123e2e640 1 -- 172.21.15.63:0/4243600824 wait complete. 2024-04-24T22:57:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 Processor -- start 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 -- start start 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 0x7f311c1132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1138e0 con 0x7f311c0fd1b0 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3123e2e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c10cf30 con 0x7f311c0fdb50 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3121bca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3121bca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33126/0 (socket says 172.21.15.63:33126) 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3121bca640 1 -- 172.21.15.63:0/3904908304 learned_addr learned my addr 172.21.15.63:0/3904908304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f31213c9640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 0x7f311c1132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3121bca640 1 -- 172.21.15.63:0/3904908304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 msgr2=0x7f311c1132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.510+0000 7f3121bca640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 0x7f311c1132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f3121bca640 1 -- 172.21.15.63:0/3904908304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c005040 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f31213c9640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 0x7f311c1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f3121bca640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f311800c790 tx=0x7f31180043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f311800a820 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c10d210 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f311800ae50 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31180115e0 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.511+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c10d730 con 0x7f311c0fd1b0 2024-04-24T22:57:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.512+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c065d50 con 0x7f311c0fd1b0 2024-04-24T22:57:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.512+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3118012400 con 0x7f311c0fd1b0 2024-04-24T22:57:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.513+0000 7f31127fc640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 0x7f3108070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.513+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f311808dd30 con 0x7f311c0fd1b0 2024-04-24T22:57:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.513+0000 7f31213c9640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 0x7f3108070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.513+0000 7f31213c9640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 0x7f3108070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f310c009a30 tx=0x7f310c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.517+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f31180570e0 con 0x7f311c0fd1b0 2024-04-24T22:57:49.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.700+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f311c109320 con 0x7f311c0fd1b0 2024-04-24T22:57:49.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.701+0000 7f31127fc640 1 -- 172.21.15.63:0/3904908304 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f311805a7a0 con 0x7f311c0fd1b0 2024-04-24T22:57:49.702 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 msgr2=0x7f3108070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 0x7f3108070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f310c009a30 tx=0x7f310c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 msgr2=0x7f311c112da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f311800c790 tx=0x7f31180043c0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 shutdown_connections 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f310806e0b0 0x7f3108070560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f311c0fdb50 0x7f311c1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 --2- 172.21.15.63:0/3904908304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f311c0fd1b0 0x7f311c112da0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 >> 172.21.15.63:0/3904908304 conn(0x7f311c0f6cb0 msgr2=0x7f311c0f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 shutdown_connections 2024-04-24T22:57:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.703+0000 7f3123e2e640 1 -- 172.21.15.63:0/3904908304 wait complete. 2024-04-24T22:57:49.714 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:49 smithi063 ceph-mon[111213]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:49.714 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:49 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/520785278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:49.714 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:49 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3202168773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:49.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 -- 172.21.15.63:0/1259478679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fdb60 msgr2=0x7fc7781061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 --2- 172.21.15.63:0/1259478679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fdb60 0x7fc7781061a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7600048d0 tx=0x7fc760009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 -- 172.21.15.63:0/1259478679 shutdown_connections 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 --2- 172.21.15.63:0/1259478679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fdb60 0x7fc7781061a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 --2- 172.21.15.63:0/1259478679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fd1c0 0x7fc7780fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 -- 172.21.15.63:0/1259478679 >> 172.21.15.63:0/1259478679 conn(0x7fc7780f6cb0 msgr2=0x7fc7780f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 -- 172.21.15.63:0/1259478679 shutdown_connections 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.815+0000 7fc77ca14640 1 -- 172.21.15.63:0/1259478679 wait complete. 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 Processor -- start 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 -- start start 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fdb60 0x7fc7781132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7781138c0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77ca14640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77810cf10 con 0x7fc7780fdb60 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33142/0 (socket says 172.21.15.63:33142) 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 -- 172.21.15.63:0/3426209525 learned_addr learned my addr 172.21.15.63:0/3426209525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc76ffff640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fdb60 0x7fc7781132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 -- 172.21.15.63:0/3426209525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fdb60 msgr2=0x7fc7781132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fdb60 0x7fc7781132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.816+0000 7fc77659c640 1 -- 172.21.15.63:0/3426209525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760005040 con 0x7fc7780fd1c0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc77659c640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc768002a50 tx=0x7fc768004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc768007c30 con 0x7fc7780fd1c0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76800f040 con 0x7fc7780fd1c0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77810d1f0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76800db50 con 0x7fc7780fd1c0 2024-04-24T22:57:49.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.817+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77810d6e0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.818+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc778065dd0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.818+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc76801c400 con 0x7fc7780fd1c0 2024-04-24T22:57:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.819+0000 7fc76effd640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 0x7fc764070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.819+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc76808d3c0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.819+0000 7fc76ffff640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 0x7fc764070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.819+0000 7fc76ffff640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 0x7fc764070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7780fe530 tx=0x7fc760009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:49.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:49.824+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc7680567a0 con 0x7fc7780fd1c0 2024-04-24T22:57:49.926 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:49 smithi192 ceph-mon[80463]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:49 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/520785278' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:49 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3202168773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:50.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.007+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fc778109310 con 0x7fc7780fd1c0 2024-04-24T22:57:50.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.007+0000 7fc76effd640 1 -- 172.21.15.63:0/3426209525 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc768059e60 con 0x7fc7780fd1c0 2024-04-24T22:57:50.009 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 msgr2=0x7fc764070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 0x7fc764070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7780fe530 tx=0x7fc760009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 msgr2=0x7fc778112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc768002a50 tx=0x7fc768004600 comp rx=0 tx=0).stop 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 shutdown_connections 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fc76406e0b0 0x7fc764070560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7780fdb60 0x7fc7781132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 --2- 172.21.15.63:0/3426209525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7780fd1c0 0x7fc778112d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 >> 172.21.15.63:0/3426209525 conn(0x7fc7780f6cb0 msgr2=0x7fc7780f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 shutdown_connections 2024-04-24T22:57:50.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.009+0000 7fc77ca14640 1 -- 172.21.15.63:0/3426209525 wait complete. 2024-04-24T22:57:50.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:57:50.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 -- 172.21.15.63:0/1614036384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102210 msgr2=0x7f6a941025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 --2- 172.21.15.63:0/1614036384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102210 0x7f6a941025e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a840048d0 tx=0x7f6a84009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 -- 172.21.15.63:0/1614036384 shutdown_connections 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 --2- 172.21.15.63:0/1614036384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102b20 0x7f6a940fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 --2- 172.21.15.63:0/1614036384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102210 0x7f6a941025e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 -- 172.21.15.63:0/1614036384 >> 172.21.15.63:0/1614036384 conn(0x7f6a94071770 msgr2=0x7f6a94071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 -- 172.21.15.63:0/1614036384 shutdown_connections 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.122+0000 7f6a996ae640 1 -- 172.21.15.63:0/1614036384 wait complete. 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 Processor -- start 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 -- start start 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102210 0x7f6a94112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941138a0 con 0x7f6a94102b20 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a996ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9410cef0 con 0x7f6a94102210 2024-04-24T22:57:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a927fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a927fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33166/0 (socket says 172.21.15.63:33166) 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a927fc640 1 -- 172.21.15.63:0/2273615569 learned_addr learned my addr 172.21.15.63:0/2273615569 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.123+0000 7f6a92ffd640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102210 0x7f6a94112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a927fc640 1 -- 172.21.15.63:0/2273615569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102210 msgr2=0x7f6a94112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a927fc640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102210 0x7f6a94112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a927fc640 1 -- 172.21.15.63:0/2273615569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84005040 con 0x7f6a94102b20 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a927fc640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a88002f40 tx=0x7f6a8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88007c30 con 0x7f6a94102b20 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a8800f040 con 0x7f6a94102b20 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9410d170 con 0x7f6a94102b20 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8800ccb0 con 0x7f6a94102b20 2024-04-24T22:57:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.124+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9410d690 con 0x7f6a94102b20 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.125+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a940fdb10 con 0x7f6a94102b20 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.125+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a880090d0 con 0x7f6a94102b20 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.126+0000 7f6a7b7fe640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 0x7f6a7c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.126+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6a8808d8c0 con 0x7f6a94102b20 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.126+0000 7f6a92ffd640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 0x7f6a7c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.126+0000 7f6a92ffd640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 0x7f6a7c070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007ac0 tx=0x7f6a84009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.130+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a88056bf0 con 0x7f6a94102b20 2024-04-24T22:57:50.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.353+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a94109370 con 0x7f6a94102b20 2024-04-24T22:57:50.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.353+0000 7f6a7b7fe640 1 -- 172.21.15.63:0/2273615569 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5210 (secure 0 0 0) 0x7f6a8805a2b0 con 0x7f6a94102b20 2024-04-24T22:57:50.355 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:50.355 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44313,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3112021694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3112021694},{"type":"v1","addr":"172.21.15.192:6827","nonce":3112021694}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:49.318934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-04-24T22:57:50.355 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 31 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 msgr2=0x7f6a7c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 0x7f6a7c070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007ac0 tx=0x7f6a84009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 msgr2=0x7f6a941132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a88002f40 tx=0x7f6a8800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 shutdown_connections 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f6a7c06e0b0 0x7f6a7c070560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a94102b20 0x7f6a941132a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 --2- 172.21.15.63:0/2273615569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a94102210 0x7f6a94112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 >> 172.21.15.63:0/2273615569 conn(0x7f6a94071770 msgr2=0x7f6a940f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 shutdown_connections 2024-04-24T22:57:50.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.355+0000 7f6a996ae640 1 -- 172.21.15.63:0/2273615569 wait complete. 2024-04-24T22:57:50.368 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 14, 'max_mds': 1, 'flags': 18} 2024-04-24T22:57:50.368 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 -- 172.21.15.63:0/3613349363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fd0c0 msgr2=0x7f71a40fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 --2- 172.21.15.63:0/3613349363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fd0c0 0x7f71a40fd490 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a00048d0 tx=0x7f71a0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 -- 172.21.15.63:0/3613349363 shutdown_connections 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 --2- 172.21.15.63:0/3613349363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fa8c0 0x7f71a40fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 --2- 172.21.15.63:0/3613349363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fd0c0 0x7f71a40fd490 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 -- 172.21.15.63:0/3613349363 >> 172.21.15.63:0/3613349363 conn(0x7f71a40717e0 msgr2=0x7f71a4071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 -- 172.21.15.63:0/3613349363 shutdown_connections 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.465+0000 7f71aba6e640 1 -- 172.21.15.63:0/3613349363 wait complete. 2024-04-24T22:57:50.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 Processor -- start 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 -- start start 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fd0c0 0x7f71a4115390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4115990 con 0x7f71a40fa8c0 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71aba6e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a410efe0 con 0x7f71a40fd0c0 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71a980a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71a980a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33190/0 (socket says 172.21.15.63:33190) 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71a980a640 1 -- 172.21.15.63:0/689717052 learned_addr learned my addr 172.21.15.63:0/689717052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.466+0000 7f71a9009640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fd0c0 0x7f71a4115390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71a980a640 1 -- 172.21.15.63:0/689717052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fd0c0 msgr2=0x7f71a4115390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71a980a640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fd0c0 0x7f71a4115390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71a980a640 1 -- 172.21.15.63:0/689717052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0005040 con 0x7f71a40fa8c0 2024-04-24T22:57:50.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71a980a640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f71a0004930 tx=0x7f71a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a00094c0 con 0x7f71a40fa8c0 2024-04-24T22:57:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71a0032e50 con 0x7f71a40fa8c0 2024-04-24T22:57:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a00433b0 con 0x7f71a40fa8c0 2024-04-24T22:57:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a410f260 con 0x7f71a40fa8c0 2024-04-24T22:57:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.467+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a410f720 con 0x7f71a40fa8c0 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.468+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a40fb8d0 con 0x7f71a40fa8c0 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.468+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f71a002f070 con 0x7f71a40fa8c0 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.469+0000 7f719a7fc640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 0x7f7190070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.469+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f71a00af960 con 0x7f71a40fa8c0 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.469+0000 7f71a9009640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 0x7f7190070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.469+0000 7f71a9009640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 0x7f7190070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71a4110710 tx=0x7f719400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.474+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f71a0078c90 con 0x7f71a40fa8c0 2024-04-24T22:57:50.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.699+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7f71a41104c0 con 0x7f71a40fa8c0 2024-04-24T22:57:50.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.699+0000 7f719a7fc640 1 -- 172.21.15.63:0/689717052 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v31) v1 ==== 107+0+4081 (secure 0 0 0) 0x7f71a007c350 con 0x7f71a40fa8c0 2024-04-24T22:57:50.701 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:50.701 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.63:6829/3016441784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3016441784},{"type":"v1","addr":"172.21.15.63:6829","nonce":3016441784}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:42:40.679489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi192.mnfuxk","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6825/780449385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":780449385},{"type":"v1","addr":"172.21.15.192:6825","nonce":780449385}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:50.702 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 msgr2=0x7f7190070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 0x7f7190070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71a4110710 tx=0x7f719400b040 comp rx=0 tx=0).stop 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 msgr2=0x7f71a4114e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f71a0004930 tx=0x7f71a0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 shutdown_connections 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f719006e0b0 0x7f7190070560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a40fd0c0 0x7f71a4115390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 --2- 172.21.15.63:0/689717052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a40fa8c0 0x7f71a4114e50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 >> 172.21.15.63:0/689717052 conn(0x7f71a40717e0 msgr2=0x7f71a40f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 shutdown_connections 2024-04-24T22:57:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.701+0000 7f71aba6e640 1 -- 172.21.15.63:0/689717052 wait complete. 2024-04-24T22:57:50.712 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:50 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3904908304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:50.713 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:50 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3426209525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:50.713 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:50 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2273615569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:57:50.714 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-24T22:57:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 -- 172.21.15.63:0/2047504490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 msgr2=0x7f9a640fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/2047504490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fa8c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a580048d0 tx=0x7f9a58009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 -- 172.21.15.63:0/2047504490 shutdown_connections 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/2047504490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fa8c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/2047504490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 0x7f9a64102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 -- 172.21.15.63:0/2047504490 >> 172.21.15.63:0/2047504490 conn(0x7f9a640717e0 msgr2=0x7f9a64071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 -- 172.21.15.63:0/2047504490 shutdown_connections 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 -- 172.21.15.63:0/2047504490 wait complete. 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.811+0000 7f9a6af5d640 1 Processor -- start 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a6af5d640 1 -- start start 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a6af5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 0x7f9a640fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a6af5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a6af5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640fda10 con 0x7f9a64102bd0 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a6af5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640fc520 con 0x7f9a64102230 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33198/0 (socket says 172.21.15.63:33198) 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 -- 172.21.15.63:0/3725463006 learned_addr learned my addr 172.21.15.63:0/3725463006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a68cf9640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 0x7f9a640fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 -- 172.21.15.63:0/3725463006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 msgr2=0x7f9a640fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 0x7f9a640fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 -- 172.21.15.63:0/3725463006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58005040 con 0x7f9a64102bd0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.812+0000 7f9a63fff640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802bda0 tx=0x7f9a58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.813+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a580094c0 con 0x7f9a64102bd0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.813+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a640fc740 con 0x7f9a64102bd0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.813+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a58032e50 con 0x7f9a64102bd0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.813+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5803b640 con 0x7f9a64102bd0 2024-04-24T22:57:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.813+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a641194d0 con 0x7f9a64102bd0 2024-04-24T22:57:50.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.814+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64065530 con 0x7f9a64102bd0 2024-04-24T22:57:50.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.814+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9a5802f070 con 0x7f9a64102bd0 2024-04-24T22:57:50.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.815+0000 7f9a617fa640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 0x7f9a4c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:50.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.815+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a580affb0 con 0x7f9a64102bd0 2024-04-24T22:57:50.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.815+0000 7f9a68cf9640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 0x7f9a4c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:50.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.815+0000 7f9a68cf9640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 0x7f9a4c070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400a900 tx=0x7f9a54008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:50.819+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9a580e4740 con 0x7f9a64102bd0 2024-04-24T22:57:50.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:50 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3904908304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:50.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:50 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3426209525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:50.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:50 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2273615569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T22:57:51.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.040+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f9a64064350 con 0x7f9a64102bd0 2024-04-24T22:57:51.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.041+0000 7f9a617fa640 1 -- 172.21.15.63:0/3725463006 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v31) v1 ==== 107+0+4925 (secure 0 0 0) 0x7f9a580792e0 con 0x7f9a64102bd0 2024-04-24T22:57:51.042 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:51.042 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.63:6829/3016441784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3016441784},{"type":"v1","addr":"172.21.15.63:6829","nonce":3016441784}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:42:40.679489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi192.mnfuxk","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6825/780449385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":780449385},{"type":"v1","addr":"172.21.15.192:6825","nonce":780449385}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:51.043 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 16 2024-04-24T22:57:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 msgr2=0x7f9a4c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 0x7f9a4c070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400a900 tx=0x7f9a54008040 comp rx=0 tx=0).stop 2024-04-24T22:57:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 msgr2=0x7f9a640fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802bda0 tx=0x7f9a58009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 shutdown_connections 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9a4c06e0b0 0x7f9a4c070560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a64102bd0 0x7f9a640fbfe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 --2- 172.21.15.63:0/3725463006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a64102230 0x7f9a640fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 >> 172.21.15.63:0/3725463006 conn(0x7f9a640717e0 msgr2=0x7f9a640fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 shutdown_connections 2024-04-24T22:57:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.043+0000 7f9a6af5d640 1 -- 172.21.15.63:0/3725463006 wait complete. 2024-04-24T22:57:51.055 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-24T22:57:51.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 -- 172.21.15.63:0/3741529635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 msgr2=0x7fd204064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/3741529635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd204064800 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80048d0 tx=0x7fd1f8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 -- 172.21.15.63:0/3741529635 shutdown_connections 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/3741529635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd204064800 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/3741529635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 0x7fd204102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 -- 172.21.15.63:0/3741529635 >> 172.21.15.63:0/3741529635 conn(0x7fd204071fd0 msgr2=0x7fd2040723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 -- 172.21.15.63:0/3741529635 shutdown_connections 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.152+0000 7fd20ad5d640 1 -- 172.21.15.63:0/3741529635 wait complete. 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 Processor -- start 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 -- start start 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 0x7fd2040fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040fd0c0 con 0x7fd204064390 2024-04-24T22:57:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd20ad5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040fbc60 con 0x7fd204102270 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd208af9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd208af9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33216/0 (socket says 172.21.15.63:33216) 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.153+0000 7fd208af9640 1 -- 172.21.15.63:0/48018247 learned_addr learned my addr 172.21.15.63:0/48018247 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd203fff640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 0x7fd2040fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd208af9640 1 -- 172.21.15.63:0/48018247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 msgr2=0x7fd2040fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd208af9640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 0x7fd2040fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd208af9640 1 -- 172.21.15.63:0/48018247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8005040 con 0x7fd204064390 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd208af9640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4002a80 tx=0x7fd1f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f4007c30 con 0x7fd204064390 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2040fbf40 con 0x7fd204064390 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1f400f040 con 0x7fd204064390 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f40096b0 con 0x7fd204064390 2024-04-24T22:57:51.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.154+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd204119380 con 0x7fd204064390 2024-04-24T22:57:51.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.155+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2040fd970 con 0x7fd204064390 2024-04-24T22:57:51.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.155+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd1f400d6b0 con 0x7fd204064390 2024-04-24T22:57:51.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.156+0000 7fd2017fa640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 0x7fd1ec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.156+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd1f408d570 con 0x7fd204064390 2024-04-24T22:57:51.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.156+0000 7fd203fff640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 0x7fd1ec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.156+0000 7fd203fff640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 0x7fd1ec070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd2040fc910 tx=0x7fd1f8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.160+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd1f40568a0 con 0x7fd204064390 2024-04-24T22:57:51.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.381+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fd204063f00 con 0x7fd204064390 2024-04-24T22:57:51.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.382+0000 7fd2017fa640 1 -- 172.21.15.63:0/48018247 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v31) v1 ==== 107+0+4146 (secure 0 0 0) 0x7fd1f4056030 con 0x7fd204064390 2024-04-24T22:57:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:51.383 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:42:40.679489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi192.mnfuxk","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6825/780449385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":780449385},{"type":"v1","addr":"172.21.15.192:6825","nonce":780449385}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:51.383 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-04-24T22:57:51.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 msgr2=0x7fd1ec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 0x7fd1ec070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd2040fc910 tx=0x7fd1f8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 msgr2=0x7fd2040fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4002a80 tx=0x7fd1f4004600 comp rx=0 tx=0).stop 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 shutdown_connections 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fd1ec06e0b0 0x7fd1ec070560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd204102270 0x7fd2040fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 --2- 172.21.15.63:0/48018247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd204064390 0x7fd2040fb1e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 >> 172.21.15.63:0/48018247 conn(0x7fd204071fd0 msgr2=0x7fd2040fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 shutdown_connections 2024-04-24T22:57:51.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.384+0000 7fd20ad5d640 1 -- 172.21.15.63:0/48018247 wait complete. 2024-04-24T22:57:51.397 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-24T22:57:51.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- 172.21.15.63:0/3793755942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69181021e0 msgr2=0x7f69181025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 --2- 172.21.15.63:0/3793755942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69181021e0 0x7f69181025b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f69000048d0 tx=0x7f6900009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- 172.21.15.63:0/3793755942 shutdown_connections 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 --2- 172.21.15.63:0/3793755942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6918102b80 0x7f69180fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 --2- 172.21.15.63:0/3793755942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69181021e0 0x7f69181025b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- 172.21.15.63:0/3793755942 >> 172.21.15.63:0/3793755942 conn(0x7f69180f7db0 msgr2=0x7f69180fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- 172.21.15.63:0/3793755942 shutdown_connections 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- 172.21.15.63:0/3793755942 wait complete. 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 Processor -- start 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.494+0000 7f691cf5d640 1 -- start start 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69181021e0 0x7f691806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918071600 con 0x7f6918102b80 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918071770 con 0x7f69181021e0 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33228/0 (socket says 172.21.15.63:33228) 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 -- 172.21.15.63:0/1181335982 learned_addr learned my addr 172.21.15.63:0/1181335982 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691659c640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69181021e0 0x7f691806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 -- 172.21.15.63:0/1181335982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69181021e0 msgr2=0x7f691806f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69181021e0 0x7f691806f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 -- 172.21.15.63:0/1181335982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900005040 con 0x7f6918102b80 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f6915d9b640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f690800c830 tx=0x7f6908004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690800a850 con 0x7f6918102b80 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f690800ae80 con 0x7f6918102b80 2024-04-24T22:57:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6918070390 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6908012ea0 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.495+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6918119330 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.496+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69180fd950 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.497+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6908012400 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.497+0000 7f690effd640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 0x7f6904070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.497+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f690808dde0 con 0x7f6918102b80 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.497+0000 7f691659c640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 0x7f6904070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.498+0000 7f691659c640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 0x7f6904070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.502+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6908057190 con 0x7f6918102b80 2024-04-24T22:57:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.726+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f6918064350 con 0x7f6918102b80 2024-04-24T22:57:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.727+0000 7f690effd640 1 -- 172.21.15.63:0/1181335982 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v31) v1 ==== 107+0+4993 (secure 0 0 0) 0x7f69080126b0 con 0x7f6918102b80 2024-04-24T22:57:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:51.729 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:42:40.679489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi192.mnfuxk","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6825/780449385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":780449385},{"type":"v1","addr":"172.21.15.192:6825","nonce":780449385}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:51.729 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 18 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 msgr2=0x7f6904070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 0x7f6904070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 msgr2=0x7f691806fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f690800c830 tx=0x7f6908004430 comp rx=0 tx=0).stop 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 shutdown_connections 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f690406e0b0 0x7f6904070560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6918102b80 0x7f691806fcb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 --2- 172.21.15.63:0/1181335982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69181021e0 0x7f691806f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 >> 172.21.15.63:0/1181335982 conn(0x7f69180f7db0 msgr2=0x7f69180f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 shutdown_connections 2024-04-24T22:57:51.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.729+0000 7f691cf5d640 1 -- 172.21.15.63:0/1181335982 wait complete. 2024-04-24T22:57:51.740 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:51 smithi063 ceph-mon[111213]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:51.741 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:51 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/689717052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-24T22:57:51.741 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:51 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3725463006' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-24T22:57:51.741 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:51 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/48018247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-24T22:57:51.742 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-24T22:57:51.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 -- 172.21.15.63:0/996522987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 msgr2=0x7fa3a00fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/996522987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a00fd560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3880048d0 tx=0x7fa388009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 -- 172.21.15.63:0/996522987 shutdown_connections 2024-04-24T22:57:51.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/996522987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 0x7fa3a0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/996522987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a00fd560 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 -- 172.21.15.63:0/996522987 >> 172.21.15.63:0/996522987 conn(0x7fa3a00f6be0 msgr2=0x7fa3a00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:51.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 -- 172.21.15.63:0/996522987 shutdown_connections 2024-04-24T22:57:51.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.839+0000 7fa3a66ae640 1 -- 172.21.15.63:0/996522987 wait complete. 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 Processor -- start 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 -- start start 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 0x7fa3a0113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0113840 con 0x7fa3a00fd190 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa3a66ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a010ce90 con 0x7fa3a00fdb30 2024-04-24T22:57:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33246/0 (socket says 172.21.15.63:33246) 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa39ffff640 1 -- 172.21.15.63:0/286173623 learned_addr learned my addr 172.21.15.63:0/286173623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.840+0000 7fa39f7fe640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 0x7fa3a0113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39ffff640 1 -- 172.21.15.63:0/286173623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 msgr2=0x7fa3a0113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39ffff640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 0x7fa3a0113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39ffff640 1 -- 172.21.15.63:0/286173623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388005040 con 0x7fa3a00fd190 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39ffff640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa388004930 tx=0x7fa388009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3880094c0 con 0x7fa3a00fd190 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa388032e50 con 0x7fa3a00fd190 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a010d110 con 0x7fa3a00fd190 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3880433b0 con 0x7fa3a00fd190 2024-04-24T22:57:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.841+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a010d5d0 con 0x7fa3a00fd190 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.842+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a0066040 con 0x7fa3a00fd190 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.842+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa38802f070 con 0x7fa3a00fd190 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.843+0000 7fa39cff9640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 0x7fa38c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.843+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa3880af8b0 con 0x7fa3a00fd190 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.843+0000 7fa39f7fe640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 0x7fa38c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:51.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.843+0000 7fa39f7fe640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 0x7fa38c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa390007fd0 tx=0x7fa390009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:51.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:51.848+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa388078be0 con 0x7fa3a00fd190 2024-04-24T22:57:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:51 smithi192 ceph-mon[80463]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:51 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/689717052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-24T22:57:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:51 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3725463006' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-24T22:57:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:51 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/48018247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-24T22:57:52.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.072+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fa3a010e400 con 0x7fa3a00fd190 2024-04-24T22:57:52.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.073+0000 7fa39cff9640 1 -- 172.21.15.63:0/286173623 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v31) v1 ==== 107+0+4203 (secure 0 0 0) 0x7fa38807c2a0 con 0x7fa3a00fd190 2024-04-24T22:57:52.074 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:52.074 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:08.099760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:52.075 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 19 2024-04-24T22:57:52.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 msgr2=0x7fa38c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 0x7fa38c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa390007fd0 tx=0x7fa390009e90 comp rx=0 tx=0).stop 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 msgr2=0x7fa3a0112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa388004930 tx=0x7fa388009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 shutdown_connections 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fa38c06e0b0 0x7fa38c070560 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a00fdb30 0x7fa3a0113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 --2- 172.21.15.63:0/286173623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3a00fd190 0x7fa3a0112d00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 >> 172.21.15.63:0/286173623 conn(0x7fa3a00f6be0 msgr2=0x7fa3a00f8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 shutdown_connections 2024-04-24T22:57:52.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.075+0000 7fa3a66ae640 1 -- 172.21.15.63:0/286173623 wait complete. 2024-04-24T22:57:52.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 -- 172.21.15.63:0/1873332870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d81021b0 msgr2=0x7f13d8102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/1873332870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d81021b0 0x7f13d8102580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13c80048d0 tx=0x7f13c8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 -- 172.21.15.63:0/1873332870 shutdown_connections 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/1873332870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d8102b50 0x7f13d80fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/1873332870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d81021b0 0x7f13d8102580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 -- 172.21.15.63:0/1873332870 >> 172.21.15.63:0/1873332870 conn(0x7f13d80f7d80 msgr2=0x7f13d80fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 -- 172.21.15.63:0/1873332870 shutdown_connections 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.189+0000 7f13ddb1d640 1 -- 172.21.15.63:0/1873332870 wait complete. 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 Processor -- start 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 -- start start 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d81021b0 0x7f13d806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8071650 con 0x7f13d8102b50 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13ddb1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80701f0 con 0x7f13d81021b0 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13d6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13d6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33262/0 (socket says 172.21.15.63:33262) 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13d6ffd640 1 -- 172.21.15.63:0/2239547844 learned_addr learned my addr 172.21.15.63:0/2239547844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.190+0000 7f13d77fe640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d81021b0 0x7f13d806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13d6ffd640 1 -- 172.21.15.63:0/2239547844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d81021b0 msgr2=0x7f13d806f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13d6ffd640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d81021b0 0x7f13d806f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13d6ffd640 1 -- 172.21.15.63:0/2239547844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8005040 con 0x7f13d8102b50 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13d6ffd640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13cc002a50 tx=0x7f13cc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc007c30 con 0x7f13d8102b50 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13cc00f040 con 0x7f13d8102b50 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80704d0 con 0x7f13d8102b50 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc00db50 con 0x7f13d8102b50 2024-04-24T22:57:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.191+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81193e0 con 0x7f13d8102b50 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.192+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d80fd920 con 0x7f13d8102b50 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.193+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f13cc01c400 con 0x7f13d8102b50 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.193+0000 7f13bffff640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 0x7f13c0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.194+0000 7f13d77fe640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 0x7f13c0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.194+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f13cc08d3c0 con 0x7f13d8102b50 2024-04-24T22:57:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.194+0000 7f13d77fe640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 0x7f13c0070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13c8003080 tx=0x7f13c8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.198+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f13cc0567a0 con 0x7f13d8102b50 2024-04-24T22:57:52.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.422+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f13d8109370 con 0x7f13d8102b50 2024-04-24T22:57:52.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.423+0000 7f13bffff640 1 -- 172.21.15.63:0/2239547844 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v31) v1 ==== 107+0+4214 (secure 0 0 0) 0x7f13cc059e60 con 0x7f13d8102b50 2024-04-24T22:57:52.425 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:52.425 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:08.105802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":0,"incarnation":20,"state":"up:replay","state_seq":2,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:52.425 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 20 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 msgr2=0x7f13c0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 0x7f13c0070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13c8003080 tx=0x7f13c8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 msgr2=0x7f13d806fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13cc002a50 tx=0x7f13cc004600 comp rx=0 tx=0).stop 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 shutdown_connections 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f13c006e0b0 0x7f13c0070560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13d8102b50 0x7f13d806fcb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 --2- 172.21.15.63:0/2239547844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d81021b0 0x7f13d806f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 >> 172.21.15.63:0/2239547844 conn(0x7f13d80f7d80 msgr2=0x7f13d80f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 shutdown_connections 2024-04-24T22:57:52.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.425+0000 7f13ddb1d640 1 -- 172.21.15.63:0/2239547844 wait complete. 2024-04-24T22:57:52.438 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.538+0000 7fee91b9d640 1 -- 172.21.15.63:0/3309605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c064390 msgr2=0x7fee8c064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.538+0000 7fee91b9d640 1 --2- 172.21.15.63:0/3309605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c064390 0x7fee8c064800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0048d0 tx=0x7fee7c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- 172.21.15.63:0/3309605860 shutdown_connections 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 --2- 172.21.15.63:0/3309605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c064390 0x7fee8c064800 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 --2- 172.21.15.63:0/3309605860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c102270 0x7fee8c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- 172.21.15.63:0/3309605860 >> 172.21.15.63:0/3309605860 conn(0x7fee8c071fd0 msgr2=0x7fee8c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- 172.21.15.63:0/3309605860 shutdown_connections 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- 172.21.15.63:0/3309605860 wait complete. 2024-04-24T22:57:52.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 Processor -- start 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- start start 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c064390 0x7fee8c112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c1137c0 con 0x7fee8c102270 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.539+0000 7fee91b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c10ce10 con 0x7fee8c064390 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33288/0 (socket says 172.21.15.63:33288) 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 -- 172.21.15.63:0/250409834 learned_addr learned my addr 172.21.15.63:0/250409834 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8b7fe640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c064390 0x7fee8c112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 -- 172.21.15.63:0/250409834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c064390 msgr2=0x7fee8c112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c064390 0x7fee8c112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 -- 172.21.15.63:0/250409834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c005040 con 0x7fee8c102270 2024-04-24T22:57:52.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee8affd640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee7c034040 tx=0x7fee7c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7c0092b0 con 0x7fee8c102270 2024-04-24T22:57:52.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee7c03b400 con 0x7fee8c102270 2024-04-24T22:57:52.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7c0435d0 con 0x7fee8c102270 2024-04-24T22:57:52.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c10d090 con 0x7fee8c102270 2024-04-24T22:57:52.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.540+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8c10d500 con 0x7fee8c102270 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.541+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8c0fd970 con 0x7fee8c102270 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.542+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fee7c02f070 con 0x7fee8c102270 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.542+0000 7fee73fff640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 0x7fee74070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.542+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fee7c0af830 con 0x7fee8c102270 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.542+0000 7fee8b7fe640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 0x7fee74070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.543+0000 7fee8b7fe640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 0x7fee74070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee80009af0 tx=0x7fee8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.547+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fee7c078c10 con 0x7fee8c102270 2024-04-24T22:57:52.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.776+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fee8c1093a0 con 0x7fee8c102270 2024-04-24T22:57:52.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.777+0000 7fee73fff640 1 -- 172.21.15.63:0/250409834 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v31) v1 ==== 107+0+4219 (secure 0 0 0) 0x7fee7c07c2d0 con 0x7fee8c102270 2024-04-24T22:57:52.779 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:52.779 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:17.516808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":0,"incarnation":20,"state":"up:reconnect","state_seq":192,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:52.779 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 21 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 msgr2=0x7fee74070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 0x7fee74070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee80009af0 tx=0x7fee8000b040 comp rx=0 tx=0).stop 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 msgr2=0x7fee8c1131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee7c034040 tx=0x7fee7c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 shutdown_connections 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fee7406e0b0 0x7fee74070560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee8c102270 0x7fee8c1131c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 --2- 172.21.15.63:0/250409834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee8c064390 0x7fee8c112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 >> 172.21.15.63:0/250409834 conn(0x7fee8c071fd0 msgr2=0x7fee8c0fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 shutdown_connections 2024-04-24T22:57:52.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.779+0000 7fee91b9d640 1 -- 172.21.15.63:0/250409834 wait complete. 2024-04-24T22:57:52.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:52 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1181335982' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T22:57:52.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:52 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:57:52.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:52 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/286173623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T22:57:52.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:52 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2239547844' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T22:57:52.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-24T22:57:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 -- 172.21.15.63:0/2111334845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b40fb150 msgr2=0x7f00b40fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 --2- 172.21.15.63:0/2111334845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b40fb150 0x7f00b40fb5c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00a80048d0 tx=0x7f00a8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 -- 172.21.15.63:0/2111334845 shutdown_connections 2024-04-24T22:57:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 --2- 172.21.15.63:0/2111334845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b40fb150 0x7f00b40fb5c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 --2- 172.21.15.63:0/2111334845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b4101170 0x7f00b4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 -- 172.21.15.63:0/2111334845 >> 172.21.15.63:0/2111334845 conn(0x7f00b4071fd0 msgr2=0x7f00b40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.889+0000 7f00bb26e640 1 -- 172.21.15.63:0/2111334845 shutdown_connections 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 -- 172.21.15.63:0/2111334845 wait complete. 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 Processor -- start 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 -- start start 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 0x7f00b410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40652f0 con 0x7f00b4101170 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00bb26e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b410e8d0 con 0x7f00b40fb150 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00b8809640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00b8809640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33304/0 (socket says 172.21.15.63:33304) 2024-04-24T22:57:52.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00b8809640 1 -- 172.21.15.63:0/658450480 learned_addr learned my addr 172.21.15.63:0/658450480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.890+0000 7f00b900a640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 0x7f00b410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b8809640 1 -- 172.21.15.63:0/658450480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 msgr2=0x7f00b410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b8809640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 0x7f00b410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b8809640 1 -- 172.21.15.63:0/658450480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8005040 con 0x7f00b4101170 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b900a640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 0x7f00b410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b8809640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00a8004930 tx=0x7f00a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a80094c0 con 0x7f00b4101170 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00a8032e50 con 0x7f00b4101170 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b410eb50 con 0x7f00b4101170 2024-04-24T22:57:52.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a80433b0 con 0x7f00b4101170 2024-04-24T22:57:52.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.891+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41170b0 con 0x7f00b4101170 2024-04-24T22:57:52.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.892+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b40fc870 con 0x7f00b4101170 2024-04-24T22:57:52.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.893+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f00a802f070 con 0x7f00b4101170 2024-04-24T22:57:52.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.893+0000 7f00b1ffb640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 0x7f009c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:52.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.893+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f00a80afad0 con 0x7f00b4101170 2024-04-24T22:57:52.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.893+0000 7f00b900a640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 0x7f009c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:52.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.893+0000 7f00b900a640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 0x7f009c070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0041d0 tx=0x7f00ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:52.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:52.898+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f00a8078e00 con 0x7f00b4101170 2024-04-24T22:57:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:52 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1181335982' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T22:57:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:52 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:57:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:52 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/286173623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T22:57:52.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:52 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2239547844' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T22:57:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.123+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f00b40fe3a0 con 0x7f00b4101170 2024-04-24T22:57:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.124+0000 7f00b1ffb640 1 -- 172.21.15.63:0/658450480 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v31) v1 ==== 107+0+4216 (secure 0 0 0) 0x7f00a807c4c0 con 0x7f00b4101170 2024-04-24T22:57:53.125 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:53.125 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:18.444358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":193,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 22 2024-04-24T22:57:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 msgr2=0x7f009c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 0x7f009c070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0041d0 tx=0x7f00ac009040 comp rx=0 tx=0).stop 2024-04-24T22:57:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 msgr2=0x7f00b410e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00a8004930 tx=0x7f00a8009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 shutdown_connections 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f009c06e0b0 0x7f009c070560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b4101170 0x7f00b410e390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 --2- 172.21.15.63:0/658450480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00b40fb150 0x7f00b410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 >> 172.21.15.63:0/658450480 conn(0x7f00b4071fd0 msgr2=0x7f00b40fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 shutdown_connections 2024-04-24T22:57:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.126+0000 7f00bb26e640 1 -- 172.21.15.63:0/658450480 wait complete. 2024-04-24T22:57:53.138 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-24T22:57:53.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- 172.21.15.63:0/556300613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 msgr2=0x7ff26c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 --2- 172.21.15.63:0/556300613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c101440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff2680048d0 tx=0x7ff268009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- 172.21.15.63:0/556300613 shutdown_connections 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 --2- 172.21.15.63:0/556300613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c0fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 --2- 172.21.15.63:0/556300613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c101440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- 172.21.15.63:0/556300613 >> 172.21.15.63:0/556300613 conn(0x7ff26c0f6be0 msgr2=0x7ff26c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- 172.21.15.63:0/556300613 shutdown_connections 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- 172.21.15.63:0/556300613 wait complete. 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 Processor -- start 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.236+0000 7ff272c21640 1 -- start start 2024-04-24T22:57:53.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff272c21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff272c21640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff272c21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c113840 con 0x7ff26c101070 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff272c21640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10ce90 con 0x7ff26c101a10 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33328/0 (socket says 172.21.15.63:33328) 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 -- 172.21.15.63:0/740205366 learned_addr learned my addr 172.21.15.63:0/740205366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff263fff640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 -- 172.21.15.63:0/740205366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 msgr2=0x7ff26c113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 -- 172.21.15.63:0/740205366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268005040 con 0x7ff26c101070 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff263fff640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2709bd640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff268004930 tx=0x7ff268009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2680094c0 con 0x7ff26c101070 2024-04-24T22:57:53.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c10d110 con 0x7ff26c101070 2024-04-24T22:57:53.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff268032e50 con 0x7ff26c101070 2024-04-24T22:57:53.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.237+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2680433b0 con 0x7ff26c101070 2024-04-24T22:57:53.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.238+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c10d5d0 con 0x7ff26c101070 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.239+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c0fc7e0 con 0x7ff26c101070 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.239+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff26802f070 con 0x7ff26c101070 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.239+0000 7ff2617fa640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 0x7ff254070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.240+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff2680afa30 con 0x7ff26c101070 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.240+0000 7ff263fff640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 0x7ff254070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.240+0000 7ff263fff640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 0x7ff254070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff26c10e5c0 tx=0x7ff25c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.244+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff2680b1e60 con 0x7ff26c101070 2024-04-24T22:57:53.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.467+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7ff26c10e370 con 0x7ff26c101070 2024-04-24T22:57:53.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.467+0000 7ff2617fa640 1 -- 172.21.15.63:0/740205366 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v31) v1 ==== 107+0+4225 (secure 0 0 0) 0x7ff268078d60 con 0x7ff26c101070 2024-04-24T22:57:53.469 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:53.469 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:20.448390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":0,"incarnation":20,"state":"up:active","state_seq":194,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-04-24T22:57:53.469 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 23 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 msgr2=0x7ff254070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 0x7ff254070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff26c10e5c0 tx=0x7ff25c00b040 comp rx=0 tx=0).stop 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 msgr2=0x7ff26c112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff268004930 tx=0x7ff268009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 shutdown_connections 2024-04-24T22:57:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7ff25406e0b0 0x7ff254070560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c101a10 0x7ff26c113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 --2- 172.21.15.63:0/740205366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff26c101070 0x7ff26c112d00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 >> 172.21.15.63:0/740205366 conn(0x7ff26c0f6be0 msgr2=0x7ff26c0f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 shutdown_connections 2024-04-24T22:57:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.470+0000 7ff272c21640 1 -- 172.21.15.63:0/740205366 wait complete. 2024-04-24T22:57:53.482 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- 172.21.15.63:0/4253687513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 msgr2=0x7f14dc064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/4253687513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc064800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0048d0 tx=0x7f14cc009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- 172.21.15.63:0/4253687513 shutdown_connections 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/4253687513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc064800 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/4253687513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 0x7f14dc102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- 172.21.15.63:0/4253687513 >> 172.21.15.63:0/4253687513 conn(0x7f14dc071fd0 msgr2=0x7f14dc0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- 172.21.15.63:0/4253687513 shutdown_connections 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- 172.21.15.63:0/4253687513 wait complete. 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 Processor -- start 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- start start 2024-04-24T22:57:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 0x7f14dc1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc1137c0 con 0x7f14dc064390 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.580+0000 7f14e3b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc10ce10 con 0x7f14dc102270 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33348/0 (socket says 172.21.15.63:33348) 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 -- 172.21.15.63:0/892906320 learned_addr learned my addr 172.21.15.63:0/892906320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1138640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 0x7f14dc1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 -- 172.21.15.63:0/892906320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 msgr2=0x7f14dc1131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 0x7f14dc1131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 -- 172.21.15.63:0/892906320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc005040 con 0x7f14dc064390 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e1939640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14d8002a50 tx=0x7f14d8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d8007c30 con 0x7f14dc064390 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14d800f040 con 0x7f14dc064390 2024-04-24T22:57:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc10d0f0 con 0x7f14dc064390 2024-04-24T22:57:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d800db50 con 0x7f14dc064390 2024-04-24T22:57:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.581+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc10d5e0 con 0x7f14dc064390 2024-04-24T22:57:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.582+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc0fd970 con 0x7f14dc064390 2024-04-24T22:57:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.583+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f14d800dcf0 con 0x7f14dc064390 2024-04-24T22:57:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.583+0000 7f14d27fc640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 0x7f14c8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.583+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f14d808d3d0 con 0x7f14dc064390 2024-04-24T22:57:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.583+0000 7f14e1138640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 0x7f14c8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.584+0000 7f14e1138640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 0x7f14c8070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0f9610 tx=0x7f14cc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.588+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f14d8056780 con 0x7f14dc064390 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/250409834' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/658450480' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T22:57:53.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:53 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/740205366' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T22:57:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.809+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f14dc0ff4a0 con 0x7f14dc064390 2024-04-24T22:57:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.810+0000 7f14d27fc640 1 -- 172.21.15.63:0/892906320 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v31) v1 ==== 107+0+5075 (secure 0 0 0) 0x7f14d8059e40 con 0x7f14dc064390 2024-04-24T22:57:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:53.811 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:20.448390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi192.ozemqi","rank":0,"incarnation":20,"state":"up:active","state_seq":194,"addr":"172.21.15.192:6827/2625956780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2625956780},{"type":"v1","addr":"172.21.15.192:6827","nonce":2625956780}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-04-24T22:57:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 msgr2=0x7f14c8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 0x7f14c8070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0f9610 tx=0x7f14cc009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 msgr2=0x7f14dc112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14d8002a50 tx=0x7f14d8004600 comp rx=0 tx=0).stop 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 shutdown_connections 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f14c806e0b0 0x7f14c8070560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14dc102270 0x7f14dc1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 --2- 172.21.15.63:0/892906320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14dc064390 0x7f14dc112c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 >> 172.21.15.63:0/892906320 conn(0x7f14dc071fd0 msgr2=0x7f14dc0fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 shutdown_connections 2024-04-24T22:57:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.812+0000 7f14e3b9d640 1 -- 172.21.15.63:0/892906320 wait complete. 2024-04-24T22:57:53.824 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-24T22:57:53.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 -- 172.21.15.63:0/3968157190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc102b50 msgr2=0x7f2bbc0fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 --2- 172.21.15.63:0/3968157190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc102b50 0x7f2bbc0fc1e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0048d0 tx=0x7f2bac009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 -- 172.21.15.63:0/3968157190 shutdown_connections 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 --2- 172.21.15.63:0/3968157190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc102b50 0x7f2bbc0fc1e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 --2- 172.21.15.63:0/3968157190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 -- 172.21.15.63:0/3968157190 >> 172.21.15.63:0/3968157190 conn(0x7f2bbc0f7d80 msgr2=0x7f2bbc0fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 -- 172.21.15.63:0/3968157190 shutdown_connections 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.921+0000 7f2bc4614640 1 -- 172.21.15.63:0/3968157190 wait complete. 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 Processor -- start 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 -- start start 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc102b50 0x7f2bbc1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc1137e0 con 0x7f2bbc1021b0 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc4614640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc10ce30 con 0x7f2bbc102b50 2024-04-24T22:57:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33370/0 (socket says 172.21.15.63:33370) 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 -- 172.21.15.63:0/1920119338 learned_addr learned my addr 172.21.15.63:0/1920119338 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc1baf640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc102b50 0x7f2bbc1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 -- 172.21.15.63:0/1920119338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc102b50 msgr2=0x7f2bbc1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc102b50 0x7f2bbc1131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.922+0000 7f2bc23b0640 1 -- 172.21.15.63:0/1920119338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac005040 con 0x7f2bbc1021b0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bc23b0640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c830 tx=0x7f2bb8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb800a850 con 0x7f2bbc1021b0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb800ae80 con 0x7f2bbc1021b0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc10d110 con 0x7f2bbc1021b0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb8012ea0 con 0x7f2bbc1021b0 2024-04-24T22:57:53.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.923+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc10d630 con 0x7f2bbc1021b0 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/250409834' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/658450480' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T22:57:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:53 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/740205366' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.924+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc0fd920 con 0x7f2bbc1021b0 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.924+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2bb8012400 con 0x7f2bbc1021b0 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.925+0000 7f2bb2ffd640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 0x7f2ba8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.925+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2bb808dde0 con 0x7f2bbc1021b0 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.925+0000 7f2bc1baf640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 0x7f2ba8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:53.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.925+0000 7f2bc1baf640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 0x7f2ba8070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000c00 tx=0x7f2bac009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:53.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:53.930+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2bb8057190 con 0x7f2bbc1021b0 2024-04-24T22:57:54.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.151+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f2bbc063f00 con 0x7f2bbc1021b0 2024-04-24T22:57:54.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.151+0000 7f2bb2ffd640 1 -- 172.21.15.63:0/1920119338 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v31) v1 ==== 107+0+4271 (secure 0 0 0) 0x7f2bb80126b0 con 0x7f2bbc1021b0 2024-04-24T22:57:54.153 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:54.153 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:33.428237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:54.153 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 25 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 msgr2=0x7f2ba8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 0x7f2ba8070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000c00 tx=0x7f2bac009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 msgr2=0x7f2bbc112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c830 tx=0x7f2bb8004430 comp rx=0 tx=0).stop 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 shutdown_connections 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f2ba806e0b0 0x7f2ba8070560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bbc102b50 0x7f2bbc1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 --2- 172.21.15.63:0/1920119338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bbc1021b0 0x7f2bbc112ca0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.153+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 >> 172.21.15.63:0/1920119338 conn(0x7f2bbc0f7d80 msgr2=0x7f2bbc0f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.154+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 shutdown_connections 2024-04-24T22:57:54.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.154+0000 7f2bc4614640 1 -- 172.21.15.63:0/1920119338 wait complete. 2024-04-24T22:57:54.166 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.263+0000 7f9155b9d640 1 -- 172.21.15.63:0/3221622217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 msgr2=0x7f91501025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.263+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3221622217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f91501025b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f91400048d0 tx=0x7f9140009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- 172.21.15.63:0/3221622217 shutdown_connections 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3221622217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 0x7f91500fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3221622217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f91501025b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- 172.21.15.63:0/3221622217 >> 172.21.15.63:0/3221622217 conn(0x7f91500f7db0 msgr2=0x7f91500fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- 172.21.15.63:0/3221622217 shutdown_connections 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- 172.21.15.63:0/3221622217 wait complete. 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 Processor -- start 2024-04-24T22:57:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- start start 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 0x7f91501131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501137e0 con 0x7f91501021e0 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.264+0000 7f9155b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915010ce30 con 0x7f9150102b80 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33376/0 (socket says 172.21.15.63:33376) 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 -- 172.21.15.63:0/3722298517 learned_addr learned my addr 172.21.15.63:0/3722298517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914effd640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 0x7f91501131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 -- 172.21.15.63:0/3722298517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 msgr2=0x7f91501131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 0x7f91501131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 -- 172.21.15.63:0/3722298517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140005040 con 0x7f91501021e0 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f914f7fe640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9140004930 tx=0x7f9140009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91400094c0 con 0x7f91501021e0 2024-04-24T22:57:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915010d0b0 con 0x7f91501021e0 2024-04-24T22:57:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9140032e50 con 0x7f91501021e0 2024-04-24T22:57:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91400433b0 con 0x7f91501021e0 2024-04-24T22:57:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.265+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915010d570 con 0x7f91501021e0 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.266+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91500fd950 con 0x7f91501021e0 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.267+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f914002f070 con 0x7f91501021e0 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.267+0000 7f9137fff640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 0x7f9138070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.267+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f91400afc40 con 0x7f91501021e0 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.267+0000 7f914effd640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 0x7f9138070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.268+0000 7f914effd640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 0x7f9138070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f915010e560 tx=0x7f9144009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.272+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9140078ef0 con 0x7f91501021e0 2024-04-24T22:57:54.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.494+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f915010e310 con 0x7f91501021e0 2024-04-24T22:57:54.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.494+0000 7f9137fff640 1 -- 172.21.15.63:0/3722298517 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v31) v1 ==== 107+0+4350 (secure 0 0 0) 0x7f914007c5b0 con 0x7f91501021e0 2024-04-24T22:57:54.496 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:54.496 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:33.435811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:54.496 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 msgr2=0x7f9138070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 0x7f9138070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f915010e560 tx=0x7f9144009040 comp rx=0 tx=0).stop 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 msgr2=0x7f9150112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9140004930 tx=0x7f9140009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 shutdown_connections 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f913806e0b0 0x7f9138070560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9150102b80 0x7f91501131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 --2- 172.21.15.63:0/3722298517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91501021e0 0x7f9150112ca0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.496+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 >> 172.21.15.63:0/3722298517 conn(0x7f91500f7db0 msgr2=0x7f91500f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.497+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 shutdown_connections 2024-04-24T22:57:54.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.497+0000 7f9155b9d640 1 -- 172.21.15.63:0/3722298517 wait complete. 2024-04-24T22:57:54.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-24T22:57:54.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 -- 172.21.15.63:0/550675959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 msgr2=0x7f9e8c0fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 --2- 172.21.15.63:0/550675959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c0fc200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e740048d0 tx=0x7f9e74009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 -- 172.21.15.63:0/550675959 shutdown_connections 2024-04-24T22:57:54.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 --2- 172.21.15.63:0/550675959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c0fc200 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 --2- 172.21.15.63:0/550675959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 0x7f9e8c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 -- 172.21.15.63:0/550675959 >> 172.21.15.63:0/550675959 conn(0x7f9e8c0f7d60 msgr2=0x7f9e8c0fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 -- 172.21.15.63:0/550675959 shutdown_connections 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.608+0000 7f9e91607640 1 -- 172.21.15.63:0/550675959 wait complete. 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 Processor -- start 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 -- start start 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 0x7f9e8c112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c1137c0 con 0x7f9e8c102b70 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e91607640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c113930 con 0x7f9e8c1021d0 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33396/0 (socket says 172.21.15.63:33396) 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 -- 172.21.15.63:0/124671284 learned_addr learned my addr 172.21.15.63:0/124671284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:54.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e8affd640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 0x7f9e8c112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 -- 172.21.15.63:0/124671284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 msgr2=0x7f9e8c112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 0x7f9e8c112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.609+0000 7f9e83fff640 1 -- 172.21.15.63:0/124671284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e74005040 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e83fff640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e74004930 tx=0x7f9e74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e740094c0 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e74032e50 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e740433b0 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c10cf60 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.610+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c10d420 con 0x7f9e8c102b70 2024-04-24T22:57:54.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.611+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c0fd940 con 0x7f9e8c102b70 2024-04-24T22:57:54.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.611+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9e7402f070 con 0x7f9e8c102b70 2024-04-24T22:57:54.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.612+0000 7f9e837fe640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 0x7f9e78070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.612+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9e740afb90 con 0x7f9e8c102b70 2024-04-24T22:57:54.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.612+0000 7f9e8affd640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 0x7f9e78070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.612+0000 7f9e8affd640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 0x7f9e78070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c007fd0 tx=0x7f9e7c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.616+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9e74078ec0 con 0x7f9e8c102b70 2024-04-24T22:57:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:54 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/892906320' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T22:57:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:54 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1920119338' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T22:57:54.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:54 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3722298517' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T22:57:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.843+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f9e8c064350 con 0x7f9e8c102b70 2024-04-24T22:57:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.844+0000 7f9e837fe640 1 -- 172.21.15.63:0/124671284 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v31) v1 ==== 107+0+5200 (secure 0 0 0) 0x7f9e7407c580 con 0x7f9e8c102b70 2024-04-24T22:57:54.845 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44313,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3112021694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3112021694},{"type":"v1","addr":"172.21.15.192:6827","nonce":3112021694}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:33.435811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 msgr2=0x7f9e78070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 0x7f9e78070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c007fd0 tx=0x7f9e7c009040 comp rx=0 tx=0).stop 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 msgr2=0x7f9e8c1131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e74004930 tx=0x7f9e74009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 shutdown_connections 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9e7806e0b0 0x7f9e78070560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e8c102b70 0x7f9e8c1131c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 --2- 172.21.15.63:0/124671284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e8c1021d0 0x7f9e8c112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 >> 172.21.15.63:0/124671284 conn(0x7f9e8c0f7d60 msgr2=0x7f9e8c0f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 shutdown_connections 2024-04-24T22:57:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.846+0000 7f9e91607640 1 -- 172.21.15.63:0/124671284 wait complete. 2024-04-24T22:57:54.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-24T22:57:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:54 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/892906320' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T22:57:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:54 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1920119338' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T22:57:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:54 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3722298517' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T22:57:54.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.955+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/4092308908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b080fb150 msgr2=0x7f9b080fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.955+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/4092308908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b080fb150 0x7f9b080fb5c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9af00048d0 tx=0x7f9af0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/4092308908 shutdown_connections 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/4092308908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b080fb150 0x7f9b080fb5c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/4092308908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b08101170 0x7f9b08101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/4092308908 >> 172.21.15.63:0/4092308908 conn(0x7f9b08071fd0 msgr2=0x7f9b080723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/4092308908 shutdown_connections 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/4092308908 wait complete. 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 Processor -- start 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- start start 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b080fb150 0x7f9b0810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08065310 con 0x7f9b08101170 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.956+0000 7f9b0cf5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0810e8e0 con 0x7f9b080fb150 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33416/0 (socket says 172.21.15.63:33416) 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 -- 172.21.15.63:0/3520339990 learned_addr learned my addr 172.21.15.63:0/3520339990 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b0659c640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b080fb150 0x7f9b0810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 -- 172.21.15.63:0/3520339990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b080fb150 msgr2=0x7f9b0810de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b080fb150 0x7f9b0810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 -- 172.21.15.63:0/3520339990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0005040 con 0x7f9b08101170 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b05d9b640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004930 tx=0x7f9af0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af00094c0 con 0x7f9b08101170 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9af0032e50 con 0x7f9b08101170 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af00433b0 con 0x7f9b08101170 2024-04-24T22:57:54.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0810eb60 con 0x7f9b08101170 2024-04-24T22:57:54.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.957+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b08117090 con 0x7f9b08101170 2024-04-24T22:57:54.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.958+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b080fc870 con 0x7f9b08101170 2024-04-24T22:57:54.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.959+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9af002f070 con 0x7f9b08101170 2024-04-24T22:57:54.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.959+0000 7f9afeffd640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 0x7f9af4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:54.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.959+0000 7f9b0659c640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 0x7f9af4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:54.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.959+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9af00af960 con 0x7f9b08101170 2024-04-24T22:57:54.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.960+0000 7f9b0659c640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 0x7f9af4070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007e60 tx=0x7f9b00004bb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:54.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:54.964+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9af0078c90 con 0x7f9b08101170 2024-04-24T22:57:55.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.186+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f9b080fe3a0 con 0x7f9b08101170 2024-04-24T22:57:55.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.187+0000 7f9afeffd640 1 -- 172.21.15.63:0/3520339990 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v31) v1 ==== 107+0+5204 (secure 0 0 0) 0x7f9af007c350 con 0x7f9b08101170 2024-04-24T22:57:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:55.188 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44313,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3112021694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3112021694},{"type":"v1","addr":"172.21.15.192:6827","nonce":3112021694}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:43.113943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:55.189 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 msgr2=0x7f9af4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 0x7f9af4070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007e60 tx=0x7f9b00004bb0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 msgr2=0x7f9b0810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004930 tx=0x7f9af0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 shutdown_connections 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9af406e0b0 0x7f9af4070560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b08101170 0x7f9b0810e370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 --2- 172.21.15.63:0/3520339990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b080fb150 0x7f9b0810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 >> 172.21.15.63:0/3520339990 conn(0x7f9b08071fd0 msgr2=0x7f9b080fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 shutdown_connections 2024-04-24T22:57:55.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.189+0000 7f9b0cf5d640 1 -- 172.21.15.63:0/3520339990 wait complete. 2024-04-24T22:57:55.203 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-24T22:57:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- 172.21.15.63:0/3749532452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101a30 msgr2=0x7f10940fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 --2- 172.21.15.63:0/3749532452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101a30 0x7f10940fb0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10800048d0 tx=0x7f1080009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- 172.21.15.63:0/3749532452 shutdown_connections 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 --2- 172.21.15.63:0/3749532452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101a30 0x7f10940fb0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 --2- 172.21.15.63:0/3749532452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101090 0x7f1094101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- 172.21.15.63:0/3749532452 >> 172.21.15.63:0/3749532452 conn(0x7f10940f6c20 msgr2=0x7f10940f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- 172.21.15.63:0/3749532452 shutdown_connections 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- 172.21.15.63:0/3749532452 wait complete. 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 Processor -- start 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.302+0000 7f109ab5d640 1 -- start start 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f109ab5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f109ab5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101a30 0x7f10940fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f109ab5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940fec10 con 0x7f1094101090 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f109ab5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940fd7b0 con 0x7f1094101a30 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33432/0 (socket says 172.21.15.63:33432) 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 -- 172.21.15.63:0/102667825 learned_addr learned my addr 172.21.15.63:0/102667825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f1093fff640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101a30 0x7f10940fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 -- 172.21.15.63:0/102667825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101a30 msgr2=0x7f10940fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101a30 0x7f10940fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 -- 172.21.15.63:0/102667825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080005040 con 0x7f1094101090 2024-04-24T22:57:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.303+0000 7f10988f9640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1088002a50 tx=0x7f1088004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.304+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1088007c30 con 0x7f1094101090 2024-04-24T22:57:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.304+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f108800f040 con 0x7f1094101090 2024-04-24T22:57:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.304+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940fda90 con 0x7f1094101090 2024-04-24T22:57:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.304+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108800dbe0 con 0x7f1094101090 2024-04-24T22:57:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.304+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10941193d0 con 0x7f1094101090 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.305+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10940fc800 con 0x7f1094101090 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.305+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f108800dd40 con 0x7f1094101090 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.306+0000 7f10917fa640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 0x7f1084070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.306+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f108808db70 con 0x7f1094101090 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.306+0000 7f1093fff640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 0x7f1084070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.306+0000 7f1093fff640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 0x7f1084070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10940fe460 tx=0x7f1080009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.311+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1088056ea0 con 0x7f1094101090 2024-04-24T22:57:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.534+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f10941081d0 con 0x7f1094101090 2024-04-24T22:57:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.534+0000 7f10917fa640 1 -- 172.21.15.63:0/102667825 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v31) v1 ==== 107+0+5201 (secure 0 0 0) 0x7f108805a560 con 0x7f1094101090 2024-04-24T22:57:55.536 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:55.536 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44313,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3112021694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3112021694},{"type":"v1","addr":"172.21.15.192:6827","nonce":3112021694}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:43.696911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:57:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 msgr2=0x7f1084070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 0x7f1084070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10940fe460 tx=0x7f1080009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 msgr2=0x7f10940fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1088002a50 tx=0x7f1088004600 comp rx=0 tx=0).stop 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 shutdown_connections 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f108406e0b0 0x7f1084070560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1094101a30 0x7f10940fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 --2- 172.21.15.63:0/102667825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094101090 0x7f10940fcd30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 >> 172.21.15.63:0/102667825 conn(0x7f10940f6c20 msgr2=0x7f10940f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 shutdown_connections 2024-04-24T22:57:55.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.536+0000 7f109ab5d640 1 -- 172.21.15.63:0/102667825 wait complete. 2024-04-24T22:57:55.549 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-24T22:57:55.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 -- 172.21.15.63:0/1065708782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 msgr2=0x7f9ffc0fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 --2- 172.21.15.63:0/1065708782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc0fa8c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0048d0 tx=0x7f9fec009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 -- 172.21.15.63:0/1065708782 shutdown_connections 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 --2- 172.21.15.63:0/1065708782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc0fa8c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 --2- 172.21.15.63:0/1065708782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 0x7f9ffc102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 -- 172.21.15.63:0/1065708782 >> 172.21.15.63:0/1065708782 conn(0x7f9ffc0717e0 msgr2=0x7f9ffc071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.646+0000 7fa00435d640 1 -- 172.21.15.63:0/1065708782 shutdown_connections 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 -- 172.21.15.63:0/1065708782 wait complete. 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 Processor -- start 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 -- start start 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 0x7f9ffc112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc113870 con 0x7f9ffc102bd0 2024-04-24T22:57:55.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa00435d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc10cec0 con 0x7f9ffc102230 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa0018f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa0018f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33442/0 (socket says 172.21.15.63:33442) 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.647+0000 7fa0018f8640 1 -- 172.21.15.63:0/3168449501 learned_addr learned my addr 172.21.15.63:0/3168449501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa0018f8640 1 -- 172.21.15.63:0/3168449501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 msgr2=0x7f9ffc112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa0020f9640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 0x7f9ffc112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa0018f8640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 0x7f9ffc112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa0018f8640 1 -- 172.21.15.63:0/3168449501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec005040 con 0x7f9ffc102bd0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa0018f8640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02bda0 tx=0x7f9fec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec0094c0 con 0x7f9ffc102bd0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc10d140 con 0x7f9ffc102bd0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fec032e50 con 0x7f9ffc102bd0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec03b400 con 0x7f9ffc102bd0 2024-04-24T22:57:55.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.648+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc10d660 con 0x7f9ffc102bd0 2024-04-24T22:57:55.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.649+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc065530 con 0x7f9ffc102bd0 2024-04-24T22:57:55.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.649+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9fec02f070 con 0x7f9ffc102bd0 2024-04-24T22:57:55.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.650+0000 7f9ff2ffd640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 0x7f9fe8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:55.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.650+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9fec0aff90 con 0x7f9ffc102bd0 2024-04-24T22:57:55.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.650+0000 7fa0020f9640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 0x7f9fe8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:55.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.650+0000 7fa0020f9640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 0x7f9fe8070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8004ce0 tx=0x7f9ff800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:55.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.654+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9fec0e4740 con 0x7f9ffc102bd0 2024-04-24T22:57:55.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:55 smithi063 ceph-mon[111213]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:55 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/124671284' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T22:57:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:55 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3520339990' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T22:57:55.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:55 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/102667825' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T22:57:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.883+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f9ffc109390 con 0x7f9ffc102bd0 2024-04-24T22:57:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.883+0000 7f9ff2ffd640 1 -- 172.21.15.63:0/3168449501 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v31) v1 ==== 107+0+5210 (secure 0 0 0) 0x7f9fec0792c0 con 0x7f9ffc102bd0 2024-04-24T22:57:55.885 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:55.885 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.dnliiv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/3118675914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3118675914},{"type":"v1","addr":"172.21.15.63:6829","nonce":3118675914}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44309,"name":"cephfs.smithi192.mnfuxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3626023513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3626023513},{"type":"v1","addr":"172.21.15.192:6825","nonce":3626023513}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44313,"name":"cephfs.smithi192.ozemqi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3112021694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3112021694},{"type":"v1","addr":"172.21.15.192:6827","nonce":3112021694}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:42:22.073567+0000","modified":"2024-04-24T22:55:45.698723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi063.twfoxu","rank":0,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.63:6827/995129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":995129537},{"type":"v1","addr":"172.21.15.63:6827","nonce":995129537}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-04-24T22:57:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 30 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 msgr2=0x7f9fe8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 0x7f9fe8070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8004ce0 tx=0x7f9ff800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 msgr2=0x7f9ffc113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02bda0 tx=0x7f9fec009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 shutdown_connections 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.885+0000 7fa00435d640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f9fe806e0b0 0x7f9fe8070560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.886+0000 7fa00435d640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc102bd0 0x7f9ffc113270 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.886+0000 7fa00435d640 1 --2- 172.21.15.63:0/3168449501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ffc102230 0x7f9ffc112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.886+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 >> 172.21.15.63:0/3168449501 conn(0x7f9ffc0717e0 msgr2=0x7f9ffc0fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.886+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 shutdown_connections 2024-04-24T22:57:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:55.886+0000 7fa00435d640 1 -- 172.21.15.63:0/3168449501 wait complete. 2024-04-24T22:57:55.898 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T22:57:55.911 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T22:57:55.912 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:57:55.912 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:57:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:55 smithi192 ceph-mon[80463]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:55 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/124671284' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T22:57:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:55 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3520339990' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T22:57:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:55 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/102667825' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T22:57:55.930 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:57:55.930 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:57:55.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2973224908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 msgr2=0x7f88c4102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2973224908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4102590 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f88b40048d0 tx=0x7f88b4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2973224908 shutdown_connections 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2973224908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 0x7f88c40fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2973224908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4102590 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2973224908 >> 172.21.15.63:0/2973224908 conn(0x7f88c40f7d30 msgr2=0x7f88c40fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2973224908 shutdown_connections 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.124+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2973224908 wait complete. 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 Processor -- start 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 -- start start 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 0x7f88c4113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4113850 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c9b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c410cea0 con 0x7f88c4102b60 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33452/0 (socket says 172.21.15.63:33452) 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.125+0000 7f88c37fe640 1 -- 172.21.15.63:0/2389158485 learned_addr learned my addr 172.21.15.63:0/2389158485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c37fe640 1 -- 172.21.15.63:0/2389158485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 msgr2=0x7f88c4113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c2ffd640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 0x7f88c4113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c37fe640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 0x7f88c4113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c37fe640 1 -- 172.21.15.63:0/2389158485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4005040 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c37fe640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004930 tx=0x7f88b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b40094c0 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b4032e50 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c410d120 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b40433b0 con 0x7f88c41021c0 2024-04-24T22:57:56.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.126+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c410d5e0 con 0x7f88c41021c0 2024-04-24T22:57:56.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.127+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c40fd930 con 0x7f88c41021c0 2024-04-24T22:57:56.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.128+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f88b402f070 con 0x7f88c41021c0 2024-04-24T22:57:56.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.128+0000 7f88abfff640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 0x7f88ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.128+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f88b40afa40 con 0x7f88c41021c0 2024-04-24T22:57:56.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.128+0000 7f88c2ffd640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 0x7f88ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.129+0000 7f88c2ffd640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 0x7f88ac070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88c410e5d0 tx=0x7f88b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.133+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f88b4078d70 con 0x7f88c41021c0 2024-04-24T22:57:56.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.319+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88c410e380 con 0x7f88c41021c0 2024-04-24T22:57:56.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.320+0000 7f88abfff640 1 -- 172.21.15.63:0/2389158485 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f88b407c430 con 0x7f88c41021c0 2024-04-24T22:57:56.322 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:56.322 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":83,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:55:33.427955+0000","last_up_change":"2024-04-24T22:54:27.067935+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T22:42:21.011439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T22:42:21.501399+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6803","nonce":3067429146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6805","nonce":3067429146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6809","nonce":3067429146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6807","nonce":3067429146}]},"public_addr":"172.21.15.63:6803/3067429146","cluster_addr":"172.21.15.63:6805/3067429146","heartbeat_back_addr":"172.21.15.63:6809/3067429146","heartbeat_front_addr":"172.21.15.63:6807/3067429146","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6811","nonce":2869549179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6813","nonce":2869549179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6817","nonce":2869549179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6815","nonce":2869549179}]},"public_addr":"172.21.15.63:6811/2869549179","cluster_addr":"172.21.15.63:6813/2869549179","heartbeat_back_addr":"172.21.15.63:6817/2869549179","heartbeat_front_addr":"172.21.15.63:6815/2869549179","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6819","nonce":2703892347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6821","nonce":2703892347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6825","nonce":2703892347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6823","nonce":2703892347}]},"public_addr":"172.21.15.63:6819/2703892347","cluster_addr":"172.21.15.63:6821/2703892347","heartbeat_back_addr":"172.21.15.63:6825/2703892347","heartbeat_front_addr":"172.21.15.63:6823/2703892347","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6801","nonce":534042226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6803","nonce":534042226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6807","nonce":534042226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6805","nonce":534042226}]},"public_addr":"172.21.15.192:6801/534042226","cluster_addr":"172.21.15.192:6803/534042226","heartbeat_back_addr":"172.21.15.192:6807/534042226","heartbeat_front_addr":"172.21.15.192:6805/534042226","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6809","nonce":2986357084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6811","nonce":2986357084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6815","nonce":2986357084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6813","nonce":2986357084}]},"public_addr":"172.21.15.192:6809/2986357084","cluster_addr":"172.21.15.192:6811/2986357084","heartbeat_back_addr":"172.21.15.192:6815/2986357084","heartbeat_front_addr":"172.21.15.192:6813/2986357084","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6817","nonce":3828556210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6819","nonce":3828556210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6823","nonce":3828556210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6821","nonce":3828556210}]},"public_addr":"172.21.15.192:6817/3828556210","cluster_addr":"172.21.15.192:6819/3828556210","heartbeat_back_addr":"172.21.15.192:6823/3828556210","heartbeat_front_addr":"172.21.15.192:6821/3828556210","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6827/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6826/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6825/780449385":"2024-04-25T22:55:08.099739+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/4263661376":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2702451571":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2342719260":"2024-04-25T22:45:09.797346+0000","172.21.15.192:0/2604587148":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.192:0/370487052":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/2143811635":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/3666997364":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1700011071":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/3056400683":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6826/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:6801/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6827/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:0/3239723549":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/3265850425":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.192:6828/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.192:6829/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:6801/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/1398662955":"2024-04-25T22:50:00.020010+0000","172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.192:6824/780449385":"2024-04-25T22:55:08.099739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:57:56.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 msgr2=0x7f88ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 0x7f88ac070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88c410e5d0 tx=0x7f88b800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:56.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 msgr2=0x7f88c4112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004930 tx=0x7f88b4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 shutdown_connections 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f88ac06e0b0 0x7f88ac070560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88c4102b60 0x7f88c4113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 --2- 172.21.15.63:0/2389158485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88c41021c0 0x7f88c4112d10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 >> 172.21.15.63:0/2389158485 conn(0x7f88c40f7d30 msgr2=0x7f88c40f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 shutdown_connections 2024-04-24T22:57:56.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.323+0000 7f88c9b9d640 1 -- 172.21.15.63:0/2389158485 wait complete. 2024-04-24T22:57:56.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:54.076844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T22:42:21.011439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T22:42:21.501399+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:57:56.336 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:57:56.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 -- 172.21.15.63:0/433561976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 msgr2=0x7f11340fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 --2- 172.21.15.63:0/433561976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f11340fb0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f11240048d0 tx=0x7f1124009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 -- 172.21.15.63:0/433561976 shutdown_connections 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 --2- 172.21.15.63:0/433561976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f11340fb0d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 --2- 172.21.15.63:0/433561976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 0x7f1134101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 -- 172.21.15.63:0/433561976 >> 172.21.15.63:0/433561976 conn(0x7f11340f6cb0 msgr2=0x7f11340f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 -- 172.21.15.63:0/433561976 shutdown_connections 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.434+0000 7f113ba25640 1 -- 172.21.15.63:0/433561976 wait complete. 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 Processor -- start 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 -- start start 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 0x7f113410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340664a0 con 0x7f1134101a40 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f113ba25640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113410e990 con 0x7f11341010a0 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33474/0 (socket says 172.21.15.63:33474) 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 -- 172.21.15.63:0/71647211 learned_addr learned my addr 172.21.15.63:0/71647211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f11397c1640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 0x7f113410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 -- 172.21.15.63:0/71647211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 msgr2=0x7f113410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 0x7f113410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.435+0000 7f1138fc0640 1 -- 172.21.15.63:0/71647211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124005040 con 0x7f1134101a40 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f1138fc0640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1124004930 tx=0x7f1124009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11240094c0 con 0x7f1134101a40 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1124032e50 con 0x7f1134101a40 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113410ec10 con 0x7f1134101a40 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11240433b0 con 0x7f1134101a40 2024-04-24T22:57:56.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.436+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1134117140 con 0x7f1134101a40 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.437+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11340fc810 con 0x7f1134101a40 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.438+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f112402f070 con 0x7f1134101a40 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.438+0000 7f1129ffb640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 0x7f1120070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.438+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f11240af9d0 con 0x7f1134101a40 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.438+0000 7f11397c1640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 0x7f1120070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.439+0000 7f11397c1640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 0x7f1120070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1130004ce0 tx=0x7f113000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.443+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1124078d00 con 0x7f1134101a40 2024-04-24T22:57:56.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.627+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1134117330 con 0x7f1134101a40 2024-04-24T22:57:56.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.627+0000 7f1129ffb640 1 -- 172.21.15.63:0/71647211 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f112407c3c0 con 0x7f1134101a40 2024-04-24T22:57:56.630 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 msgr2=0x7f1120070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 0x7f1120070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1130004ce0 tx=0x7f113000b040 comp rx=0 tx=0).stop 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 msgr2=0x7f113410e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1124004930 tx=0x7f1124009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 shutdown_connections 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f112006e0b0 0x7f1120070560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1134101a40 0x7f113410e420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 --2- 172.21.15.63:0/71647211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11341010a0 0x7f113410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 >> 172.21.15.63:0/71647211 conn(0x7f11340f6cb0 msgr2=0x7f11340f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.629+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 shutdown_connections 2024-04-24T22:57:56.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.630+0000 7f113ba25640 1 -- 172.21.15.63:0/71647211 wait complete. 2024-04-24T22:57:56.641 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:56 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3168449501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T22:57:56.641 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:56 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2389158485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:56.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T22:57:56.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 -- 172.21.15.63:0/1355540883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fdcc0 msgr2=0x7fba7c106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 --2- 172.21.15.63:0/1355540883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c106390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba700048d0 tx=0x7fba70009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 -- 172.21.15.63:0/1355540883 shutdown_connections 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 --2- 172.21.15.63:0/1355540883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c106390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 --2- 172.21.15.63:0/1355540883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c0fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 -- 172.21.15.63:0/1355540883 >> 172.21.15.63:0/1355540883 conn(0x7fba7c0f6e60 msgr2=0x7fba7c0f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 -- 172.21.15.63:0/1355540883 shutdown_connections 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.741+0000 7fba8115d640 1 -- 172.21.15.63:0/1355540883 wait complete. 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 Processor -- start 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 -- start start 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c113a90 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba8115d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c10d0e0 con 0x7fba7c0fdcc0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33490/0 (socket says 172.21.15.63:33490) 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 -- 172.21.15.63:0/2096848511 learned_addr learned my addr 172.21.15.63:0/2096848511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7a59c640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 -- 172.21.15.63:0/2096848511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 msgr2=0x7fba7c113490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c113490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7ad9d640 1 -- 172.21.15.63:0/2096848511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70005040 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.742+0000 7fba7a59c640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c113490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba7ad9d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00eb70 tx=0x7fba6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba6c007c30 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba6c00f040 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c10d360 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba6c0095b0 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.743+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c10d880 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.744+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c065dd0 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.744+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fba6c009730 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.745+0000 7fba637fe640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 0x7fba64070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:56.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.745+0000 7fba7a59c640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 0x7fba64070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:56.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.745+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fba6c08d910 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.745+0000 7fba7a59c640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 0x7fba64070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba7c10e810 tx=0x7fba70009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:56.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.750+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fba6c056c40 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:56 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3168449501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T22:57:56.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:56 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2389158485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.935+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fba7c109510 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.936+0000 7fba637fe640 1 -- 172.21.15.63:0/2096848511 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fba6c05a300 con 0x7fba7c0fd3b0 2024-04-24T22:57:56.938 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 msgr2=0x7fba64070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 0x7fba64070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba7c10e810 tx=0x7fba70009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 msgr2=0x7fba7c112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00eb70 tx=0x7fba6c00b040 comp rx=0 tx=0).stop 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 shutdown_connections 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fba6406e0b0 0x7fba64070560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba7c0fdcc0 0x7fba7c113490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 --2- 172.21.15.63:0/2096848511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba7c0fd3b0 0x7fba7c112f50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 >> 172.21.15.63:0/2096848511 conn(0x7fba7c0f6e60 msgr2=0x7fba7c0f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 shutdown_connections 2024-04-24T22:57:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:56.938+0000 7fba8115d640 1 -- 172.21.15.63:0/2096848511 wait complete. 2024-04-24T22:57:56.951 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- 172.21.15.63:0/1867606010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102190 msgr2=0x7fb1e0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 --2- 172.21.15.63:0/1867606010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102190 0x7fb1e0102560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00048d0 tx=0x7fb1d0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- 172.21.15.63:0/1867606010 shutdown_connections 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 --2- 172.21.15.63:0/1867606010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102b30 0x7fb1e00fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 --2- 172.21.15.63:0/1867606010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102190 0x7fb1e0102560 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- 172.21.15.63:0/1867606010 >> 172.21.15.63:0/1867606010 conn(0x7fb1e00f7da0 msgr2=0x7fb1e00fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- 172.21.15.63:0/1867606010 shutdown_connections 2024-04-24T22:57:57.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- 172.21.15.63:0/1867606010 wait complete. 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 Processor -- start 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.049+0000 7fb1e555d640 1 -- start start 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102190 0x7fb1e0112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0113800 con 0x7fb1e0102b30 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e010ce50 con 0x7fb1e0102190 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33512/0 (socket says 172.21.15.63:33512) 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 -- 172.21.15.63:0/351811782 learned_addr learned my addr 172.21.15.63:0/351811782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1deffd640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102190 0x7fb1e0112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 -- 172.21.15.63:0/351811782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102190 msgr2=0x7fb1e0112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102190 0x7fb1e0112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 -- 172.21.15.63:0/351811782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0005040 con 0x7fb1e0102b30 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1de7fc640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c830 tx=0x7fb1c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c800a850 con 0x7fb1e0102b30 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1c800ae80 con 0x7fb1e0102b30 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e010d130 con 0x7fb1e0102b30 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c8012ea0 con 0x7fb1e0102b30 2024-04-24T22:57:57.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.050+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e010d650 con 0x7fb1e0102b30 2024-04-24T22:57:57.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.051+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e00fd900 con 0x7fb1e0102b30 2024-04-24T22:57:57.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.052+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb1c8012400 con 0x7fb1e0102b30 2024-04-24T22:57:57.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.052+0000 7fb1d77fe640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 0x7fb1cc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.052+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb1c808dde0 con 0x7fb1e0102b30 2024-04-24T22:57:57.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.052+0000 7fb1deffd640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 0x7fb1cc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.053+0000 7fb1deffd640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 0x7fb1cc070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.057+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb1c8057190 con 0x7fb1e0102b30 2024-04-24T22:57:57.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.241+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb1e004bcc0 con 0x7fb1e0102b30 2024-04-24T22:57:57.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.241+0000 7fb1d77fe640 1 -- 172.21.15.63:0/351811782 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb1c80126b0 con 0x7fb1e0102b30 2024-04-24T22:57:57.244 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:57.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 msgr2=0x7fb1cc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 0x7fb1cc070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 msgr2=0x7fb1e0113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c830 tx=0x7fb1c8004430 comp rx=0 tx=0).stop 2024-04-24T22:57:57.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 shutdown_connections 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7fb1cc06e0b0 0x7fb1cc070560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1e0102b30 0x7fb1e0113200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 --2- 172.21.15.63:0/351811782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1e0102190 0x7fb1e0112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.243+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 >> 172.21.15.63:0/351811782 conn(0x7fb1e00f7da0 msgr2=0x7fb1e00f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.244+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 shutdown_connections 2024-04-24T22:57:57.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.244+0000 7fb1e555d640 1 -- 172.21.15.63:0/351811782 wait complete. 2024-04-24T22:57:57.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:57:57.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- 172.21.15.63:0/1215145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 msgr2=0x7f80200fa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 --2- 172.21.15.63:0/1215145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80200fa8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8010009c50 tx=0x7f8010004d10 comp rx=0 tx=0).stop 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- 172.21.15.63:0/1215145 shutdown_connections 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 --2- 172.21.15.63:0/1215145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80200fa8b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 --2- 172.21.15.63:0/1215145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 0x7f8020102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- 172.21.15.63:0/1215145 >> 172.21.15.63:0/1215145 conn(0x7f8020070890 msgr2=0x7f8020070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- 172.21.15.63:0/1215145 shutdown_connections 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- 172.21.15.63:0/1215145 wait complete. 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 Processor -- start 2024-04-24T22:57:57.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.354+0000 7f8026e07640 1 -- start start 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f8026e07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 0x7f8020112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f8026e07640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f8026e07640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201138d0 con 0x7f8020102b60 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f8026e07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802010cf20 con 0x7f80201021c0 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33526/0 (socket says 172.21.15.63:33526) 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 -- 172.21.15.63:0/3163641698 learned_addr learned my addr 172.21.15.63:0/3163641698 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f8024ba3640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 0x7f8020112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 -- 172.21.15.63:0/3163641698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 msgr2=0x7f8020112d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 0x7f8020112d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 -- 172.21.15.63:0/3163641698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100098b0 con 0x7f8020102b60 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.355+0000 7f801ffff640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8010004a30 tx=0x7f8010004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.356+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801003a070 con 0x7f8020102b60 2024-04-24T22:57:57.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.356+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8010009710 con 0x7f8020102b60 2024-04-24T22:57:57.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.356+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802010d1a0 con 0x7f8020102b60 2024-04-24T22:57:57.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.356+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801003ea10 con 0x7f8020102b60 2024-04-24T22:57:57.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.356+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802010d660 con 0x7f8020102b60 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.357+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200652a0 con 0x7f8020102b60 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.357+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8010032070 con 0x7f8020102b60 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.358+0000 7f801d7fa640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 0x7f800c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.358+0000 7f8024ba3640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 0x7f800c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.358+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f80100af970 con 0x7f8020102b60 2024-04-24T22:57:57.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.358+0000 7f8024ba3640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 0x7f800c070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8018009af0 tx=0x7f801800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.363+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8010078dd0 con 0x7f8020102b60 2024-04-24T22:57:57.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.559+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8020109340 con 0x7f8020102b60 2024-04-24T22:57:57.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.559+0000 7f801d7fa640 1 -- 172.21.15.63:0/3163641698 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v83) v1 ==== 81+0+2317 (secure 0 0 0) 0x7f801007c490 con 0x7f8020102b60 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6827/2625956780 2024-04-25T22:55:33.427869+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6826/2625956780 2024-04-25T22:55:33.427869+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6825/780449385 2024-04-25T22:55:08.099739+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3274298824 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/622073725 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2614941819 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/201758584 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:57.562 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4263661376 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2508233848 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3694082065 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2702451571 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2342719260 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/2604587148 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1454404068 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1764753728 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2691264506 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2691264506 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/370487052 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/2143811635 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/3666997364 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.563 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1700011071 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3585828238 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3056400683 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/776980192 2024-04-25T22:42:37.667991+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/622073725 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/776980192 2024-04-25T22:42:37.667991+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3239723549 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3265850425 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2572161006 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6828/4284728814 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2164211416 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6829/4284728814 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3227237455 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1398662955 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.564 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/201758584 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:57.565 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2614941819 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:57.565 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2293418655 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:57.565 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3227237455 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:57.565 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6824/780449385 2024-04-25T22:55:08.099739+0000 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:listed 39 entries 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 msgr2=0x7f800c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 0x7f800c070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8018009af0 tx=0x7f801800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 msgr2=0x7f80201132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8010004a30 tx=0x7f8010004d10 comp rx=0 tx=0).stop 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 shutdown_connections 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f800c06e0b0 0x7f800c070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8020102b60 0x7f80201132d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 --2- 172.21.15.63:0/3163641698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80201021c0 0x7f8020112d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 >> 172.21.15.63:0/3163641698 conn(0x7f8020070890 msgr2=0x7f8020104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 shutdown_connections 2024-04-24T22:57:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.561+0000 7f8026e07640 1 -- 172.21.15.63:0/3163641698 wait complete. 2024-04-24T22:57:57.575 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-24T22:57:57.575 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:57:57.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 -- 172.21.15.63:0/2826062251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 msgr2=0x7f33f8064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 --2- 172.21.15.63:0/2826062251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8064350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f33e00048d0 tx=0x7f33e0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 -- 172.21.15.63:0/2826062251 shutdown_connections 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 --2- 172.21.15.63:0/2826062251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8064350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 --2- 172.21.15.63:0/2826062251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 0x7f33f81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 -- 172.21.15.63:0/2826062251 >> 172.21.15.63:0/2826062251 conn(0x7f33f80711c0 msgr2=0x7f33f80715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 -- 172.21.15.63:0/2826062251 shutdown_connections 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.727+0000 7f33fd95d640 1 -- 172.21.15.63:0/2826062251 wait complete. 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 Processor -- start 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 -- start start 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 0x7f33f8112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8113870 con 0x7f33f8102b90 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33fd95d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f810cec0 con 0x7f33f81021f0 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33548/0 (socket says 172.21.15.63:33548) 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 -- 172.21.15.63:0/427059820 learned_addr learned my addr 172.21.15.63:0/427059820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33f6ffd640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 0x7f33f8112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 -- 172.21.15.63:0/427059820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 msgr2=0x7f33f8112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 0x7f33f8112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.728+0000 7f33effff640 1 -- 172.21.15.63:0/427059820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0005040 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33effff640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33e002bda0 tx=0x7f33e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e00094c0 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33e0032e50 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e003b400 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f810d140 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.729+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f810d660 con 0x7f33f8102b90 2024-04-24T22:57:57.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.730+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f80fd990 con 0x7f33f8102b90 2024-04-24T22:57:57.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.730+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f33e002f070 con 0x7f33f8102b90 2024-04-24T22:57:57.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.731+0000 7f33ef7fe640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 0x7f33e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:57.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.731+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f33e00afec0 con 0x7f33f8102b90 2024-04-24T22:57:57.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.731+0000 7f33f6ffd640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 0x7f33e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:57.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.731+0000 7f33f6ffd640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 0x7f33e4070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33e8004ed0 tx=0x7f33e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:57.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.735+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f33e00791f0 con 0x7f33f8102b90 2024-04-24T22:57:57.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.921+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33f8109370 con 0x7f33f8102b90 2024-04-24T22:57:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.922+0000 7f33ef7fe640 1 -- 172.21.15.63:0/427059820 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f33e002d040 con 0x7f33f8102b90 2024-04-24T22:57:57.924 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-24T22:57:57.925 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":83,"fsid":"c8a2676e-028a-11ef-bc93-c7b262605968","created":"2024-04-24T22:35:21.764694+0000","modified":"2024-04-24T22:55:33.427955+0000","last_up_change":"2024-04-24T22:54:27.067935+0000","last_in_change":"2024-04-24T22:40:47.713765+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:54.076844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T22:42:21.011439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T22:42:21.501399+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"61754976-3983-4832-86d4-497489f4e01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6803","nonce":3067429146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6805","nonce":3067429146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6809","nonce":3067429146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3067429146},{"type":"v1","addr":"172.21.15.63:6807","nonce":3067429146}]},"public_addr":"172.21.15.63:6803/3067429146","cluster_addr":"172.21.15.63:6805/3067429146","heartbeat_back_addr":"172.21.15.63:6809/3067429146","heartbeat_front_addr":"172.21.15.63:6807/3067429146","state":["exists","up"]},{"osd":1,"uuid":"712ed6a3-ce72-4ba3-9b29-ec0f4f2b78b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6811","nonce":2869549179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6813","nonce":2869549179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6817","nonce":2869549179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2869549179},{"type":"v1","addr":"172.21.15.63:6815","nonce":2869549179}]},"public_addr":"172.21.15.63:6811/2869549179","cluster_addr":"172.21.15.63:6813/2869549179","heartbeat_back_addr":"172.21.15.63:6817/2869549179","heartbeat_front_addr":"172.21.15.63:6815/2869549179","state":["exists","up"]},{"osd":2,"uuid":"8955baa8-680f-44bd-86b8-7ac429ed953b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6819","nonce":2703892347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6821","nonce":2703892347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6825","nonce":2703892347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2703892347},{"type":"v1","addr":"172.21.15.63:6823","nonce":2703892347}]},"public_addr":"172.21.15.63:6819/2703892347","cluster_addr":"172.21.15.63:6821/2703892347","heartbeat_back_addr":"172.21.15.63:6825/2703892347","heartbeat_front_addr":"172.21.15.63:6823/2703892347","state":["exists","up"]},{"osd":3,"uuid":"36de3a3c-de49-41f6-9ef3-d999b18c99f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6801","nonce":534042226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6803","nonce":534042226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6807","nonce":534042226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":534042226},{"type":"v1","addr":"172.21.15.192:6805","nonce":534042226}]},"public_addr":"172.21.15.192:6801/534042226","cluster_addr":"172.21.15.192:6803/534042226","heartbeat_back_addr":"172.21.15.192:6807/534042226","heartbeat_front_addr":"172.21.15.192:6805/534042226","state":["exists","up"]},{"osd":4,"uuid":"33041276-b462-4198-b31a-6c0360b1159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6809","nonce":2986357084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6811","nonce":2986357084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6815","nonce":2986357084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2986357084},{"type":"v1","addr":"172.21.15.192:6813","nonce":2986357084}]},"public_addr":"172.21.15.192:6809/2986357084","cluster_addr":"172.21.15.192:6811/2986357084","heartbeat_back_addr":"172.21.15.192:6815/2986357084","heartbeat_front_addr":"172.21.15.192:6813/2986357084","state":["exists","up"]},{"osd":5,"uuid":"5e17e5ed-ac3f-489f-a2fd-b399694502b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6817","nonce":3828556210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6819","nonce":3828556210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6823","nonce":3828556210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3828556210},{"type":"v1","addr":"172.21.15.192:6821","nonce":3828556210}]},"public_addr":"172.21.15.192:6817/3828556210","cluster_addr":"172.21.15.192:6819/3828556210","heartbeat_back_addr":"172.21.15.192:6823/3828556210","heartbeat_front_addr":"172.21.15.192:6821/3828556210","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:38:54.119159+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:21.202455+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:50.476052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:13.386169+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:40:39.443233+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:41:04.703484+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6827/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6826/2625956780":"2024-04-25T22:55:33.427869+0000","172.21.15.192:6825/780449385":"2024-04-25T22:55:08.099739+0000","172.21.15.63:0/3274298824":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6800/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6800/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:6801/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/4263661376":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/2508233848":"2024-04-25T22:36:26.080101+0000","172.21.15.63:0/3694082065":"2024-04-25T22:37:41.558564+0000","172.21.15.63:0/2702451571":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2342719260":"2024-04-25T22:45:09.797346+0000","172.21.15.192:0/2604587148":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1454404068":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/1764753728":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6801/2691264506":"2024-04-25T22:36:26.080101+0000","172.21.15.192:0/370487052":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/2143811635":"2024-04-25T22:46:56.899597+0000","172.21.15.192:0/3666997364":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/1700011071":"2024-04-25T22:45:09.797346+0000","172.21.15.63:0/3585828238":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/3056400683":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6826/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:6801/622073725":"2024-04-25T22:45:09.797346+0000","172.21.15.63:6827/776980192":"2024-04-25T22:42:37.667991+0000","172.21.15.63:0/3239723549":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/3265850425":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/2572161006":"2024-04-25T22:37:41.558564+0000","172.21.15.192:6828/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:0/2164211416":"2024-04-25T22:35:55.097488+0000","172.21.15.192:6829/4284728814":"2024-04-25T22:46:56.899597+0000","172.21.15.63:6801/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.63:0/1398662955":"2024-04-25T22:50:00.020010+0000","172.21.15.63:6800/201758584":"2024-04-25T22:37:41.558564+0000","172.21.15.63:6801/2614941819":"2024-04-25T22:35:55.097488+0000","172.21.15.63:0/2293418655":"2024-04-25T22:36:26.080101+0000","172.21.15.63:6800/3227237455":"2024-04-25T22:50:00.020010+0000","172.21.15.192:6824/780449385":"2024-04-25T22:55:08.099739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:57:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 msgr2=0x7f33e4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 0x7f33e4070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33e8004ed0 tx=0x7f33e800b040 comp rx=0 tx=0).stop 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 msgr2=0x7f33f8113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33e002bda0 tx=0x7f33e0009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 shutdown_connections 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f33e406e0b0 0x7f33e4070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33f8102b90 0x7f33f8113270 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 --2- 172.21.15.63:0/427059820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33f81021f0 0x7f33f8112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 >> 172.21.15.63:0/427059820 conn(0x7f33f80711c0 msgr2=0x7f33f80fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 shutdown_connections 2024-04-24T22:57:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:57.924+0000 7f33fd95d640 1 -- 172.21.15.63:0/427059820 wait complete. 2024-04-24T22:57:57.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:57 smithi192 ceph-mon[80463]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:57.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:57 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/71647211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:57 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/2096848511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:57 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/351811782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.928 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:57 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3163641698' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:57:57.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:57 smithi063 ceph-mon[111213]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:57.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:57 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/71647211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:57 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/2096848511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:57 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/351811782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:57.937 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:57 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3163641698' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:57:57.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:54.076844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T22:42:21.011439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T22:42:21.501399+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:57:57.939 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:57:58.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- 172.21.15.63:0/2573085113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e200fa840 msgr2=0x7f1e200facb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 --2- 172.21.15.63:0/2573085113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e200fa840 0x7f1e200facb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0048d0 tx=0x7f1e1c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- 172.21.15.63:0/2573085113 shutdown_connections 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 --2- 172.21.15.63:0/2573085113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e200fa840 0x7f1e200facb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 --2- 172.21.15.63:0/2573085113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20102240 0x7f1e20102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- 172.21.15.63:0/2573085113 >> 172.21.15.63:0/2573085113 conn(0x7f1e20071700 msgr2=0x7f1e20071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- 172.21.15.63:0/2573085113 shutdown_connections 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- 172.21.15.63:0/2573085113 wait complete. 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 Processor -- start 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- start start 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fa840 0x7f1e20112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20113970 con 0x7f1e20102240 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.037+0000 7f1e2815d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010cf30 con 0x7f1e200fa840 2024-04-24T22:57:58.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33572/0 (socket says 172.21.15.63:33572) 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 -- 172.21.15.63:0/1094204451 learned_addr learned my addr 172.21.15.63:0/1094204451 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e25ef9640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fa840 0x7f1e20112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 -- 172.21.15.63:0/1094204451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fa840 msgr2=0x7f1e20112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fa840 0x7f1e20112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 -- 172.21.15.63:0/1094204451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c005040 con 0x7f1e20102240 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e256f8640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004930 tx=0x7f1e1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c0094c0 con 0x7f1e20102240 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2010d1b0 con 0x7f1e20102240 2024-04-24T22:57:58.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e1c032e50 con 0x7f1e20102240 2024-04-24T22:57:58.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c0433b0 con 0x7f1e20102240 2024-04-24T22:57:58.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.038+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2010d670 con 0x7f1e20102240 2024-04-24T22:57:58.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.039+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e200652b0 con 0x7f1e20102240 2024-04-24T22:57:58.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.040+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e1c02f070 con 0x7f1e20102240 2024-04-24T22:57:58.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.040+0000 7f1e127fc640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 0x7f1e0c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.040+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1e1c0afb20 con 0x7f1e20102240 2024-04-24T22:57:58.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.040+0000 7f1e25ef9640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 0x7f1e0c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.041+0000 7f1e25ef9640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 0x7f1e0c070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e20064e10 tx=0x7f1e14008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.045+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1e1c078e50 con 0x7f1e20102240 2024-04-24T22:57:58.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.228+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e201094a0 con 0x7f1e20102240 2024-04-24T22:57:58.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.229+0000 7f1e127fc640 1 -- 172.21.15.63:0/1094204451 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e1c07c510 con 0x7f1e20102240 2024-04-24T22:57:58.231 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-04-24T22:57:58.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 msgr2=0x7f1e0c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 0x7f1e0c070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e20064e10 tx=0x7f1e14008040 comp rx=0 tx=0).stop 2024-04-24T22:57:58.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 msgr2=0x7f1e201132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004930 tx=0x7f1e1c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 shutdown_connections 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f1e0c06e0b0 0x7f1e0c070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e20102240 0x7f1e201132e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 --2- 172.21.15.63:0/1094204451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e200fa840 0x7f1e20112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 >> 172.21.15.63:0/1094204451 conn(0x7f1e20071700 msgr2=0x7f1e200f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 shutdown_connections 2024-04-24T22:57:58.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.231+0000 7f1e2815d640 1 -- 172.21.15.63:0/1094204451 wait complete. 2024-04-24T22:57:58.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T22:57:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 -- 172.21.15.63:0/3566039074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 msgr2=0x7feecc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 --2- 172.21.15.63:0/3566039074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc102610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feec40048d0 tx=0x7feec4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 -- 172.21.15.63:0/3566039074 shutdown_connections 2024-04-24T22:57:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 --2- 172.21.15.63:0/3566039074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 0x7feecc0facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 --2- 172.21.15.63:0/3566039074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc102610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 -- 172.21.15.63:0/3566039074 >> 172.21.15.63:0/3566039074 conn(0x7feecc071700 msgr2=0x7feecc071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 -- 172.21.15.63:0/3566039074 shutdown_connections 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.342+0000 7feed16ae640 1 -- 172.21.15.63:0/3566039074 wait complete. 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 Processor -- start 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 -- start start 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 0x7feecc112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc1138c0 con 0x7feecc102240 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feed16ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc10cf10 con 0x7feecc0fa840 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33602/0 (socket says 172.21.15.63:33602) 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 -- 172.21.15.63:0/436007837 learned_addr learned my addr 172.21.15.63:0/436007837 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feecaffd640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 0x7feecc112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 -- 172.21.15.63:0/436007837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 msgr2=0x7feecc112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 0x7feecc112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 -- 172.21.15.63:0/436007837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4005040 con 0x7feecc102240 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.343+0000 7feeca7fc640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c790 tx=0x7feeb40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.344+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb400a820 con 0x7feecc102240 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.344+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeb400ae50 con 0x7feecc102240 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.344+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feecc10d1f0 con 0x7feecc102240 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.344+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb40115e0 con 0x7feecc102240 2024-04-24T22:57:58.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.344+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc10d710 con 0x7feecc102240 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.345+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feecc0652b0 con 0x7feecc102240 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.345+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feeb4012400 con 0x7feecc102240 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.345+0000 7feec37fe640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 0x7feeb8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.346+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7feeb408dd30 con 0x7feecc102240 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.346+0000 7feecaffd640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 0x7feeb8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.346+0000 7feecaffd640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 0x7feeb8070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.350+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feeb40570e0 con 0x7feecc102240 2024-04-24T22:57:58.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.534+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feecc109420 con 0x7feecc102240 2024-04-24T22:57:58.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.534+0000 7feec37fe640 1 -- 172.21.15.63:0/436007837 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7feeb405a7a0 con 0x7feecc102240 2024-04-24T22:57:58.537 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:58.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 msgr2=0x7feeb8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 0x7feeb8070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 msgr2=0x7feecc1132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c790 tx=0x7feeb40043c0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 shutdown_connections 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7feeb806e0b0 0x7feeb8070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feecc102240 0x7feecc1132c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 --2- 172.21.15.63:0/436007837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feecc0fa840 0x7feecc112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 >> 172.21.15.63:0/436007837 conn(0x7feecc071700 msgr2=0x7feecc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 shutdown_connections 2024-04-24T22:57:58.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.537+0000 7feed16ae640 1 -- 172.21.15.63:0/436007837 wait complete. 2024-04-24T22:57:58.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T22:57:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 -- 172.21.15.63:0/1944305699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fdba0 msgr2=0x7f3e801061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 --2- 172.21.15.63:0/1944305699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fdba0 0x7f3e801061e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c004900 tx=0x7f3e6c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 -- 172.21.15.63:0/1944305699 shutdown_connections 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 --2- 172.21.15.63:0/1944305699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fdba0 0x7f3e801061e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 --2- 172.21.15.63:0/1944305699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fd200 0x7f3e800fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 -- 172.21.15.63:0/1944305699 >> 172.21.15.63:0/1944305699 conn(0x7f3e800f6cb0 msgr2=0x7f3e800f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 -- 172.21.15.63:0/1944305699 shutdown_connections 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.651+0000 7f3e86cae640 1 -- 172.21.15.63:0/1944305699 wait complete. 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 Processor -- start 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 -- start start 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fdba0 0x7f3e80113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80113860 con 0x7f3e800fd200 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e86cae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8010ceb0 con 0x7f3e800fdba0 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36578/0 (socket says 172.21.15.63:36578) 2024-04-24T22:57:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 -- 172.21.15.63:0/3971335155 learned_addr learned my addr 172.21.15.63:0/3971335155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e7ffff640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fdba0 0x7f3e80113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 -- 172.21.15.63:0/3971335155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fdba0 msgr2=0x7f3e80113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fdba0 0x7f3e80113260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.652+0000 7f3e84a4a640 1 -- 172.21.15.63:0/3971335155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c005040 con 0x7f3e800fd200 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e84a4a640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e74002a50 tx=0x7f3e74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e74007c30 con 0x7f3e800fd200 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8010d190 con 0x7f3e800fd200 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e7400f040 con 0x7f3e800fd200 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e7400dbe0 con 0x7f3e800fd200 2024-04-24T22:57:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.653+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8010d6b0 con 0x7f3e800fd200 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.654+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e80065d90 con 0x7f3e800fd200 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.654+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e7400dd40 con 0x7f3e800fd200 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.655+0000 7f3e7d7fa640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 0x7f3e70070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.655+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3e7408db70 con 0x7f3e800fd200 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.655+0000 7f3e7ffff640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 0x7f3e70070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.655+0000 7f3e7ffff640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 0x7f3e70070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.660+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3e74056ea0 con 0x7f3e800fd200 2024-04-24T22:57:58.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:58 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/427059820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:58.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:58 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/1094204451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:58.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:58 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/436007837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:58.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.843+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3e801092a0 con 0x7f3e800fd200 2024-04-24T22:57:58.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.844+0000 7f3e7d7fa640 1 -- 172.21.15.63:0/3971335155 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3e7405a560 con 0x7f3e800fd200 2024-04-24T22:57:58.846 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-04-24T22:57:58.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 msgr2=0x7f3e70070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 0x7f3e70070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c009da0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 msgr2=0x7f3e80112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e74002a50 tx=0x7f3e74004600 comp rx=0 tx=0).stop 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 shutdown_connections 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3e7006e0b0 0x7f3e70070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800fdba0 0x7f3e80113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 --2- 172.21.15.63:0/3971335155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e800fd200 0x7f3e80112d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 >> 172.21.15.63:0/3971335155 conn(0x7f3e800f6cb0 msgr2=0x7f3e800f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 shutdown_connections 2024-04-24T22:57:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.846+0000 7f3e86cae640 1 -- 172.21.15.63:0/3971335155 wait complete. 2024-04-24T22:57:58.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:57:58.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:58 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/427059820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T22:57:58.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:58 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/1094204451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T22:57:58.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:58 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/436007837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.955+0000 7f3a5ae21640 1 -- 172.21.15.63:0/1883597325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 msgr2=0x7f3a54102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.955+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/1883597325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54102580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009c50 tx=0x7f3a4c004d10 comp rx=0 tx=0).stop 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.955+0000 7f3a5ae21640 1 -- 172.21.15.63:0/1883597325 shutdown_connections 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.955+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/1883597325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 0x7f3a540fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.955+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/1883597325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54102580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- 172.21.15.63:0/1883597325 >> 172.21.15.63:0/1883597325 conn(0x7f3a540f7dc0 msgr2=0x7f3a540fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- 172.21.15.63:0/1883597325 shutdown_connections 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- 172.21.15.63:0/1883597325 wait complete. 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 Processor -- start 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- start start 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 0x7f3a54113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541138c0 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.956+0000 7f3a5ae21640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5410cec0 con 0x7f3a54102b50 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36592/0 (socket says 172.21.15.63:36592) 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 -- 172.21.15.63:0/4051705497 learned_addr learned my addr 172.21.15.63:0/4051705497 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a53fff640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 0x7f3a54113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 -- 172.21.15.63:0/4051705497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 msgr2=0x7f3a54113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 0x7f3a54113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 -- 172.21.15.63:0/4051705497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0098b0 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a58bbd640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c004a30 tx=0x7f3a4c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c03a070 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a4c035b20 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5410d140 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c03eaa0 con 0x7f3a541021b0 2024-04-24T22:57:58.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.957+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5410d660 con 0x7f3a541021b0 2024-04-24T22:57:58.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.958+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a540fd920 con 0x7f3a541021b0 2024-04-24T22:57:58.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.959+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3a4c032070 con 0x7f3a541021b0 2024-04-24T22:57:58.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.959+0000 7f3a517fa640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 0x7f3a3c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:57:58.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.959+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3a4c0b0320 con 0x7f3a541021b0 2024-04-24T22:57:58.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.959+0000 7f3a53fff640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 0x7f3a3c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:57:58.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.960+0000 7f3a53fff640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 0x7f3a3c070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a48006fd0 tx=0x7f3a48008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:57:58.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:58.964+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3a4c0796d0 con 0x7f3a541021b0 2024-04-24T22:57:59.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.161+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a5404bcc0 con 0x7f3a541021b0 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.161+0000 7f3a517fa640 1 -- 172.21.15.63:0/4051705497 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v83) v1 ==== 81+0+2317 (secure 0 0 0) 0x7f3a4c07cd90 con 0x7f3a541021b0 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6827/2625956780 2024-04-25T22:55:33.427869+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6826/2625956780 2024-04-25T22:55:33.427869+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6825/780449385 2024-04-25T22:55:08.099739+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3274298824 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/622073725 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2614941819 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/201758584 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4263661376 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.164 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2508233848 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3694082065 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2702451571 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2342719260 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/2604587148 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1454404068 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1764753728 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2691264506 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2691264506 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/370487052 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/2143811635 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:0/3666997364 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1700011071 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3585828238 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3056400683 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/776980192 2024-04-25T22:42:37.667991+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/622073725 2024-04-25T22:45:09.797346+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/776980192 2024-04-25T22:42:37.667991+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3239723549 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3265850425 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2572161006 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6828/4284728814 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2164211416 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6829/4284728814 2024-04-25T22:46:56.899597+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3227237455 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1398662955 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/201758584 2024-04-25T22:37:41.558564+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2614941819 2024-04-25T22:35:55.097488+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2293418655 2024-04-25T22:36:26.080101+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3227237455 2024-04-25T22:50:00.020010+0000 2024-04-24T22:57:59.166 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.192:6824/780449385 2024-04-25T22:55:08.099739+0000 2024-04-24T22:57:59.167 INFO:teuthology.orchestra.run.smithi063.stderr:listed 39 entries 2024-04-24T22:57:59.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 msgr2=0x7f3a3c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:59.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 0x7f3a3c070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a48006fd0 tx=0x7f3a48008040 comp rx=0 tx=0).stop 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 msgr2=0x7f3a54112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c004a30 tx=0x7f3a4c004d10 comp rx=0 tx=0).stop 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 shutdown_connections 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:6800/3731828681,v1:172.21.15.63:6801/3731828681] conn(0x7f3a3c06e0b0 0x7f3a3c070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a54102b50 0x7f3a54113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 --2- 172.21.15.63:0/4051705497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a541021b0 0x7f3a54112cf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 >> 172.21.15.63:0/4051705497 conn(0x7f3a540f7dc0 msgr2=0x7f3a540f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 shutdown_connections 2024-04-24T22:57:59.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-24T22:57:59.163+0000 7f3a5ae21640 1 -- 172.21.15.63:0/4051705497 wait complete. 2024-04-24T22:57:59.177 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-04-24T22:57:59.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:57:59.177 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T22:57:59.201 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:57:59.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:59 smithi063 ceph-mon[111213]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:59.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:59 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/3971335155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:59.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:57:59 smithi063 ceph-mon[111213]: from='client.? 172.21.15.63:0/4051705497' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:57:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:59 smithi192 ceph-mon[80463]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:57:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:59 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/3971335155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T22:57:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:57:59 smithi192 ceph-mon[80463]: from='client.? 172.21.15.63:0/4051705497' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T22:58:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:01.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:01 smithi192 ceph-mon[80463]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:02.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:01 smithi063 ceph-mon[111213]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:03 smithi192 ceph-mon[80463]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:03 smithi063 ceph-mon[111213]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:05 smithi192 ceph-mon[80463]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:06.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:05 smithi063 ceph-mon[111213]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:07.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:07 smithi192 ceph-mon[80463]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:07 smithi063 ceph-mon[111213]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:09.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:09 smithi063 ceph-mon[111213]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:09 smithi192 ceph-mon[80463]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:11 smithi192 ceph-mon[80463]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:11 smithi063 ceph-mon[111213]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:13 smithi192 ceph-mon[80463]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:14.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:13 smithi063 ceph-mon[111213]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:15.866 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:15 smithi063 ceph-mon[111213]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:15.866 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:15 smithi192 ceph-mon[80463]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:17 smithi192 ceph-mon[80463]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:18.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:17 smithi063 ceph-mon[111213]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:19.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:19 smithi063 ceph-mon[111213]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:19 smithi192 ceph-mon[80463]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:21 smithi192 ceph-mon[80463]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:21 smithi063 ceph-mon[111213]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:23 smithi192 ceph-mon[80463]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:23 smithi063 ceph-mon[111213]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:25 smithi192 ceph-mon[80463]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:26.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:25 smithi063 ceph-mon[111213]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:27 smithi192 ceph-mon[80463]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:27 smithi063 ceph-mon[111213]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:29.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:29 smithi063 ceph-mon[111213]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:29 smithi192 ceph-mon[80463]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:31 smithi192 ceph-mon[80463]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:31 smithi063 ceph-mon[111213]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:33.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:33 smithi063 ceph-mon[111213]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:33 smithi192 ceph-mon[80463]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:35.929 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:35 smithi192 ceph-mon[80463]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:36.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:35 smithi063 ceph-mon[111213]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:37 smithi192 ceph-mon[80463]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:38.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:37 smithi063 ceph-mon[111213]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:39 smithi192 ceph-mon[80463]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:39 smithi063 ceph-mon[111213]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:41 smithi192 ceph-mon[80463]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:41 smithi063 ceph-mon[111213]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:43 smithi192 ceph-mon[80463]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:44.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:43 smithi063 ceph-mon[111213]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:45 smithi192 ceph-mon[80463]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:45 smithi063 ceph-mon[111213]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:58:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:47 smithi192 ceph-mon[80463]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:48.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:47 smithi063 ceph-mon[111213]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:49 smithi192 ceph-mon[80463]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:50.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:49 smithi063 ceph-mon[111213]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:51 smithi192 ceph-mon[80463]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:51 smithi063 ceph-mon[111213]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:53 smithi192 ceph-mon[80463]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:58:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:53 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:54.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:53 smithi063 ceph-mon[111213]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:58:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:53 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:58:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:58:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:58:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:58:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:58:56.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:55 smithi063 ceph-mon[111213]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:56.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:55 smithi192 ceph-mon[80463]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:58.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:57 smithi063 ceph-mon[111213]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:58:58.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:57 smithi192 ceph-mon[80463]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:00.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:58:59 smithi063 ceph-mon[111213]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:58:59 smithi192 ceph-mon[80463]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:01.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:02.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:01 smithi063 ceph-mon[111213]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:02.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:01 smithi192 ceph-mon[80463]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:03.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:02 smithi063 ceph-mon[111213]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:03.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:02 smithi192 ceph-mon[80463]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:05.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:05 smithi063 ceph-mon[111213]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:05 smithi192 ceph-mon[80463]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:07.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:07 smithi063 ceph-mon[111213]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:07.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:07 smithi192 ceph-mon[80463]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:09.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:09 smithi063 ceph-mon[111213]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:09 smithi192 ceph-mon[80463]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:11.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:11 smithi063 ceph-mon[111213]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:11 smithi192 ceph-mon[80463]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:13.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:13 smithi063 ceph-mon[111213]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:13 smithi192 ceph-mon[80463]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:15 smithi063 ceph-mon[111213]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:15.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:15 smithi192 ceph-mon[80463]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:15.930 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:17.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:17 smithi063 ceph-mon[111213]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:17 smithi192 ceph-mon[80463]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:19 smithi063 ceph-mon[111213]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:19 smithi192 ceph-mon[80463]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:21.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:21 smithi063 ceph-mon[111213]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:21 smithi192 ceph-mon[80463]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:23.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:23 smithi063 ceph-mon[111213]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:23 smithi192 ceph-mon[80463]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:25.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:25 smithi063 ceph-mon[111213]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:25 smithi192 ceph-mon[80463]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:27.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:27 smithi063 ceph-mon[111213]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:27 smithi192 ceph-mon[80463]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:29.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:29 smithi063 ceph-mon[111213]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:29 smithi192 ceph-mon[80463]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:30.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:31.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:31 smithi063 ceph-mon[111213]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:31 smithi192 ceph-mon[80463]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:33.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:33 smithi063 ceph-mon[111213]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:33 smithi192 ceph-mon[80463]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:35.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:35 smithi063 ceph-mon[111213]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:35 smithi192 ceph-mon[80463]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:37.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:37 smithi063 ceph-mon[111213]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:37 smithi192 ceph-mon[80463]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:39.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:39 smithi063 ceph-mon[111213]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:39 smithi192 ceph-mon[80463]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:41 smithi192 ceph-mon[80463]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:41 smithi063 ceph-mon[111213]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:43.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:43 smithi192 ceph-mon[80463]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:44.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:43 smithi063 ceph-mon[111213]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:45 smithi063 ceph-mon[111213]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:45 smithi192 ceph-mon[80463]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T22:59:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:47 smithi192 ceph-mon[80463]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:48.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:47 smithi063 ceph-mon[111213]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:49.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:49 smithi063 ceph-mon[111213]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:49.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:49 smithi192 ceph-mon[80463]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:51.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:51 smithi192 ceph-mon[80463]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:51 smithi063 ceph-mon[111213]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:53.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:53 smithi192 ceph-mon[80463]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:54.054 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:53 smithi063 ceph-mon[111213]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:54.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:54 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:59:55.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:54 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T22:59:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:55 smithi192 ceph-mon[80463]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:59:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:59:55.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:59:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:55 smithi063 ceph-mon[111213]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T22:59:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T22:59:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T22:59:57.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:57 smithi192 ceph-mon[80463]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:57 smithi063 ceph-mon[111213]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:59.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 22:59:59 smithi063 ceph-mon[111213]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T22:59:59.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 22:59:59 smithi192 ceph-mon[80463]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:00 smithi192 ceph-mon[80463]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T23:00:00.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:00 smithi063 ceph-mon[111213]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-24T23:00:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:01.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:01 smithi192 ceph-mon[80463]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:02.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:01 smithi063 ceph-mon[111213]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:03.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:03 smithi192 ceph-mon[80463]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:04.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:03 smithi063 ceph-mon[111213]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:05.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:05 smithi192 ceph-mon[80463]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:06.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:05 smithi063 ceph-mon[111213]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:07.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:07 smithi192 ceph-mon[80463]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:07 smithi063 ceph-mon[111213]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:09.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:09 smithi063 ceph-mon[111213]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:09.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:09 smithi192 ceph-mon[80463]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:11.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:11 smithi192 ceph-mon[80463]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:11 smithi063 ceph-mon[111213]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:13.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:13 smithi192 ceph-mon[80463]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:14.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:13 smithi063 ceph-mon[111213]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:15 smithi192 ceph-mon[80463]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:15.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:16.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:15 smithi063 ceph-mon[111213]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:16.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:17.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:17 smithi192 ceph-mon[80463]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:18.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:17 smithi063 ceph-mon[111213]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:19.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:19 smithi063 ceph-mon[111213]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:19.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:19 smithi192 ceph-mon[80463]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:21.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:21 smithi192 ceph-mon[80463]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:21 smithi063 ceph-mon[111213]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:23.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:23 smithi192 ceph-mon[80463]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:23 smithi063 ceph-mon[111213]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:25.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:25 smithi192 ceph-mon[80463]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:25 smithi063 ceph-mon[111213]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:27.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:27 smithi192 ceph-mon[80463]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:28.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:27 smithi063 ceph-mon[111213]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:29.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:29 smithi192 ceph-mon[80463]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:30.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:29 smithi063 ceph-mon[111213]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:30.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:31.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:31 smithi192 ceph-mon[80463]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:31 smithi063 ceph-mon[111213]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:33.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:33 smithi192 ceph-mon[80463]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:33 smithi063 ceph-mon[111213]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:35.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:35 smithi192 ceph-mon[80463]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:36.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:35 smithi063 ceph-mon[111213]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:37.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:37 smithi192 ceph-mon[80463]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:38.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:37 smithi063 ceph-mon[111213]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:39.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:39 smithi192 ceph-mon[80463]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:39 smithi063 ceph-mon[111213]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:41.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:41 smithi192 ceph-mon[80463]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:41 smithi063 ceph-mon[111213]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:43.922 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:43 smithi192 ceph-mon[80463]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:44.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:43 smithi063 ceph-mon[111213]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:45 smithi192 ceph-mon[80463]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:45.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:45 smithi063 ceph-mon[111213]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:00:47.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:47 smithi192 ceph-mon[80463]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:48.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:47 smithi063 ceph-mon[111213]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:50.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:49 smithi063 ceph-mon[111213]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:49 smithi192 ceph-mon[80463]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:51 smithi063 ceph-mon[111213]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:51 smithi192 ceph-mon[80463]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:53.983 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:53 smithi192 ceph-mon[80463]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:53 smithi063 ceph-mon[111213]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:56.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:55 smithi063 ceph-mon[111213]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:00:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:00:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:00:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:00:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:55 smithi192 ceph-mon[80463]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:00:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:00:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:00:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:00:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:57 smithi063 ceph-mon[111213]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:00:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:57 smithi192 ceph-mon[80463]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:00:59 smithi063 ceph-mon[111213]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:00:59 smithi192 ceph-mon[80463]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:01.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:01.927 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:01 smithi192 ceph-mon[80463]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:02.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:01 smithi063 ceph-mon[111213]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:04.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:03 smithi063 ceph-mon[111213]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:03 smithi192 ceph-mon[80463]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:06.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:05 smithi063 ceph-mon[111213]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:06.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:05 smithi192 ceph-mon[80463]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:08.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:07 smithi063 ceph-mon[111213]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:07 smithi192 ceph-mon[80463]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:10.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:09 smithi063 ceph-mon[111213]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:10.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:09 smithi192 ceph-mon[80463]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:11.979 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:11 smithi192 ceph-mon[80463]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:12.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:11 smithi063 ceph-mon[111213]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:14.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:13 smithi063 ceph-mon[111213]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:14.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:13 smithi192 ceph-mon[80463]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:16.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:15 smithi063 ceph-mon[111213]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:16.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:16.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:15 smithi192 ceph-mon[80463]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:16.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:18.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:17 smithi063 ceph-mon[111213]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:18.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:17 smithi192 ceph-mon[80463]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:20.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:19 smithi063 ceph-mon[111213]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:19 smithi192 ceph-mon[80463]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:22.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:21 smithi063 ceph-mon[111213]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:22.180 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:21 smithi192 ceph-mon[80463]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:24.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:23 smithi063 ceph-mon[111213]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:23 smithi192 ceph-mon[80463]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:26.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:25 smithi063 ceph-mon[111213]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:25 smithi192 ceph-mon[80463]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:28.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:27 smithi063 ceph-mon[111213]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:28.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:27 smithi192 ceph-mon[80463]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:30.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:29 smithi063 ceph-mon[111213]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:29 smithi192 ceph-mon[80463]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:31.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:32.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:31 smithi063 ceph-mon[111213]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:31 smithi192 ceph-mon[80463]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:34.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:33 smithi063 ceph-mon[111213]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:33 smithi192 ceph-mon[80463]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:36.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:35 smithi063 ceph-mon[111213]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:36.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:35 smithi192 ceph-mon[80463]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:38.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:37 smithi063 ceph-mon[111213]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:37 smithi192 ceph-mon[80463]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:40.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:39 smithi063 ceph-mon[111213]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:39 smithi192 ceph-mon[80463]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:42.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:41 smithi063 ceph-mon[111213]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:41 smithi192 ceph-mon[80463]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:44.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:43 smithi063 ceph-mon[111213]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:44.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:43 smithi192 ceph-mon[80463]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:45 smithi063 ceph-mon[111213]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:46.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:45 smithi192 ceph-mon[80463]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:01:48.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:47 smithi063 ceph-mon[111213]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:47 smithi192 ceph-mon[80463]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:50.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:49 smithi063 ceph-mon[111213]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:49 smithi192 ceph-mon[80463]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:52.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:51 smithi063 ceph-mon[111213]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:51 smithi192 ceph-mon[80463]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:54.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:53 smithi063 ceph-mon[111213]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:53 smithi192 ceph-mon[80463]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:56.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:55 smithi063 ceph-mon[111213]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:56.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:55 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:01:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:55 smithi192 ceph-mon[80463]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:55 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:01:57.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:56 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:01:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:56 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:01:57.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:56 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:01:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:56 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:01:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:56 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:01:57.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:56 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:01:58.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:57 smithi063 ceph-mon[111213]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:01:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:57 smithi192 ceph-mon[80463]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:00.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:01:59 smithi063 ceph-mon[111213]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:00.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:01:59 smithi192 ceph-mon[80463]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:01.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:00 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:01.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:00 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:02.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:01 smithi192 ceph-mon[80463]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:02.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:01 smithi063 ceph-mon[111213]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:04.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:03 smithi192 ceph-mon[80463]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:04.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:03 smithi063 ceph-mon[111213]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:06.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:05 smithi192 ceph-mon[80463]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:06.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:05 smithi063 ceph-mon[111213]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:08.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:07 smithi192 ceph-mon[80463]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:08.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:07 smithi063 ceph-mon[111213]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:10.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:09 smithi192 ceph-mon[80463]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:10.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:09 smithi063 ceph-mon[111213]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:12.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:11 smithi192 ceph-mon[80463]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:12.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:11 smithi063 ceph-mon[111213]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:14.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:13 smithi192 ceph-mon[80463]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:14.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:13 smithi063 ceph-mon[111213]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:16.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:15 smithi063 ceph-mon[111213]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:16.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:15 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:16.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:15 smithi192 ceph-mon[80463]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:16.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:15 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:18.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:17 smithi192 ceph-mon[80463]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:18.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:17 smithi063 ceph-mon[111213]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:20.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:19 smithi192 ceph-mon[80463]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:20.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:19 smithi063 ceph-mon[111213]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:22.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:21 smithi192 ceph-mon[80463]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:22.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:21 smithi063 ceph-mon[111213]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:24.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:23 smithi192 ceph-mon[80463]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:24.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:23 smithi063 ceph-mon[111213]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:26.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:25 smithi192 ceph-mon[80463]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:26.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:25 smithi063 ceph-mon[111213]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:28.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:27 smithi192 ceph-mon[80463]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:28.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:27 smithi063 ceph-mon[111213]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:30.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:29 smithi192 ceph-mon[80463]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:30.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:29 smithi063 ceph-mon[111213]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:31.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:30 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:31.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:30 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:32.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:31 smithi192 ceph-mon[80463]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:32.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:31 smithi063 ceph-mon[111213]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:34.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:33 smithi192 ceph-mon[80463]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:34.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:33 smithi063 ceph-mon[111213]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:36.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:35 smithi192 ceph-mon[80463]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:36.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:35 smithi063 ceph-mon[111213]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:38.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:37 smithi192 ceph-mon[80463]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:38.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:37 smithi063 ceph-mon[111213]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:40.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:39 smithi192 ceph-mon[80463]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:40.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:39 smithi063 ceph-mon[111213]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:42.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:41 smithi192 ceph-mon[80463]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:42.305 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:41 smithi063 ceph-mon[111213]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:44.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:43 smithi192 ceph-mon[80463]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:44.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:43 smithi063 ceph-mon[111213]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:45 smithi192 ceph-mon[80463]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:46.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:45 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:46.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:45 smithi063 ceph-mon[111213]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:46.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:45 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:02:48.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:47 smithi192 ceph-mon[80463]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:48.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:47 smithi063 ceph-mon[111213]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:50.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:49 smithi192 ceph-mon[80463]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:50.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:49 smithi063 ceph-mon[111213]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:52.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:51 smithi192 ceph-mon[80463]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:52.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:51 smithi063 ceph-mon[111213]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:54.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:53 smithi192 ceph-mon[80463]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:54.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:53 smithi063 ceph-mon[111213]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:56.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:55 smithi192 ceph-mon[80463]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:56.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:55 smithi063 ceph-mon[111213]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:57.176 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:56 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:57.302 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:56 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:02:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:57 smithi192 ceph-mon[80463]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:57 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:02:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:57 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:02:58.177 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:57 smithi192 ceph-mon[80463]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:02:58.279 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-24T23:02:58.280 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:02:58.282 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:02:58.300 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:02:58.300 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:02:58.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:02:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:57 smithi063 ceph-mon[111213]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 998 MiB used, 535 GiB / 536 GiB avail 2024-04-24T23:02:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:57 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:02:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:57 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:02:58.303 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:57 smithi063 ceph-mon[111213]: from='mgr.34104 172.21.15.63:0/204677641' entity='mgr.smithi063.upcnvd' 2024-04-24T23:02:58.335 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:02:58.367 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:02:58.367 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:02:58.393 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:02:58.426 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:02:58.426 INFO:tasks.cephadm.mon.smithi063:Stopping mon.smithi063... 2024-04-24T23:02:58.426 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063 2024-04-24T23:02:58.743 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:58 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:02:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:58 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[111209]: 2024-04-24T23:02:58.742+0000 7fcca684a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:02:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:58 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063[111209]: 2024-04-24T23:02:58.742+0000 7fcca684a700 -1 mon.smithi063@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:02:59.053 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:58 smithi063 podman[152628]: 2024-04-24 23:02:58.773202319 +0000 UTC m=+0.135832895 container died 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T23:02:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:59 smithi063 podman[152628]: 2024-04-24 23:02:59.351230735 +0000 UTC m=+0.713861310 container cleanup 780ec798c65be922d02008394e17b2412f1619b5a2c666e0e558c736aa152604 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:02:59.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Apr 24 23:02:59 smithi063 bash[152628]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi063 2024-04-24T23:02:59.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi063.service' 2024-04-24T23:02:59.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:02:59.826 INFO:tasks.cephadm.mon.smithi063:Stopped mon.smithi063 2024-04-24T23:02:59.827 INFO:tasks.cephadm.mon.smithi192:Stopping mon.smithi192... 2024-04-24T23:02:59.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi192 2024-04-24T23:03:00.121 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:02:59 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:03:00 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192[80459]: 2024-04-24T23:03:00.119+0000 7f4671c25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:03:00 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192[80459]: 2024-04-24T23:03:00.119+0000 7f4671c25700 -1 mon.smithi192@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:03:00.427 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:03:00 smithi192 podman[113321]: 2024-04-24 23:03:00.150486006 +0000 UTC m=+0.126581474 container died 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T23:03:00.888 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:03:00 smithi192 podman[113321]: 2024-04-24 23:03:00.619746136 +0000 UTC m=+0.595841645 container cleanup 1bfbbd31af1613f4c9dd1c6bdee141d497b1f577e54d35d43729b2dbdb0b3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:03:00.888 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 24 23:03:00 smithi192 bash[113321]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-mon-smithi192 2024-04-24T23:03:01.046 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@mon.smithi192.service' 2024-04-24T23:03:01.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:01.093 INFO:tasks.cephadm.mon.smithi192:Stopped mon.smithi192 2024-04-24T23:03:01.093 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:03:01.094 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:03:01.440 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:01 smithi063 systemd[1]: Stopping Ceph osd.0 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:01.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:01 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T23:03:01.438+0000 7f8e88594700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:01.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:01 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T23:03:01.438+0000 7f8e88594700 -1 osd.0 83 *** Got signal Terminated *** 2024-04-24T23:03:01.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:01 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0[122761]: 2024-04-24T23:03:01.438+0000 7f8e88594700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:07.303 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:06 smithi063 podman[152764]: 2024-04-24 23:03:06.931246013 +0000 UTC m=+5.642798374 container died 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:03:07.708 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:07 smithi063 podman[152764]: 2024-04-24 23:03:07.447454735 +0000 UTC m=+6.159007098 container cleanup 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:03:07.708 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:07 smithi063 bash[152764]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:03:08.053 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:07 smithi063 podman[152783]: 2024-04-24 23:03:07.707337592 +0000 UTC m=+0.769658220 container remove 4e96a1c73c3f29ca7e02855e0c40b63f4d753c8704251681c94014ae9649dcfc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T23:03:08.420 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:07.984897942 +0000 UTC m=+0.019114633 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:08.420 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:08.100032136 +0000 UTC m=+0.134248811 container create c3c7b9ec302f42a9f59edcc0c321f931057e32b78a2712d0de629c06abd946a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:03:08.420 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:08.319559772 +0000 UTC m=+0.353776464 container init c3c7b9ec302f42a9f59edcc0c321f931057e32b78a2712d0de629c06abd946a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-24T23:03:08.420 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:08.323269278 +0000 UTC m=+0.357485955 container start c3c7b9ec302f42a9f59edcc0c321f931057e32b78a2712d0de629c06abd946a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:03:08.420 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:08.419086844 +0000 UTC m=+0.453303527 container attach c3c7b9ec302f42a9f59edcc0c321f931057e32b78a2712d0de629c06abd946a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:08.803 INFO:journalctl@ceph.osd.0.smithi063.stdout:Apr 24 23:03:08 smithi063 podman[152876]: 2024-04-24 23:03:08.622776642 +0000 UTC m=+0.656993319 container died c3c7b9ec302f42a9f59edcc0c321f931057e32b78a2712d0de629c06abd946a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:03:09.160 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:03:09.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:09.202 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:03:09.203 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:03:09.203 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:03:09.427 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:09 smithi063 systemd[1]: Stopping Ceph osd.1 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:09.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:09 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T23:03:09.559+0000 7f8898555700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:09.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:09 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T23:03:09.559+0000 7f8898555700 -1 osd.1 83 *** Got signal Terminated *** 2024-04-24T23:03:09.803 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:09 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1[128568]: 2024-04-24T23:03:09.559+0000 7f8898555700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:15.183 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:14 smithi063 podman[152984]: 2024-04-24 23:03:14.874124469 +0000 UTC m=+5.443481394 container died 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:03:15.528 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:15 smithi063 podman[152984]: 2024-04-24 23:03:15.28411188 +0000 UTC m=+5.853468808 container cleanup 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:03:15.528 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:15 smithi063 bash[152984]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:03:15.802 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:15 smithi063 podman[153015]: 2024-04-24 23:03:15.527271142 +0000 UTC m=+0.652259492 container remove 40c2a390f714f90537ba15b271f211e29afbc26b9f050513570236ac57dc008e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:03:16.248 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:15 smithi063 podman[153110]: 2024-04-24 23:03:15.854531854 +0000 UTC m=+0.018984368 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:16.248 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:15 smithi063 podman[153110]: 2024-04-24 23:03:15.961464785 +0000 UTC m=+0.125917285 container create 2b1275d7f01be7fbba5dfa1a4c9b00b1414cf75c97ec4be5a15656d1e1c7bb1e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:03:16.248 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:16 smithi063 podman[153110]: 2024-04-24 23:03:16.1808195 +0000 UTC m=+0.345272001 container init 2b1275d7f01be7fbba5dfa1a4c9b00b1414cf75c97ec4be5a15656d1e1c7bb1e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T23:03:16.248 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:16 smithi063 podman[153110]: 2024-04-24 23:03:16.184130811 +0000 UTC m=+0.348583312 container start 2b1275d7f01be7fbba5dfa1a4c9b00b1414cf75c97ec4be5a15656d1e1c7bb1e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:03:16.248 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:16 smithi063 podman[153110]: 2024-04-24 23:03:16.247197049 +0000 UTC m=+0.411649564 container attach 2b1275d7f01be7fbba5dfa1a4c9b00b1414cf75c97ec4be5a15656d1e1c7bb1e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:03:16.553 INFO:journalctl@ceph.osd.1.smithi063.stdout:Apr 24 23:03:16 smithi063 podman[153110]: 2024-04-24 23:03:16.491818745 +0000 UTC m=+0.656271246 container died 2b1275d7f01be7fbba5dfa1a4c9b00b1414cf75c97ec4be5a15656d1e1c7bb1e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:03:17.134 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:03:17.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:17.175 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:03:17.175 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:03:17.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:03:17.505 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:17 smithi063 systemd[1]: Stopping Ceph osd.2 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:17.803 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:17 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T23:03:17.503+0000 7f7665197700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:17.803 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:17 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T23:03:17.503+0000 7f7665197700 -1 osd.2 83 *** Got signal Terminated *** 2024-04-24T23:03:17.803 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:17 smithi063 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2[134388]: 2024-04-24T23:03:17.503+0000 7f7665197700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:23.298 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 podman[153215]: 2024-04-24 23:03:23.008419426 +0000 UTC m=+5.607756660 container died 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:23.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 podman[153215]: 2024-04-24 23:03:23.387608606 +0000 UTC m=+5.986945830 container cleanup 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:03:23.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 bash[153215]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:03:23.959 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 podman[153232]: 2024-04-24 23:03:23.655781046 +0000 UTC m=+0.647764583 container remove 53024f97d574adf884ff017c14b4144b623eef10b18bf1670e1a5ad54f0faaa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T23:03:24.234 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 podman[153322]: 2024-04-24 23:03:23.861709608 +0000 UTC m=+0.019014485 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:24.234 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:23 smithi063 podman[153322]: 2024-04-24 23:03:23.981485798 +0000 UTC m=+0.138790659 container create 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:03:24.235 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:24 smithi063 podman[153322]: 2024-04-24 23:03:24.167342327 +0000 UTC m=+0.324647188 container init 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:24.235 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:24 smithi063 podman[153322]: 2024-04-24 23:03:24.17051269 +0000 UTC m=+0.327817554 container start 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-24T23:03:24.235 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:24 smithi063 podman[153322]: 2024-04-24 23:03:24.233692201 +0000 UTC m=+0.390997081 container attach 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:03:24.553 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:24 smithi063 podman[153322]: 2024-04-24 23:03:24.476362833 +0000 UTC m=+0.633667696 container died 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:25.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Apr 24 23:03:25 smithi063 podman[153348]: 2024-04-24 23:03:25.00370863 +0000 UTC m=+0.522230203 container remove 73cd3c4fcadfc2bc4b72e141306149ae38b1e2d16335a7d18fcc5979d2ce8bb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:03:25.023 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:03:25.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:25.065 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:03:25.066 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:03:25.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:03:25.427 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:25 smithi192 systemd[1]: Stopping Ceph osd.3 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:25.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:25 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T23:03:25.663+0000 7ff71a4f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:25.678 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:25 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T23:03:25.663+0000 7ff71a4f4700 -1 osd.3 83 *** Got signal Terminated *** 2024-04-24T23:03:25.678 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:25 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T23:03:25.663+0000 7ff71a4f4700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:29.177 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:28 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T23:03:28.809+0000 7ff7172e9700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.200485+0000 front 2024-04-24T23:03:03.200423+0000 (oldest deadline 2024-04-24T23:03:28.500256+0000) 2024-04-24T23:03:29.642 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:29 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:29.279+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:29.927 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:29 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3[90133]: 2024-04-24T23:03:29.782+0000 7ff7172e9700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.200485+0000 front 2024-04-24T23:03:03.200423+0000 (oldest deadline 2024-04-24T23:03:28.500256+0000) 2024-04-24T23:03:29.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:29 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:29.640+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:30.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:30 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:30.305+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:31.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:30 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:30.684+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:31.433 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:31 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:31.300+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:31.433 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:31 smithi192 podman[113445]: 2024-04-24 23:03:31.178641513 +0000 UTC m=+5.929520858 container died 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:03:31.718 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:31 smithi192 podman[113445]: 2024-04-24 23:03:31.531134222 +0000 UTC m=+6.282013566 container cleanup 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:03:31.718 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:31 smithi192 bash[113445]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:03:32.136 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:31 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:31.716+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:32.137 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:31 smithi192 podman[113474]: 2024-04-24 23:03:31.832491075 +0000 UTC m=+0.649297841 container remove 8d7ee6c982383178fa1a3a031a351e7cb6e7fd20dfb0e8c7410d9d87f040dada (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:03:32.395 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:32 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:32.306+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:32.395 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.03902363 +0000 UTC m=+0.018395705 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:32.395 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.158314022 +0000 UTC m=+0.137686099 container create afecf77f12ee47adcef025083de14899e31a556614f87c76dc3a3003be8f2087 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T23:03:32.395 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.394403025 +0000 UTC m=+0.373775100 container init afecf77f12ee47adcef025083de14899e31a556614f87c76dc3a3003be8f2087 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:03:32.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.397736102 +0000 UTC m=+0.377108177 container start afecf77f12ee47adcef025083de14899e31a556614f87c76dc3a3003be8f2087 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:03:32.677 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.527661416 +0000 UTC m=+0.507033520 container attach afecf77f12ee47adcef025083de14899e31a556614f87c76dc3a3003be8f2087 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:03:33.007 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 24 23:03:32 smithi192 podman[113563]: 2024-04-24 23:03:32.686626858 +0000 UTC m=+0.665998933 container died afecf77f12ee47adcef025083de14899e31a556614f87c76dc3a3003be8f2087 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T23:03:33.007 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:32 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:32.710+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:33.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:03:33.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:33.361 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:03:33.361 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:03:33.361 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:03:33.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:33 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:33.279+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:33.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:33 smithi192 systemd[1]: Stopping Ceph osd.4 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:34.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:33 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:33.715+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:34.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:33 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:33.697+0000 7f40a5038700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:34.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:33 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:33.697+0000 7f40a5038700 -1 osd.4 83 *** Got signal Terminated *** 2024-04-24T23:03:34.177 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:33 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:33.697+0000 7f40a5038700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:34.687 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:34 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:34.321+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:35.178 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:34 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:34.763+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:35.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:35 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:35.285+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:36.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:35 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:35.773+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:36.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:36 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:36.332+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:36.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:36 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:36.332+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:09.467592+0000 front 2024-04-24T23:03:09.467542+0000 (oldest deadline 2024-04-24T23:03:35.867909+0000) 2024-04-24T23:03:37.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:36 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:36.765+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:37.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:37 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:37.333+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:37.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:37 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:37.333+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:09.467592+0000 front 2024-04-24T23:03:09.467542+0000 (oldest deadline 2024-04-24T23:03:35.867909+0000) 2024-04-24T23:03:38.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:37 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:37.808+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:38.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:37 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:37.808+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:38.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:38 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:38.330+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:04.966688+0000 front 2024-04-24T23:03:04.966707+0000 (oldest deadline 2024-04-24T23:03:28.966741+0000) 2024-04-24T23:03:38.677 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:38 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4[95126]: 2024-04-24T23:03:38.330+0000 7f40a1e2d700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:09.467592+0000 front 2024-04-24T23:03:09.467542+0000 (oldest deadline 2024-04-24T23:03:35.867909+0000) 2024-04-24T23:03:39.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:38 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:38.759+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:39.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:38 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:38.759+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:39.470 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:39 smithi192 podman[113666]: 2024-04-24 23:03:39.211437446 +0000 UTC m=+5.631213344 container died 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:03:39.775 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:39 smithi192 podman[113666]: 2024-04-24 23:03:39.559774665 +0000 UTC m=+5.979550563 container cleanup 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:03:39.775 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:39 smithi192 bash[113666]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:03:39.775 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:39 smithi192 podman[113683]: 2024-04-24 23:03:39.719323896 +0000 UTC m=+0.503382729 container remove 713be80c3f24ff56e7bb0e79c9ee1dafb3e8e7083169aa91abfbb030fb6f2ef7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T23:03:40.128 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:39 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:39.773+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:40.128 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:39 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:39.773+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:40.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.030873165 +0000 UTC m=+0.018504914 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:40.428 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.128748027 +0000 UTC m=+0.116379806 container create a9e942b93cfe696ef6482d7e0c3e61175b5a4f23f0a791724d9623cf539faeaa (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T23:03:40.428 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.373015951 +0000 UTC m=+0.360647687 container init a9e942b93cfe696ef6482d7e0c3e61175b5a4f23f0a791724d9623cf539faeaa (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:03:40.428 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.376045596 +0000 UTC m=+0.363677332 container start a9e942b93cfe696ef6482d7e0c3e61175b5a4f23f0a791724d9623cf539faeaa (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:03:40.764 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.46448551 +0000 UTC m=+0.452117245 container attach a9e942b93cfe696ef6482d7e0c3e61175b5a4f23f0a791724d9623cf539faeaa (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T23:03:40.764 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 24 23:03:40 smithi192 podman[113772]: 2024-04-24 23:03:40.668323471 +0000 UTC m=+0.655955214 container died a9e942b93cfe696ef6482d7e0c3e61175b5a4f23f0a791724d9623cf539faeaa (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:03:40.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:40 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:40.763+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:40.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:40 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:40.763+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:41.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:03:41.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:41.228 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:03:41.228 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:03:41.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:03:41.427 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 systemd[1]: Stopping Ceph osd.5 for c8a2676e-028a-11ef-bc93-c7b262605968... 2024-04-24T23:03:41.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:41.537+0000 7f451b83b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:41.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:41.537+0000 7f451b83b700 -1 osd.5 83 *** Got signal Terminated *** 2024-04-24T23:03:41.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:41.537+0000 7f451b83b700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:03:41.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:41.783+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:41.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:41 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:41.783+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:43.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:42 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:42.815+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:43.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:42 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:42.815+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:44.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:43 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:43.837+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:44.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:43 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:43.837+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:45.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:44 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:44.796+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:45.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:44 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:44.796+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:46.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:45.754+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-04-24T23:03:03.936172+0000 front 2024-04-24T23:03:03.936219+0000 (oldest deadline 2024-04-24T23:03:29.235977+0000) 2024-04-24T23:03:46.177 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:45 smithi192 ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5[99978]: 2024-04-24T23:03:45.754+0000 7f4514628700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-04-24T23:03:11.536908+0000 front 2024-04-24T23:03:11.536965+0000 (oldest deadline 2024-04-24T23:03:36.836755+0000) 2024-04-24T23:03:47.279 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:47 smithi192 podman[113879]: 2024-04-24 23:03:47.012717834 +0000 UTC m=+5.569550751 container died b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:47.664 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:47 smithi192 podman[113879]: 2024-04-24 23:03:47.36209552 +0000 UTC m=+5.918928439 container cleanup b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:03:47.664 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:47 smithi192 bash[113879]: ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:03:47.927 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:47 smithi192 podman[113898]: 2024-04-24 23:03:47.663289033 +0000 UTC m=+0.647440963 container remove b7243340f774ec10a583f1b606f1a8f379e16245780b7a53016caa6eb7e54f83 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:03:48.325 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:47.928313304 +0000 UTC m=+0.018125770 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:03:48.325 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:48.039229398 +0000 UTC m=+0.129041895 container create 2c84671278dd0c6e2fa5476f3dec9cad3b23fcf0ad2e149954856f51fb61f513 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T23:03:48.326 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:48.249826968 +0000 UTC m=+0.339639419 container init 2c84671278dd0c6e2fa5476f3dec9cad3b23fcf0ad2e149954856f51fb61f513 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:03:48.326 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:48.252942487 +0000 UTC m=+0.342754947 container start 2c84671278dd0c6e2fa5476f3dec9cad3b23fcf0ad2e149954856f51fb61f513 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:03:48.326 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:48.324756375 +0000 UTC m=+0.414568842 container attach 2c84671278dd0c6e2fa5476f3dec9cad3b23fcf0ad2e149954856f51fb61f513 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:03:48.677 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 24 23:03:48 smithi192 podman[113987]: 2024-04-24 23:03:48.538784535 +0000 UTC m=+0.628596986 container died 2c84671278dd0c6e2fa5476f3dec9cad3b23fcf0ad2e149954856f51fb61f513 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-c8a2676e-028a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:03:49.166 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8a2676e-028a-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:03:49.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:03:49.208 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:03:49.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8a2676e-028a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:03:52.650 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[74317]: fuse finished with error 0 and tester_r 0 2024-04-24T23:04:12.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8a2676e-028a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:04:27.409 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:27.444 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:27.470 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:04:27.472 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi063/crash 2024-04-24T23:04:27.473 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:04:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:04:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:04:27.512 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi192/crash 2024-04-24T23:04:27.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:04:27.536 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:04:27.536 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:04:27.537 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:04:27.537 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-24T23:04:27.591 INFO:teuthology.orchestra.run.smithi063.stdout:1713998816.7535868 mon.smithi063 (mon.0) 923 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:04:27.592 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-24T23:04:27.592 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:04:27.592 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:04:27.635 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:04:27.658 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr: /var/log/ceph/ceph-client.1.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi192.ctmwwc.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi192.ctmwwc.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:04:27.659 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:04:27.663 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi192.log 2024-04-24T23:04:27.663 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.ozemqi.log 2024-04-24T23:04:27.663 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi192.log: 87.9% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:04:27.663 INFO:teuthology.orchestra.run.smithi192.stderr: 91.2% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:04:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-04-24T23:04:27.667 INFO:teuthology.orchestra.run.smithi063.stderr: -- /var/log/ceph/ceph-client.0.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi063.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi063.upcnvd.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi063.log: --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi063.upcnvd.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.dnliiv.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.0.log: 87.8% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.dnliiv.log: 91.0% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:04:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:04:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.twfoxu.log 2024-04-24T23:04:27.670 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log: 94.1% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.dnliiv.log.gz 2024-04-24T23:04:27.670 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:04:27.674 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:04:27.674 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.ozemqi.log: gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.mnfuxk.log 2024-04-24T23:04:27.675 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.3% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:04:27.676 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:04:27.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.mnfuxk.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:04:27.692 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.twfoxu.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:04:27.710 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:04:27.717 INFO:teuthology.orchestra.run.smithi063.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:04:27.729 INFO:teuthology.orchestra.run.smithi192.stderr: 89.8% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi192.ctmwwc.log.gz 2024-04-24T23:04:27.730 INFO:teuthology.orchestra.run.smithi192.stderr: 91.1% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:04:27.972 INFO:teuthology.orchestra.run.smithi192.stderr: 92.6% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi192.log.gz 2024-04-24T23:04:28.081 INFO:teuthology.orchestra.run.smithi063.stderr: 88.7% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mgr.smithi063.upcnvd.log.gz 2024-04-24T23:04:28.945 INFO:teuthology.orchestra.run.smithi063.stderr: 90.4% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mon.smithi063.log.gz 2024-04-24T23:04:31.516 INFO:teuthology.orchestra.run.smithi063.stderr: 93.7% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:04:32.019 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:04:32.579 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T23:04:33.498 INFO:teuthology.orchestra.run.smithi063.stderr: 93.7% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:04:33.950 INFO:teuthology.orchestra.run.smithi063.stderr: 94.8% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi063.twfoxu.log.gz 2024-04-24T23:04:34.019 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:04:34.201 INFO:teuthology.orchestra.run.smithi192.stderr: 94.8% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.ozemqi.log.gz 2024-04-24T23:04:34.774 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:04:41.865 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-24T23:04:41.865 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-24T23:04:43.602 INFO:teuthology.orchestra.run.smithi063.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T23:04:43.604 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-24T23:04:43.604 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m15.956s 2024-04-24T23:04:43.604 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m36.809s 2024-04-24T23:04:43.604 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m1.189s 2024-04-24T23:05:55.195 INFO:teuthology.orchestra.run.smithi192.stderr: 92.6% -- replaced with /var/log/ceph/c8a2676e-028a-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi192.mnfuxk.log.gz 2024-04-24T23:05:55.197 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-04-24T23:05:55.197 INFO:teuthology.orchestra.run.smithi192.stderr:real 1m27.548s 2024-04-24T23:05:55.197 INFO:teuthology.orchestra.run.smithi192.stderr:user 2m2.986s 2024-04-24T23:05:55.197 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m3.853s 2024-04-24T23:05:55.197 INFO:tasks.cephadm:Archiving logs... 2024-04-24T23:05:55.198 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi063/log 2024-04-24T23:05:55.199 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:06:00.209 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi192/log 2024-04-24T23:06:00.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:06:18.623 INFO:tasks.cephadm:Removing cluster... 2024-04-24T23:06:18.623 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8a2676e-028a-11ef-bc93-c7b262605968 --force 2024-04-24T23:06:19.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8a2676e-028a-11ef-bc93-c7b262605968 --force 2024-04-24T23:06:19.793 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T23:06:19.793 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:06:19.811 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:06:19.826 INFO:tasks.cephadm:Teardown complete 2024-04-24T23:06:19.827 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:06:19.842 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:06:19.843 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:06:19.843 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:06:19.854 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:06:19.963 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:06:19.974 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:06:19.974 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:06:19.977 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:06:19.990 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:06:19.990 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:^- hv01.front.sepia.ceph.com 2 6 377 6 +1294us[+1296us] +/- 582ms 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 +33us[ +34us] +/- 80ms 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv03.front.sepia.ceph.com 3 6 377 6 -33us[ -32us] +/- 72ms 2024-04-24T23:06:19.994 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:06:19.995 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:06:19.995 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-04-24T23:06:19.995 INFO:teuthology.orchestra.run.smithi063.stdout:^- hv01.front.sepia.ceph.com 2 8 377 202 +1323us[+1350us] +/- 580ms 2024-04-24T23:06:19.995 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +43us[ +54us] +/- 80ms 2024-04-24T23:06:19.996 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv03.front.sepia.ceph.com 3 6 377 8 -34us[ -24us] +/- 72ms 2024-04-24T23:06:19.996 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:06:19.996 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:06:20.017 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:06:20.018 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:06:20.075 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:06:20.160 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:06:20.231 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:06:20.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:06:20.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:06:20.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:06:20.287 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:06:20.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:06:20.350 INFO:teuthology.task.internal:Duration was 2298.884231 seconds 2024-04-24T23:06:20.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:06:20.396 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:06:20.396 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:06:20.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:06:20.435 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:06:20.435 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:06:20.592 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:06:20.592 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-04-24T23:06:20.592 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:06:20.653 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-04-24T23:06:20.653 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:06:20.683 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:06:20.683 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:06:20.695 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:06:20.748 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:06:20.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:06:20.761 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:06:21.412 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:06:21.426 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:06:21.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:06:21.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:06:21.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:06:21.489 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:06:21.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:06:21.514 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-04-24T23:06:21.538 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-04-24T23:06:21.562 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:06:21.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:06:21.585 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:06:21.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:06:21.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:06:21.629 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:06:21.631 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi063 2024-04-24T23:06:21.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:06:21.686 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136/remote/smithi192 2024-04-24T23:06:21.686 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:06:21.727 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:06:21.728 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:06:21.730 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:06:21.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:06:21.790 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:06:21.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:06:21.807 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:06:21.808 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:06:21.810 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:06:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 24 23:06 /home/ubuntu/cephtest 2024-04-24T23:06:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: 550940 4 d--------- 2 ubuntu ubuntu 4096 Apr 24 22:42 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:06:21.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:06:21.823 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-24T23:06:21.824 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-24T23:06:21.824 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi063 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-24T23:06:21.825 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:06:21.853 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:06:21.865 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:06:21.865 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T23:06:21.901 DEBUG:teuthology.nuke:shortname: smithi063 2024-04-24T23:06:21.901 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:06:21.942 DEBUG:teuthology.nuke:shortname: smithi192 2024-04-24T23:06:21.942 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:06:21.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:22:04.938762', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnWeREf0wZA9HoL8TVtsgt25Y0k/NF9+50M4c/BKXpkUAQFt27MIJ111hT9PXCx1cxjrtEAB0JlSQg4eIGyVsQ='} 2024-04-24T23:06:21.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:22:04.940802', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB1OAOWMcVcm35UHsST8EUiB/4wum1Or/Nk1DshIz+NJrdH9b2dk8WO7ITaQ7z0jeMK7k2jrgY09ntzMcEWHVhU='} 2024-04-24T23:06:22.049 INFO:teuthology.orchestra.console.smithi063:Power off 2024-04-24T23:06:22.049 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:06:22.084 INFO:teuthology.orchestra.console.smithi192:Power off 2024-04-24T23:06:22.085 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:06:22.105 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off 2024-04-24T23:06:22.106 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:22.134 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-04-24T23:06:22.135 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:22.145 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-04-24T23:06:22.189 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-04-24T23:06:26.147 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:26.190 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:26.257 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-04-24T23:06:26.300 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-04-24T23:06:30.262 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:30.301 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:06:30.372 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-04-24T23:06:30.411 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-04-24T23:06:30.474 INFO:teuthology.orchestra.console.smithi063:Power off completed 2024-04-24T23:06:30.512 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-04-24T23:06:30.617 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2298.884230852127 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-24T23:06:30.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:06:30.715 INFO:teuthology.run:FAIL